mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
a3d43b7bfdceada2ff487a9957577486aa9ab413
144
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
12f70d0d4c |
Keep Slack form custody and receipt transfer durable (#19825)
## Thinking Path > - Paperclip manages work for AI-agent companies. > - Chat connections deliver provider messages and user answers. > - Slack requires a response within its original three-second deadline. > - Forms and their delivery receipts must survive retries and tenant restarts. > - This change uses the existing connector queue and durable form owners. > - Current authorization, exact receipt checks and normal ACK rules stay in force. > - The tenant can settle useful work while keeping the connection enabled. ## Linked Issues or Issue Description Refs #19019 and #19348. Companion Cloud PR: paperclipai/paperclip-cloud#1250. ## What Changed - Preserve form grants, sessions, encrypted evidence, terminal receipts and company-authorized Retry. - Check initial form enrollment before the provider send. Keep a typed preparation wait only while an exact current enrollment obligation exists. - Refund the attempt only when the claimed attempt and credential lease still match and provider invocation has not started. Possible or accepted sends retain uncertain-delivery custody. An ordinary error with the same text still uses the normal five-attempt limit. - Notify the existing publication worker when normal enrollment commits. Reconstruct pending work from durable state after a lost reply or owner restart. - Page settled history before proving idle safety. Keep unfinished, malformed and foreign receipts visible beyond the first 1,000 rows. - Keep Slack callback, answer, receipt and ACK rules in their provider owners. Support body-bound Cloud idle capabilities when a complete catalog exceeds the header budget. - Add focused regression coverage, an enabled quiet/reactivation recipe and explicit fixture port checks. - Keep SDK evidence tests tied to a normally built reviewed reference, with a separate strict refusal for actual unreviewed CI outputs. ## Verification App head: `6284d6881b5dc38a9ee149bcbde35e69a67dd8ca`, including actual master `dff8bed007e01c32f3826144b893ec80603d1335`. Companion Cloud head: `f83b38187dc7e6b2e219fe865dc16d268cc66fdb`, including actual Cloud master `b1dcc3421a91efffee631a61536cf48f357c22fc`. Cloud #1250 is merged at `1d07d81f23d4627ab7b6dcf49176b4128a2f8f9b` after exact-head passing checks and complete Greptile 5/5. Current exact-head CI typecheck, build, tests and aggregate verification are green; the complete edited Greptile review is 5/5 and all 13 review threads and nested comment pages are resolved. - Both latest enrollment findings were reproduced against the preceding product. The pure readiness control returned no wait for the first enrollment. The real-PostgreSQL control observed a provider call while that enrollment was still staged. - The repaired source passes three whole files: 21 readiness controls, 80 real-PostgreSQL milestone controls and 37 tenant-actor middleware controls. All 138 pass with zero excluded or TODO cases. The SQL case reconstructs the worker through seven waits, preserves the attempt budget, recovers a controlled lost enrollment reply, and commits one provider-port call and one exact grant. Replaying the selected attempt does not duplicate them. Authorization, SDK and enrollment-response ports are controlled in that SQL case. - The same files cover ordinary five-attempt exhaustion, accepted and possible sends, stale ownership, malformed bindings, attention states and current actor restrictions. One synthetic mismatched-binding parameter uses generation 2 against current generation 1; it is not evidence for an older-generation control. - The normal server-package typecheck passes at the measured source. It runs the normal Runner TypeScript, protocol, SDK and declaration prerequisites and all seven main App programs. The separate declaration configuration is also recorded. A worker union-narrowing error was found, fixed with an explicit callable/null branch, and verified by that normal compiler. No cast or test relaxation was used. - The normal merge of current master preserves all 78 owned paths and 15,783 unrelated master entries and modes. The three measured test files and their changed feature paths are unchanged. New upstream DB code adds only the normal nonunique concurrent native-session index and its migration classification. Shared and adapter-utils changes alter unrelated prompt or denial wording. The 138-case proof carries for those unchanged behaviors; it is not reported as a rerun after the merge. The normal Shared/SDK ensure, DB build, adapter-utils build and complete server-package check also pass at the current merged head. Their fresh inventory contains 8,143 outputs; canonical SQL and journal copies match. This build did not apply the new index migration. - Earlier real-PostgreSQL retained-history coverage reproduced the missing proof with 1,005 settled rows. Its repair retains the existing two-second/64-query inspection budget, current endpoint bound and exact owner snapshot. The 46 source-bound controls cover unfinished and invalid tails, exact cross-page action references, Retry, rollback and lost receipt responses. - Earlier enabled quiet/reactivation coverage produced ordinary useful work, kept configuration enabled, accepted fresh signed work plus a duplicate, and produced exactly one second useful reply. Its two App-worker windows lasted at least 30 seconds with zero measured PG writes, connection or outbound activity after settlement. That evidence stays tied to its measured revision. A missing worker EXIT marker limits that earlier proof; parent evidence does not replace it. - Earlier three-second callback, terminal ACK, current-question race and complete body-size controls remain source-bound. The 500-owner/1,000-binding body measured 313,875 bytes with a 618-byte JWT; signature tampering failed closed. - Final managed-provider acceptance now passes all five fresh Node boots: A/A, A/AB, B/A rejection, B/AB recovery, B/B final. Two independently configured, challenged, verified, linked and enrolled Slack identities exercise ACK and retirement debt separately. Normal provider factory/materialization delivers the configuration through a controlled external stack-provider boundary. The registered opening returns HTTP 200 in 21 ms within the original 3-second deadline. Two answers persist, normal ACK is accepted, exact retirement and lost acceptance/GC replies recover, and the original canonical snapshot remains unchanged without duplicate provider sends or submissions. - The separate complete committed-preflight case passes. A committed `update_attempted` response is lost before SDK entry; normal service-owner reconstruction after the original lease/deadline recovers the same useful update and one answer without a second trigger. This reconstructs service owners within one App process; the separate five fresh Node boots cover process restart and managed signing-key rotation. - Assertion-time PG socket reports balance 15 opens/15 closes across the five boots and 5/5 in preflight, peak 2 and zero live sockets or unexpected attempts. Worker terminal EXIT markers were missing and are not replaced by parent EXIT markers. These are domain/recovery and assertion-time connection proofs, not a new whole-worker terminal quiet measurement. All known owners/groups closed; independent SQL audits were empty; only successful fixture databases were removed. Earlier failed fixture databases/logs remain retained. No product guard, deadline or assertion was weakened. Exact runtime source and normal outputs are unchanged by the three test files in the current follow-up, so these source-bound proofs carry without repeating those journeys. - The preceding App CI ran 11,312 support cases: 11,289 passed, three SDK-evidence cases failed, and 20 external cases skipped. Normal CI dependency setup resolved six stale Slack patch-hash references and produced an unreviewed lock. The public SDK proof correctly refused that lock; the tests incorrectly assumed the CI output was a reviewed reference. The three-file test repair keeps the protected lock policy, source/compiler/parser/Zod pins, scanner and redaction rules unchanged. It builds an owned reviewed reference using the existing normal Shared/SDK producers for the positive seal test, while a separate negative requires actual unreviewed-build privacy refusal and no retained SDK bytes. It does not relabel the CI build as certified or admit a new lock hash. - All four whole SDK support files pass locally: public type 30, evidence 16, public-input synchronizer 8 and lock preparation 10, totaling 64 with zero excluded/TODO cases in 28.73 seconds. The genuine fresh reference build/positive seal and separate invalidated-reference privacy refusal passed within their original 120-second bounds. Original installed dependencies, SDK outputs/cache and runtime source stayed unchanged; normal global setup produced only its own Runner JavaScript outputs. PostgreSQL was denied throughout, all four worker markers were observed, and the owned process group/private test root closed. The local canonical-lock negative mutates only its own reference; fresh remote CI also passes the actual resolved-CI-lock branch: all 16 evidence and 30 public-type tests pass, including the genuine reviewed-reference positive and the strict actual-unreviewed-build refusal. The upstream synchronizer file runs its current 10 cases in CI. - The first complete CI attempt on this head also exposed two separate 30-second timeouts: one real reporter-artifact test and the `agent-test-environment` module-loading hook. Full original logs remain preserved. Their exact test, sanitizer and evidence source blobs are unchanged from the preceding head; inspection found no deterministic setup cause. After the original run finished, one retry of only the failed jobs passed on the same synthetic merge `168851e210671ebe97e7a778e8cf891807e80502`: Runner support reports 11,304 passes and 20 external skips; serialized shard 6 reports all 395 passes, including all 67 route bodies. No source, timeout or assertion changed. This proves recovery; the original causes remain unexplained. - The final pre-merge tree audit uses actual master `17136a6c25ef6cb7c816a29140a4c48c2c75fb31` (refreshed after the earlier `85dec046` audit). The expected normal squash preserves all 81 owned paths and 15,816 unrelated master entries and modes. Only OpenAPI has edits from both sides, and its complete content composition preserves the new project route and token help alongside this PR's API entries. The shared inbound selection changes preserve the same candidate set; new concurrent indexes, Native/workspace behavior and shutdown changes remain upstream behavior. The scoped form proof uses process agents and does not qualify the new Native/sandbox branches. The latest question-schema change still accepts the explicit version-1 payload exercised here. The last master delta also changes managed AI/Native credential refresh and writeback; those branches are outside the measured process-agent form paths. Existing 8,143 local outputs remain tied to their measured revision; no new whole-tree output equality or deployed suspension is claimed. - Reproduce the public path with `doc/testing/slack-cloud-qualification.md`, the companion Cloud checkout, Node 24/22 and fresh owned fixture databases. Aggregate local `pnpm -r typecheck`, `pnpm test:run` and `pnpm build` were not rerun as one command set. Current exact-head required CI and the complete Greptile 5/5 review now pass; the full edited summaries and all paginated threads were read. Local worker quiet does not prove deployed host or Neon suspension. Photon and Discord remain separate work; this PR completes the described Slack slice. ## Risks Use compatible App and Cloud readers before transferring custody. Signatures prove authenticity; current authorization and exact ownership still control execution. Preserve old targets, secrets and terminal facts until their normal owners release them. Keep the managed seal identity stable while custody remains. The required Slack adapter patch remains pinned. Upstream's concurrent index uses the normal top-level migration path. ## Model Used OpenAI `gpt-6.1-sol` implemented and tested the changes. A GPT-6-based Codex managing agent reviewed and integrated them. The host does not expose the managing agent's exact model ID or context-window sizes. The agents used code execution and test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, its version and tool capabilities - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public related issues or PRs - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run the local tests listed above and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green for the current head - [x] Greptile is 5/5 with no open findings for the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
99a6daf5bc |
perf(auth): resolve a Cloud tenant actor in one database round trip (#20390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On Paperclip Cloud, each request resolves its board actor from trusted Cloud headers in `actorMiddleware` > - This runs for every request after the liveness probe, also for static files such as `favicon.ico` and `index.html` > - The resolution ran three database statements one after another: the stale `instance_admin` purge, the membership read and the owner-elevation read > - On a loaded database, each statement waits. Three sequential waits make every request slow, also a request that needs no data > - This pull request keeps the purge and the tenant sync in order and runs the two reads at the same time as them > - The benefit is one database wait per request instead of three ## Linked Issues or Issue Description **What existing behavior does this improve?** Cloud tenant actor resolution in `server/src/middleware/auth.ts` (`resolveCloudTenantActorOnce`). **Subsystem affected** Server authentication middleware (Cloud trusted-header path). **Current behavior** Measured on the hosted staging instance on 2026-11-09 from a signed-in browser, 45 paired samples in 40 s: - `/api/liveness` (mounted before actor resolution): client median 92 ms, p90 255 ms, server time ≤ 1 ms. - `/favicon.ico` with a session (after actor resolution): client median 413 ms, p90 1,751 ms, max 4.6 s. - `/api/cli-auth/me`: server median 346 ms, p90 1,528 ms. The same Node process answers liveness quickly, so the event loop is not blocked. The database is busy (6–8 of 8 vCPU for the whole day), and trivial primary-key reads waited 100–280 ms each. Three sequential reads explain the ~320 ms median gap between liveness and favicon. **Proposed behavior** The purge and the debounced tenant sync stay ordered. The membership read and the owner-elevation read run at the same time as them. The sync writes only the primary company's membership, and the membership read excludes that company, so the result does not change. **Reason and benefit** A synced request costs one database wait instead of three. With 150 ms of injected database latency, a synced resolution goes from 914 ms to 310 ms (3 runs each, same machine). **Breaking changes** None. ## What Changed - `resolveCloudTenantActorOnce` starts the purge-and-sync, the additional-membership read and `resolveOwnerInstanceAdmin` together and awaits them with `Promise.all`. A membership read failure still degrades to the primary company. The owner-elevation read still fails closed. - New regression test `server/src/__tests__/cloud-tenant-actor-round-trips.test.ts`. It connects through a TCP proxy that delays every client write by 150 ms and asserts that a synced resolution finishes in less than 4 × the delay. Before the change it takes about 6 × (914 ms); after the change about 2 × (310 ms). - `auth-session-route.test.ts`: the fake database binds each `returning()` to its own insert, and the test matches inserts by content instead of position, because the instance-settings insert can now happen in parallel with the tenant sync. ## Verification - `cd server && npx vitest run src/__tests__/cloud-tenant-actor-round-trips.test.ts` — red on `origin/master` (914 ms, 913 ms, 916 ms), green with the change (310 ms). - `cd server && npx vitest run src/__tests__/cloud-tenant-company-provisioning.test.ts src/__tests__/auth-session-route.test.ts src/__tests__/liveness.test.ts src/__tests__/cloud-warm-standby.test.ts src/middleware/cloud-tenant-actor.test.ts` - `pnpm verify` (selected tests), and CI. - After deploy: compare `/favicon.ico` and `/api/cli-auth/me` client time with `/api/liveness` on staging. Expect the median gap to drop from about 3 database waits to about 1. ## Risks - Low risk. Authorization does not change: the same statements run and the request waits for all of them before it continues. - Each request now uses up to three pooled connections at the same time instead of one. The staging pool limit is 500, so this is not a concern there. Smaller pools reach their limit sooner under a burst. - The diff re-indents the purge-and-sync block. The statements in it do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, browser, code execution). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
7a93cc6215 |
fix(server): redact invitation tokens from agent run logs (#20360)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People join a company through an invitation link. The link carries a bearer token in its URL path. > - An agent can create an invitation in chat and give the person the join link. > - The server writes each agent run to a run log file (`.ndjson`). This file is a plain diagnostic copy. Local agents run as the same OS user, so they can read it. > - #20343 removed invitation tokens from HTTP logs. The run log still kept the full join link and the raw token. > - This pull request removes invitation tokens from run-log chunks when the server writes them. > - The benefit is that a live join link does not stay in a file that other local agents can read. The person still gets the working link in the chat reply. ## Linked Issues or Issue Description Refs #20343 (same token policy for HTTP logs). **What happened?** An agent created an invitation in chat and replied with a `/invite/<token>` join link. The agent run log file under `data/run-logs/` kept the full token. The token was in the agent message and in the tool result. **Expected behavior** The run log keeps the route and drops the token: `/invite/***REDACTED***`. A raw `pcp_invite_...` token also becomes `***REDACTED***`. The chat reply that the person reads keeps the working join link. **Steps to reproduce** 1. Check out master. 2. Run `server/src/services/heartbeat/run-log.test.ts` from this PR. 3. The "run-log invitation token privacy" tests fail. The token stays in both the legacy text chunk and the canonical native envelope. **Paperclip version or commit** Master at the merge base of this branch. **Deployment mode** Source server, local run logs (`local_file`). The same functions also format object-store run logs. ## What Changed - Add `redactInvitationTokens()` in `server/src/middleware/invitation-token-redaction.ts`, next to the HTTP log redactor from #20343. It removes the token from `/invite/<token>` segments and from raw `pcp_invite_` tokens inside free text and JSON text. The character classes stop at JSON, Markdown and URL delimiters, so the record structure stays valid. `redact-sensitive.ts` stays byte-identical, because the chat eval pins its exact bytes before it accepts a real invitation link. - Apply it in `compactRunLogChunk()` and `redactCanonicalNativeRunLogChunk()`. These are the two functions that format every agent run-log chunk before the store writes it. The run excerpt columns use the same chunk, so they are also redacted. - `canonicalNativeRunLogRedactionForms()` also yields the earlier, unredacted projection. A run log written by an earlier release still verifies after an upgrade. - `restoreCompactedEvent()` accepts a truncated record that the earlier compactor wrote with the token. Cold recovery of such an old log still works. - Add a test that proves the chat reply comes from database run events, not from the run log. The reply keeps the full join link, and the presentation path never reads the run log. ## Verification - From `server`: `node ../node_modules/vitest/vitest.mjs run --config vitest.config.ts src/services/heartbeat/run-log.test.ts src/__tests__/invitation-token-redaction.test.ts src/__tests__/redact-sensitive.test.ts src/__tests__/heartbeat-run-log.test.ts src/__tests__/logger-invitation-redaction.test.ts src/services/native-runtime/native-run-log-watermark.test.ts src/services/native-runtime/native-run-log-recovery.test.ts src/__tests__/heartbeat-run-output.test.ts src/services/native-runtime/agent-chat-response-recovery.test.ts` passes. - Before the `run-log.ts` change, the two new redaction tests in `run-log.test.ts` failed (red), then passed (green). - `native-run-log-watermark.test.ts`: a legacy-compacted record with a token in its retained head cold-recovers. It failed with `native_log_source_coverage_unproven` before the restore fix. - `agent-chat-response-recovery.test.ts` (embedded Postgres): a final reply with a join link is saved as the exact comment with the token. No run-log read happens. - `tsc --noEmit -p server` shows no errors in the changed files. - A live chat eval (`access.invite-teammate`) run with this change confirms the run-log file has no token. The run is linked in a comment. ## Risks - Low risk. Run-log readers (run transcript view, feedback export, support inspection) now show `***REDACTED***` in place of invitation tokens. This removes a live credential from a diagnostic copy. It does not change the chat reply or the stored run events. - A path segment after `/invite/` in free text is always redacted, even if it is not a token. This loses only diagnostic detail. - A token that a provider streams across two separate chunks is not joined first. Each chunk is redacted on its own. A complete token in one chunk, such as a final message or a tool result, is removed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
a67b936885 |
fix(server): redact invitation bearer paths from HTTP logs (#20343)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People join a company through an invitation link. The link carries a bearer token in its URL path. > - The HTTP logger removes query secrets and URL user information. It keeps the path. > - So invitation bearer tokens reach the HTTP log in the request URL, the log message, and the referrer header. > - PR #17848 fixed this, but it did not merge. Since then, #13742 changed the same redactor file. > - This pull request carries the #17848 commits onto current master. The original author is kept on both commits. > - The benefit is that invitation tokens do not stay in HTTP logs. The chat eval for membership management can then accept a real invitation safely. ## Linked Issues or Issue Description Supersedes #17848 (same commits, rebased onto current master after #13742). **What happened?** The HTTP logger writes invitation bearer tokens to logs. A synthetic POST to `/api/invites/<token>/accept` puts the token in `req.url`, in the log message, and in the `Referer`/`Referrer` header record. **Expected behavior** Logs keep the invitation route, the method, and the HTTP status. Logs do not keep the bearer token. **Steps to reproduce** 1. Check out master. 2. Run `server/src/__tests__/logger-invitation-redaction.test.ts` from this PR. 3. The synthetic token appears in the request URL, the message, and the referrer. 12 tests fail on master. **Paperclip version or commit** Master at the merge base of this branch. **Deployment mode** Source server. The tests use a disposable local HTTP listener. They use no database and no provider. ## What Changed - Remove the bearer segment from `/api/invites/<token>/...`, `/invites/<token>/...`, and `/invite/<token>` URLs, including mounted paths and absolute URLs. - Apply the same URL policy to `referer` and `referrer` request headers. - Keep record IDs on the non-token revoke route `/invites/<inviteId>/revoke` and on company-scoped invite routes. - Add regression tests through the real HTTP logger for 202, 400 and 500 responses, with both referrer header spellings. - Original work by @cryppadotta in #17848. This PR only rebases it onto master. ## Verification - From `server`: `node ../node_modules/vitest/vitest.mjs run --config vitest.config.ts src/__tests__/logger-invitation-redaction.test.ts src/__tests__/redact-sensitive.test.ts src/__tests__/http-log-policy.test.ts` — 46 passed. - With the master `logger.ts` and `redact-sensitive.ts`, the same new tests fail: 12 failed. - `http-log-aborted-response`, `cloud-principal-log` and `setup-token-session` tests pass locally. Some other HTTP-log suites could not load locally because workspace package builds were missing in this worktree. CI covers them. ## Risks - Low risk. Referrer query and fragment values are now removed from request logs, as they already are for request URLs. This removes some diagnostic detail to protect secrets. - A route that uses `/invites/<id>/...` for a non-bearer record ID, other than `revoke`, now shows `[REDACTED]` in logs for that segment. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution, used to rebase and verify. The original change was authored in #17848. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 5.5
|
ac0d0374f2 |
perf(issue-detail): measure task opens on real instances (#20276)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People open a task from their issues list to see what an agent is doing and to reply > - On a busy staging instance, the composer and the latest messages take 1.1–2.0 s to appear after a click. For one task with a slow run log, they took 13.9 s > - Local fixtures do not reproduce this. They do not have real network round trips, real server contention, or real object-store latency > - This pull request adds the measurement we need: a browser harness that times a task open on a real instance, and a Server-Timing entry on every API response that separates server time from network time > - It also records the staging baseline and the improvement plan > - The benefit is that each later performance change can be measured on real data with the same tool, before and after deploy ## Linked Issues or Issue Description **Problem or motivation** Opening a task from the issues list is slow on instances with real data. The composer and the latest chat messages can take seconds to appear. We could not measure this on staging in a repeatable way. We also could not tell server time from network and queueing time for most API requests. **Proposed solution** Add a reusable browser harness for task opens, and add a `paperclip_api` Server-Timing entry to every API response. **Alternatives considered** Local Playwright fixtures only. They do not reproduce real network round trips, server contention or object-store latency, so they did not show the delays that people see. Per-endpoint Server-Timing only. It does not cover the many other requests that run during a task open. **Roadmap alignment** This is measurement for existing task-loading behaviour. It does not add product features. The staging baseline and the measured causes are in `doc/plans/2026-11-09-task-loading-performance.md`. Follow-up pull requests change the loading behaviour itself, starting with #20277. ## What Changed - Added `server/src/middleware/api-server-timing.ts`. Every `/api` response now has a `paperclip_api;dur=<ms>` Server-Timing entry. The value is the server time from request arrival until headers are written, including admission waits. The entry is appended, so the existing `paperclip_issue` phases stay first. - Registered the middleware first in `server/src/app.ts`. - Added `tests/perf/issue-detail/staging-click-harness.js`. It clicks a task row and records, frame by frame, when the composer and the latest message become visible, layout shifts, movement of the latest message, and the requests that run before the page is ready. The summary labels each sample instead of using the task reference. It replaces query values and every path segment that contains a digit. A fetch that is still running when the sample ends is listed as pending. - Documented the harness in `tests/perf/issue-detail/README.md`. - Added the dated plan `doc/plans/2026-11-09-task-loading-performance.md` with the staging baseline, the causes, and the prioritized slices. ## Verification - `cd server && npx vitest run src/__tests__/api-server-timing.test.ts`. 3 tests pass. They cover the JSON responses, the time spent in earlier middleware, preserving route phases, and explicit `writeHead` calls. - `tests/perf/issue-detail/issue-detail.perf.spec.ts` reads the first `dur=` value. That value is still `paperclip_issue`. - The harness was used on staging to capture the baseline in the plan document. ## Risks - Low risk. The header has only a duration, with no request, user or company data. - The middleware wraps `res.writeHead` once for each request. This is the same pattern that `on-headers` uses. - Server-Timing is visible to same-origin scripts only. No `Timing-Allow-Origin` header is added. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, browser automation, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandClaude Opus 5.5
|
43fe87795a |
fix(authz): make board-only denials final and actionable for agents (#20151)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People talk to their agents in Agent Chat, and the agents act through the Paperclip API > - Many API routes are board-only. The shared `assertBoard` check returns 403 `"Board access required"` to an agent, with no other detail > - The chat guidance also told the agent to "use permitted discovery to identify the supported operator path" after such a denial > - So the agent keeps searching the API, `call_api` and the shell for a second route. On a slow model, this search uses the whole reply window and the person gets no answer > - Earlier single-route fixes (#19990, #20041, #20008) showed the remedy: a denial that says it is final and names where the person acts. Reply time fell from 10-24 minutes to about 2 minutes > - This pull request applies that remedy once, in the shared board-only check and in the chat guidance > - The benefit is a prompt, correct handoff for every board-only action that an agent cannot do ## Linked Issues or Issue Description Refs #19990, #20008, #20041 (same remedy for single routes). **What happened** In about 30 Agent Chat evaluation cases on a slower model, the agent got 403 "Board access required" (or read `authorization.actor: "board"` in the API catalog). Then it searched for another route until the attempt deadline. Examples: archive a company, set up a managed agent profile, add a company secret. In one recorded run, the agent got the 403 at minute 1, then tried PATCH, environment variables, the CLI and UI paths for more than 3 more minutes, and never replied. **Expected behavior** A board-only denial is final. The agent stops, tells the person that a board user must do the action, links the page in the Paperclip web app where they can do it, and ends its reply. **Steps to reproduce** 1. Open Agent Chat with a CEO agent. 2. Ask it to archive the company (or to add a company secret). 3. The agent calls `POST /api/companies/{id}/archive` and gets `{ "error": "Board access required" }`. 4. The body says nothing about finality or where the person acts. The agent searches for other routes and replies late or not at all. ## What Changed - `server/src/routes/authz.ts`: `assertBoard` now throws `boardAccessRequired(req)`. For an agent actor, the 403 body adds `code: "board_access_required"` and a `remediation`. The remediation says the action is board-only by design, the denial is final, and no other route can do it. When the API path maps to a known board page, it links that page with a `<prefix>` placeholder: company settings, company secrets, budgets, members, tool gateways, connection settings (decision model, prompt injection prevention), an approval, or the agent profile tab that holds the control (API keys, permissions, budget, identity, harness settings). Otherwise it says to link a page only when the route is already known, and not to search for one. Board and anonymous actors get the same body as before. - `environments.ts`, `instance-settings.ts` and the issue force-release route used their own copy of the check. They now call `assertBoard`, so they return the same denial. Status and error text do not change. - `server/src/middleware/error-handler.ts`: for this code, the handler does not repeat `code` and `remediation` inside `details`. They are already top-level, and the copy only made the agent's context longer. - `AGENT_CHAT_DIRECTIVE`: the board-identity paragraph now says that a board-only operation (catalog actor `board`, or a 403 "Board access required") is final. The agent must stop searching, tell the person which control in the web app to use, and link it when it knows the route. The old instruction to "discover the operator path" is removed. - Claude Managed Agent setup: the Paperclip Runner configuration doc and the `POST /api/companies/{companyId}/managed-agent-profiles` catalog entry now say that a board operator creates and qualifies the profile (limited environment, retention acknowledgement) and that agents cannot. Before, an agent asked for an "Anthropic managed agent" explored hiring and adapter docs for its whole reply window. - Tests: new `board-access-denial.test.ts` covers the agent body for each mapped page, the agent profile tabs, the general fallback, the unmapped fast-response route, the org-access guard, the unchanged non-agent body, the guidance and the managed profile catalog entry. Two exact-body tests now also expect the new code. ## Verification - `cd server && npx vitest run src/__tests__/board-access-denial.test.ts src/__tests__/costs-service.test.ts src/__tests__/browser-use-route-scope.test.ts src/__tests__/primary-agent-guidance.test.ts src/__tests__/agent-cross-tenant-authz-routes.test.ts src/__tests__/company-portability-routes.test.ts src/__tests__/issue-comment-cancel-routes.test.ts src/__tests__/error-handler.test.ts src/__tests__/managed-agent-profile-routes-authz.test.ts src/__tests__/companies-route-cross-company-authz.test.ts src/__tests__/environment-routes.test.ts src/__tests__/environment-instance-routes.test.ts`: all pass. - Real chat journeys on the chat evaluation branch (runner-opencode, `deepseek-v4.1-flash`). Before this change, each case below got no reply before its deadline: - `companies.archive-company`: passed. Reply 47 s after the request (before: none in 270 s). The agent read the remediation and linked `/COM/company/settings`. - `companies.configure-decision-model`: passed. Reply 68 s after the request (before: none in 389 s). It linked `/COM/company/settings/connections`. - `companies.prompt-injection-guard`: reply 85 s after the request (before: none in 390 s), with the correct handoff and link. The case still fails later, in the scripted settings step: the saved model is the default, not the model the script typed. That is a separate defect. - `navigation-preferences.sidebar-order`: reply 99 s after the request (before: none in 150 s). The case still fails in its scripted sidebar step (a locator was not found). - `secrets.configure-company-secret`: reply 6.5 min after the request (before: none in 7.5 min). The reply is correct, but a grader keyword check reads "chat is not a safe place to send it" as a request for the secret. - `agents.managed-profile`: still no reply in its 150 s window. After the catalog change, the agent found the board-only profile setup within 30 s, but then spent about 90 s deciding what "Anthropic managed agent" means. ## Risks - Low risk. The HTTP status and the `error` text stay the same for every actor. Board users and anonymous callers get the same body as before. - Agent actors now get two more top-level fields (`code`, `remediation`). A client that compares the whole 403 body of an agent call exactly will see a change. Two server tests did this and are updated. - The page map uses the API path. A wrong map entry gives a less useful link but does not change authorization. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Dotta
|
eb7c7b083a |
Recover Muse verification after contact, restart and access restoration (#19909)
## Thinking Path > - Paperclip manages AI agents and their work under company access rules. > - Muse readiness uses a saved connection, receiver contact and an authenticated challenge reply. > - A failed challenge write could leave a saved contact with no later recovery trigger. > - A server restart or restored access must recover that contact under current authority. > - This pull request uses the existing delivery coordinator and saved Muse state for recovery. > - It keeps the normal receiver schedule, authorization checks and real deadlines. > - It lets eligible verification recover and lets idle verification stop work. ## Linked Issues or Issue Description **What happened?** The first Muse receiver contact could persist while readiness challenge creation failed. The contact then had no durable retry or startup recovery path. Later membership, lifecycle or feature restoration could also leave pending verification without a notice. **Expected behavior** Retry a temporary failure through the existing worker. Recover pending contact after restart. Recheck the current company, agent, operator, credential and binding generation before issuing a challenge. Stop verification work while it is ineligible or complete. Do not revive revoked credentials. **Steps to reproduce** 1. Use the normal invitation and OAuth registration, PKCE and token flow. 2. Send one receiver contact and let its normal persistence schedule run. 3. Fail the first readiness challenge operation before it commits. 4. Observe whether the existing worker retries without another contact or a manual Verify action. 5. Restart the App with a new database owner. Check that it creates one usable challenge from the saved contact. **Paperclip version or commit** The causal baseline test used the unchanged production implementation at `90148317b2a447794ff29a4acae5648ad11a9770` plus its normal fixture. The accepted positive test used `9d383041028776ad8469ac4af3fe14ba6c369425`. The initial published head was `fe518c9b72504daf6374225987f24d45bc7bcf75`. The lease/auth correction was locally qualified at `cfd515a207416097fe8b586277d70c4fd4a679fa`. The current candidate is `6efe722e7eb1e22acd466125c7b4a627611d0273`; its added Cloud actor fixture correction passed the full normal owning program and all 35 tests. Its current normal build, lifecycle owner controls, healthy-wait and restoration measurements are described below. Earlier results below are bound to their stated revisions. Related public changes: Refs #19819 for original-principal authentication. Refs #19750 for the transaction-owned external feature read. These are related foundations, not duplicate fixes. The current Muse title and head-branch searches found no duplicate contact-verification recovery PR. Related maintenance work is #17000 and terminal assignment settlement is #17793. ## What Changed - Add a typed, exact-binding verification notice through the existing Muse delivery queue. - Register the first-contact notice before its durable write in the same transaction. - Recover pending contact from saved state at startup. Retry temporary failures through the existing scheduler. - Recheck current feature, lifecycle, credential and access state after commit. Park unavailable authority without recurring verification scans. - Add narrowly scoped notices to the existing access, lifecycle, configuration, company, budget and settings write owners. - Save a healthy lifecycle verification wait in the existing operation JSONB. Keep generic unresolved-work and error retries unchanged. - Commit authenticated confirmation and its lifecycle recheck in one transaction. Retain a raced recheck under the active lease, and recover saved waits once at startup. - Retain a committed lifecycle recheck while an expired attempt still owns the row, so its late wait cannot park accepted confirmation. Replacement claims keep their existing owner fence and fresh fact read. - Keep the receiver gap, persistence and per-segment contact limits unchanged. - Add real App OAuth, failure, retry, restart, challenge and quiet evidence. Add two normal-route restoration tests. - Document recovery, reconnect requirements and the existing connected-receiver sleep restriction. ## Verification - Causal baseline: the real invitation and OAuth flow saved one contact. The first challenge operation failed. The normal test failed because it observed one verification attempt instead of a retry. No setup or cleanup failure was counted as this regression. - At `9d383041`: normal own Shared/SDK, DB with migration 0441 assets, Runner TypeScript core and surfaces, module boundaries, and all seven normal App programs passed. One new redundant role comparison first failed typecheck; that diagnostic remains recorded and the corrected full App gate passed. - At `9d383041`: all 17 maintenance component tests passed with zero skips. All original deadline, rollback, drain, terminal settlement and startup controls remain. - At `9d383041`: the one normal App case passed. Real OAuth authorized one receiver signal. A retry occurred about 5.01 seconds after the first failure. A new Db/App owner created one challenge, advanced revision 2 to 3, and accepted authenticated confirmation and mailbox ACK. Query instrumentation had an active positive control. All measured query, callback, session and verifier counters stayed equal across the unchanged scheduled 15-second quiet window. Token revocation and old-token/signal rejection passed. - Known App, worker, HTTP and database owners closed without cleanup errors. The tests used bounded local PostgreSQL and HTTP fixtures. They did not call a live Muse model. - Current integrated semantics: the full normal App driver completed all programs. Only a new test callback had two undefined-value diagnostics. An explicit defined-value guard corrected that test. The full 416-root owning program then passed with empty diagnostics. A separate collector incorrectly required an explicit child `noEmit` field; the normal inherited setting was verified without repeating the compiler. The original failures remain recorded. - The first restoration run failed both cases. Permission parking recorded 120 main-pool queries without more verifier calls. The lifecycle counter recorded two denials without their exact scope. These observations remain unexplained at the individual-query/call level. Source separately shows a lifecycle readiness retry every two seconds while the real external reply is absent. - At `fe518c9b`: normal Shared/SDK and DB producers, module policy, all seven full App programs, and all 33 pure diagnostic controls passed. The lifecycle owner file passed all 19 cases, including both confirmation/park orders, rollback, duplicate notices, startup recovery and authority controls. - At `fe518c9b`: the normal OAuth failure/retry/new-Db reconstruction case passed. Its new healthy-wait window measured 15,002 ms with unchanged total activity before genuine confirmation; the existing ready quiet and revocation checks also passed. - At `fe518c9b`: both normal restoration tests passed in Vitest. Ordinary viewer-to-operator restoration and Pause/Resume recovered the same binding, grant and single contact; each committed one challenge and advanced revision 2 to 3. All six parked, restored and revoked/disabled quiet windows measured at least 15 seconds with active positive read controls and unchanged total queries, callbacks, sessions and verifier calls. Both cleanups passed. - The restoration wrapper still recorded failure: it required lifetime lifecycle denial count zero, whereas the measured count was one before the parked quiet window. The reviewed fixture explicitly permits a current-authority denial from an earlier observation and requires no committed challenge before restoration. Precise denial timing relative to Pause was not serialized. The original wrapper failure is retained; no rerun or retrospective explanation of the earlier failures is claimed. - Review exposed an expired-lease race. The added real-store baseline test failed after the late defer restored `waitingForSignal`; no setup or timeout was counted as the regression. On `cfd515a`, the whole lifecycle owner file passed all 20 tests, including that case. The full normal production declaration program (1067 roots), verification program (662), and auth owning program (405) passed; all 1942 declarations remained byte-identical. - The original remote auth-session shard failed three tests because their partial query adapters lacked the new binding query's join/order/limit chain. A fixture-only correction preserved the real notifier and existing authorization assertions. All 12 auth-session tests passed locally. - Complete remote inspection also found 33 failures in the 35-test Cloud actor fixture with the same missing `innerJoin` method and downstream incomplete-auth assertions. Its original full log and exact synthetic Source comparison are retained. At `6efe722e`, the three-line fixture adapter repair passed the complete normal 662-root test program and the whole 35-test file with zero failures or skips. All existing lock, authorization, settings, debounce and unfinished-auth assertions remain. - The earlier healthy/restoration quiet measurements remain bound to `fe518c9b`. Those steady 15-second windows did not exercise a still-owned attempt passing its 120-second lease. The targeted expired-owner regression qualifies that new boundary; it is not a fresh measurement of the earlier windows. - The initial `fe518c9b` remote gates failed the auth fixtures and reported a Greptile 4/5 P1. The aggregate failure reported the underlying shard. Those actual causes were fixed and qualified above. - Current `6efe722e` remote gates completed with **57 successful checks and three explicit skips**. The full latest edited Greptile summary is **5/5**, explicitly reviews this exact head, and has no new finding. The original P1 thread is resolved. All general, inline and nested review pages were read. - One current-head CI attempt timed out after 30 seconds in the Runner evidence/privacy file, with 6,864 passes and one failure. That file and its transitive inputs are byte-identical to the CI base. A separate local diagnostic preserved every assertion and the original timeout, ran the full normal Runner test project and the entire 41-case file, and passed all 41 with zero skips. Its private bounded timing records measured the target body at 74 ms. The diagnostic was then removed, restoring the exact original test. This is local recovery evidence; the original CI stalled await was not recorded and its cause remains **unproven**. - One retry of only the failed jobs on the unchanged head passed the Runner support suite and dependent aggregate verification. The complete second attempt and original failure logs are preserved. No test, timeout, privacy guard or product code was weakened. This result does not turn the original timeout into an explained failure or a diagnosed flake. - Full recursive typecheck, full `test:run` and a full release build were not repeated locally. The normal full App programs, focused producers and tests above passed, and the required current-head remote gates passed. Final review is against the exact unchanged head; local quiet remains separate from deployed suspension. Reviewer commands use the normal project entry points: `node server/scripts/typecheck-layers.mjs`, the whole `server/src/services/muse-maintenance.test.ts` file, and both normal App PostgreSQL files. PostgreSQL fixtures require explicit owned fixture/admin URLs and normal migrations before imports. Run the files serially with one worker. The private controller retains exact Source, output, worker and owner-closure receipts. ## Risks - Missing or early producer notices could lose a restoration trigger. Notices stay in the transaction that commits the write. The worker checks current authority after commit. - A stale notice must not grant access. Binding generation, current membership, company, agent and credential checks remain authoritative. Revoked credentials require reconnecting. - The change adds no table, migration, alternate task queue or periodic verification scan. It adds two optional fields to existing lifecycle operation JSONB. Generic pending/error deadlines, setup holds and manual repair remain obligations. - Confirmation can race with an active verification attempt. The park compare-and-set must retain the durable recheck. Startup recovery uses a fixed cutoff and keyset pages; it must not become another recurring scan. Wrong company, stale version, pause and revoked credentials must stay fenced. - The connected receiver still has its documented tenant-sleep restriction. Worker quiet is not proof of deployed suspension or external wake. - The normal challenge reply is sent by an authenticated protocol fixture. Live Muse background replies, cost/usage, full Muse lifecycle and full hosted Dot behavior remain unqualified. Partial Dot profile ACK and unsigned-device cutover stay withheld. - Restoration coverage for the additional company, budget, configuration and feature producers remains a separate qualification obligation. This pull request does not mark the whole hosted area complete. ## Model Used OpenAI `gpt-6.1-sol` authored the implementation and tests with local code and tool use. A GPT-6-based Codex coordinator reviewed the changes and owns integration. Its exact underlying model ID is not exposed. Context window sizes are unavailable. No context size is inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
3112e4c955 |
Verify original Dot OAuth authority for signed Cloud requests (#19819)
## Thinking Path > - Paperclip manages AI agents and their work. > - Dot uses a user OAuth grant to act in one company. > - Cloud must use that original grant when it resolves a selected user's identity. > - A Cloud signature alone must not create user or board authority. > - The tenant's OAuth owner must check the token and its current restrictions. > - This pull request adds a signed private HTTP boundary for that check. > - The result contains only the checked identity fields. It grants no task or board access. ## Linked Issues or Issue Description **Subsystem affected** Server external-agent identity, public MCP OAuth, Cloud control verification, and HTTP logging. **Problem or motivation** Cloud needs a private way to resolve the original Dot token for the selected user, client, company and resource. The existing generic Cloud control route creates an operator actor. That route is not the right owner for this identity check. **Proposed solution** Add `POST /mcp/runner/oauth/cloud-principal`. Require the existing Cloud signature and replay verifier. Bind the whole request to a new, closed `mcp-dot:principal` action with a maximum 60-second lifetime. Authenticate the original token once through the normal Dot OAuth owner. Check current namespace authority before and after that call. **Alternatives considered** A Cloud-created grant or a board actor would change the authority owner. Neither is used. The private route reuses the existing OAuth and Cloud identity owners. **Roadmap alignment** This is a narrow boundary for the existing MCP and Cloud areas in `ROADMAP.md`. It adds no work queue, public hosted profile, or provider enablement. Related public work: Refs #14846, #19427, #19750 and #19776. Searches for the exact private path and action found no duplicate PR. The matching private Cloud caller has merged. The current App head passed its required checks and review. ## What Changed - Add an original-token principal reader. Keep the original client, selected user, company, resource, purpose, scope, membership and feature restrictions. - Add a 16 KiB raw JSON private route. Reject compressed, invalid UTF-8, malformed and unknown-field bodies. Bind an ordered whole-body digest to the signature and existing nonce fence. - Compare the current stack, claim, instance, origin and connector identity before and after OAuth. An absent owner or namespace fails closed. - Mount the route in normal `createApp` after hostname, runtime-claim and standby checks, before session/actor work. Do not add it to generic board-control routing. - Keep private request and response logs content-free, including raw buffers, headers, query data and reflected errors. - Add controlled HTTP/privacy tests and a real normal-App OAuth/Cloud-signer integration case. Document the machine-only contract and correct the existing `assistant-connections/client-use` access entry. No new capability ID or agent API entry is added. No schema, lockfile, workflow, Runner, Muse or useful-task fixture changes belong to this PR. ## Verification - Current `70c03593`: normal module-boundary policy and all seven App compiler programs passed. The public module factory has an explicit portable Express return contract. Whole HTTP/log controls passed again: 30/30. The existing real paired OAuth/Cloud-signer/private HTTP case then passed again on a fresh PostgreSQL fixture: 1/1, zero skipped. All known child processes and five database connections closed; the final independent SQL audit was empty. - The original remote policy failure rejected an App import of a module-internal path; dependent E2E/verify gates stopped on that failure. The fix delegates through the public module API. A first local declaration run reported TS2883 for its inferred return type; the explicit public type fixed it. Original logs remain preserved. No checker, compiler option, test body or authority check was weakened. - The normal full App graph passed on the integrated source. The later fixture-only repairs each passed the complete affected owning program:405 entries, including 403 source roots. The remaining programs and own dependency outputs carried only where their source was exact. - Whole private HTTP/log files passed:30 tests, zero failures or skips. The HTTP controls use a controlled OAuth owner with real signatures. The log controls use the actual HTTP logger. - The paired Cloud source passed its normal strict build and whole 26 signer/client/namespace and credential-error controls. These are controlled caller tests, not provider tests. - One normal-App paired case passed with a dedicated PostgreSQL database. It used real DCR 201, PKCE, consent, token exchange and a persisted unpaired grant; actual signed metadata publication and a committed Cloud receipt; the real Cloud signer/client and actual App HTTP; exactly-once authentication; refresh; current namespace restrictions; and revocation. The Cloud registry/provider were the existing controlled in-memory/Fake fixtures. Recorded inbound Hosts matched, with zero metadata overflow. No grant, readiness or mirror ACK was fabricated. - The original principal helper's 21 controlled tests and two real-OAuth tests passed before the HTTP slice. Those three files remained unchanged through integration. They are separate source-scoped evidence, not an extra combined run. - Preserved failures: a helper import and test typing error were corrected before the passed checks. An earlier suite failure did not serialize its hook error and remains unexplained. Later runs captured a missing normal managed-config initialization and HTTP 404 from the loopback mapper. The fixture now calls the normal initialization owners and uses the existing Host-preserving canonical HTTP adapter. All original authority assertions and deadlines remain. - Reproduce the local files with `pnpm --filter @paperclipai/server exec vitest run src/modules/external-agents/cloud-principal-http.test.ts src/modules/external-agents/cloud-principal-log.test.ts --maxWorkers=1 --no-file-parallelism`. The real integration file is `server/src/__tests__/cloud-principal-normal-app.postgres.test.ts`; it requires its explicit dedicated test DB and an own compiled matching Cloud checkout. - The normal `node scripts/check-maps.mjs` check passed for 35 areas and 192 capability IDs. This validates map structure, not business acceptance. The later edits are documentation/map only; all executed business source and own outputs from the paired case remained exact. - Current-head remote CI passes: 59 successful checks and three intentional skips. Greptile reviewed this exact head at 5/5 with no findings or review threads. The prior Ponytail review was Ship 5/5 at the earlier head; no current-head Ponytail turn is present, which is nonblocking under the authorized review policy. The three full-repository commands were not repeated locally; the focused normal App graph and affected tests above passed, and required remote CI supplies the broad gates. No deployed, provider-capacity, cold-wake or full hosted lifecycle result is claimed. ## Risks - This is a security boundary. Signature, current namespace and original OAuth restrictions must continue to fail closed. The response gives identity only; callers still need normal operation authority. - The request carries the original token. The reserved route uses a closed log projection on both accepted and refused requests. Privacy regressions remain a material risk. - Both repositories must agree on the action and digest contract. Older code rejects the unsupported action; no fallback creates authority. - This is a prerequisite private boundary. The remaining hosted discovery, wake, capability, operation, event, retry/reactivation and revocation lifecycle work, plus Muse, remains open. No profile or capability is enabled here. - No migration is needed. Rollback removes the private boundary and paired caller; normal OAuth and Personal MCP remain with their existing owners. - The engineering documentation describes this private machine boundary and its access restriction. No operator-facing feature is enabled. Documentation for full hosted enablement remains with that later work. ## Model Used OpenAI `gpt-6.1-sol` (Sol6.1), through Codex, produced the source and tests with tool use and local compiler/test execution. A GPT-6-based Codex coordinator reviewed the work. Its exact model ID is unavailable. Context window sizes are unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
6be71028ad |
Prepare online ephemeral enrollment through signed Cloud control (#19660)
## Thinking Path
> - Paperclip manages agent work through a control-plane web server.
> - A replacement web server needs shared durable data and the original
signing identity.
> - Existing enrollment exports that identity only after a global drain
and offline migration.
> - Online enrollment needs a separate preparation operation while agent
work continues.
> - This change adds a purpose-bound Cloud control operation with
existing data and key owners.
> - The result is an audited preparation receipt, with no permission to
stop agents or detach storage.
## Linked Issues or Issue Description
Refs #19038. Refs #19024.
**What are you trying to improve?**
Allow Cloud to prepare an already migrated tenant for volume removal
without the offline migrator or a global-idle gate.
**What problem does this solve?**
Cloud must preserve the original decision key before starting a
replacement with static ephemeral mode. A tenant session must never gain
key-export authority.
**What would the improvement look like?**
A fresh exact-action Cloud assertion obtains a private, deployment-bound
preparation receipt. Unfinished durable-data migration and invalid
replacement configuration block export.
## What Changed
- Add the signed `ephemeral-enrollment:prepare` control action and a
strict UUID-only POST route.
- Check existing durable-data eligibility and ephemeral configuration
before reading the original key. Never create a missing key.
- Audit only public preparation metadata. Keep HTTP diagnostics
secret-sensitive and the response non-cacheable.
- Bind the receipt to the attempt, stack, build commit, Railway
deployment, time, static mode and key digest.
- Always return `enrollmentReady: false`. Cloud still needs to arm a
replacement, verify old writers have stopped, repeat eligibility and
qualify native handoff before detaching the retained volume.
## Verification
- PASS: 80 focused tests across the preparation owner, signed route,
existing Cloud control, HTTP privacy and OpenAPI route coverage.
- PASS: 2 real PostgreSQL cases. Concurrent copies of one signed request
export once; the durable receipt still rejects reuse after clearing the
process fence. Normal run admission before skill pins is allowed;
missing durable skill installation remains blocked.
- Genuine prepatch RED: the same two PostgreSQL cases exported twice and
rejected ordinary admission. The fixed owner was restored automatically
after the comparison.
- PASS: server TypeScript check with workspace dependencies built.
- Fixed the first CI route-coverage failure by declaring this private
control endpoint in the existing explicit OpenAPI exclusion registry.
- Initial local HTTP execution lacked sandbox socket permission; the
reviewed run passed. Initial PostgreSQL fixtures enabled durable files
too late and reused an issue prefix; those fixture defects were
corrected before the red/green comparison.
- All required CI gates pass on
|
||
|
|
e482f0f632 |
fix(server): shed only retryable object-journal reads under load (#19176)
Client polling can exhaust the shared object journal queue and reject log persistence. Bound ordinary object-log and trace reads, keep persistence and internal verification queued, and return retryable 503 responses when reads are busy. Leave feedback exports unsent until every log page is available.
Validation: 72 latest-master affected tests and 10 workflow tests pass locally, with the workflow fixture passing strict TypeScript checks. The preceding broader run passed 280 server tests with two opt-in S3 skips. All applicable checks on
|
||
|
Devin FoleyandClaude Opus 5.5
|
2c7ee84221 |
fix(server): stop reporting work cancelled by a disconnected client (#19037)
A route that aborts its storage read when the response closes rejects with AbortError after the client has gone. The error handler reported that cancellation as a server crash. Skip the error sinks for an AbortError when the response was closed before it finished; report it otherwise. Fixes PAPERCLIP-BACKEND-5A Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
StefanandArthur Dent
|
131c0c4609 |
fix(server): redact the credentialValues envelope in HTTP logs (#13742)
> **Attribution:** this pull request was written and opened by the
Paperclip agent **Arthur Dent** (Platform & Reliability Engineer,
Tharsia) from an automated run under the host GitHub credential. GitHub
records the author as `stefanriegel`; Stefan Riegel did not write it. No
other edit was made to this description.
## Thinking Path
> - Paperclip is the open source app that people use to manage AI agents
for work.
> - The server writes request details to the user journal. A middleware
module names the fields that are secret.
> - The connect endpoint accepts a `credentialValues` object. The object
maps a provider config path to a secret string.
> - One provider config path is `credentials.authorization`. The caller
chooses the key names. Therefore the key is one flat string, and it is
not a nested path.
> - The redaction path list did not cover the `credentialValues`
envelope. The recursive redactor compares keys by exact match, so
`credentials` and `authorization` in the key set did not match the flat
key `credentials.authorization`.
> - A failed connect request therefore wrote a GitHub token to the
journal in cleartext.
> - This pull request redacts the `credentialValues` envelope in both
layers.
> - The benefit is that no provider secret reaches the journal, for any
key name that a caller chooses.
## Linked Issues or Issue Description
No public issue exists. The report follows the bug report template.
**What happened?**
A `POST /api/companies/{companyId}/tools/apps/connect` request returned
HTTP 502. The server logs request bodies for non-2xx responses. The
journal line contained a GitHub fine-grained personal access token in
cleartext. The token was 93 characters long and started with
`github_pat_`. The line contained no redaction marker.
The logged body had this shape:
```
"reqBody":{"galleryKey":"github","connectionMethodKey":"mcp-key","name":"GitHub",
"credentialValues":{"credentials.authorization":"<TOKEN>"}, ... }
```
**Expected behavior**
The journal line shows a redaction marker in place of the token. No
cleartext credential reaches the journal on any response status.
**Steps to reproduce**
1. Run a Paperclip server on version 2026.916.0.
2. Send a `POST` request to
`/api/companies/{companyId}/tools/apps/connect`.
3. Put a secret under `credentialValues` with the key
`credentials.authorization`.
4. Make the request fail, so that the server returns a non-2xx status.
5. Read the journal line for the request.
Result: the secret appears in cleartext. The two redaction layers both
miss it.
**Paperclip version or commit**
2026.916.0. Reproduced against `master` at
`c65fc9e3c81c41aafe421aa90a00514b84343285`.
The two middleware files are unchanged at `master`
`0f5fafe16b069b8f0c7f17eb9080d010aa6cd513`, so the defect is present at
the current head of `master` as well.
**Deployment mode**
Self-hosted.
**Installation method**
npm package.
**Access context**
Authenticated user, company-scoped route.
**Relevant logs or output**
Token value removed. The journal line has the shape above.
## What Changed
- Add `reqBody.credentialValues` to `HTTP_LOG_REDACT_PATHS` in
`server/src/middleware/http-log-redaction.ts`. This redacts the whole
envelope at the pino serialization boundary.
- Add `credentialvalues` to `SENSITIVE_KEYS` in
`server/src/middleware/redact-sensitive.ts`. This redacts the envelope
by name in the recursive pass.
- Add a test that sends a `credentialValues` envelope through a real
pino logger with the production path list. The test asserts that the
secret is absent.
- Add a test for the flat-dotted-key shape that leaked, and for the
nested shape as a regression.
- Add a test that the envelope is redacted below the top level.
The key-set change is the durable half. The path list only covers body
shapes that someone has already seen leak. The recursive pass matches
the envelope by name for any shape.
## Verification
Run these commands:
```
npx vitest run server/src/__tests__/http-log-redaction.test.ts server/src/__tests__/redact-sensitive.test.ts
```
Against the patched source, both files pass: **72 tests passed, 0
failed**. That is 54 tests in `http-log-redaction.test.ts` and 18 tests
in `redact-sensitive.test.ts`.
The same two files were then run against the **unpatched** source, which
is the parent commit of this branch. **Four tests fail there.** Those
four are exactly the tests this pull request adds:
- `http-log-redaction.test.ts` — "defines the HTTP auth and cookie
header paths that must be redacted". The path list does not contain
`reqBody.credentialValues`.
- `http-log-redaction.test.ts` — "redacts a credentialValues envelope at
the serialization boundary". This is the leak: a real pino logger writes
the token in cleartext.
- `redact-sensitive.test.ts` — "redacts the connector credentialValues
envelope on an app-connect body".
- `redact-sensitive.test.ts` — "redacts a credentialValues envelope
nested below the top level".
The four failures show that each half of the change is load-bearing, and
that both layers failed, not one. The control run also confirms that the
68 tests which pass on the unpatched source still pass on the patched
source, so the change regresses nothing that already held.
The tests assert that the secret string is absent, and they do not pin a
redaction marker. This is deliberate: the two layers write different
markers. `fast-redact` writes `[Redacted]`. `redactSensitive` writes
`[REDACTED]`. An assertion that requires `[REDACTED]` alone fails on a
correct path-list-only fix.
## Risks
Low risk. The change adds two redaction paths. It removes no existing
path.
One behavioral shift is intentional. The recursive pass now replaces the
whole `credentialValues` subtree with one marker, instead of replacing
individual leaves. Any non-secret sibling field placed inside
`credentialValues` is also redacted. This is correct, because the
envelope holds only provider secrets by definition.
One adjacent pull request, `#10784`, adds header paths to the same array
in `http-log-redaction.ts`. The two changes touch different regions of
the array. The second one to merge needs a trivial rebase at most.
## Model Used
- Provider and model: Anthropic Claude, via the Paperclip `claude_local`
agent adapter
- Capability details: tool use and code execution in a sandboxed
workspace
- The patch was authored and verified by automated agent runs
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [ ] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
Left unchecked on purpose: the CI and Greptile items cannot be true
before the pull request exists.
"I have run tests locally and they pass" stays unchecked as a
whole-workspace statement. The two test files named under Verification
were run locally, against both the patched and the unpatched source, and
the numbers above are from those runs. The full workspace suite has not
run.
## Related pull requests
Refs `#10784`, `#6373`, `#8841`.
Co-authored-by: Arthur Dent <arthur.dent@paperclip.local>
|
||
|
Devin FoleyandClaude Opus 5.5
|
c56aa689ac |
fix(server): say where secret-sensitive requests fail in Sentry (#19040)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The server sends unexpected 5xx errors to Sentry through the shared
Express error handler
> - Some routes carry credentials: chat setup, provider webhooks, MCP,
company observability settings, Muse, avatars, and similar routes
> - On these routes the handler replaces the error with a generic
`Secret-sensitive request failed` error, because the original name and
message can contain secrets
> - That replacement also removes every useful fact. The Sentry event
does not show the route, the status, the error type, or the throw site,
so a maintainer cannot find the cause
> - This pull request adds a small fixed-shape record to that generic
error. The record says where the failure came from, and it never copies
request data or error text
> - The benefit is that the next event on these routes points to a route
and a code location, while the privacy rule stays the same
## Linked Issues or Issue Description
**What happened?**
A hosted staging server reported one event with the message
`Secret-sensitive request failed`. The event had only the stack of the
sanitizer itself. It had no route, no status, no error type, and no
frame from the original error. The trace had no spans and no logs. A
maintainer cannot find the root cause from this event.
**Expected behavior**
The event keeps the generic message, so no secret leaves the process.
The event also tells the maintainer which secret-sensitive route family
failed, the HTTP method and status, the error type, a safe error code
when one exists, the matched route definition, and the original throw
site.
**Steps to reproduce**
1. Make a route handler throw on a secret-sensitive path, for example
`PUT /api/companies/:companyId/observability`.
2. Enable the backend Sentry DSN.
3. Look at the captured event. It shows only `Secret-sensitive request
failed` and the sanitizer stack.
**Paperclip version or commit**
`master` at
|
||
|
Devin FoleyandClaude Opus 5.5
|
3c7af35103 |
fix(errors): return tool gateway rejections with their own status (#19158)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents call connected tools through the tool gateway. Some tool calls need a human review before they run. > - A user approves a tool review from a task card. The approve route calls the gateway. > - The gateway can reject the approval with `ToolGatewayHttpError(409, "approved_tool_target_changed")`. This is an expected rejection: the tool or connection changed after the review. > - `ToolGatewayHttpError` does not extend `HttpError`. The shared error handler only keeps the status of an `HttpError`. Thus the route answers 500 "Internal server error" and sends the rejection to Sentry as a crash. > - The user does not see the reason. The user clicks Approve again, and each click makes one more crash report. One user made 69 reports on staging this way. > - This pull request makes the shared error handler keep the status and reason code of client (4xx) gateway errors. > - The benefit is a clear 409 with a reason for the user, and no false crash reports. ## Linked Issues or Issue Description Refs #12775 (related community PR. It changes `ToolGatewayHttpError` to extend `HttpError`, which affects every `instanceof HttpError` check. This PR is smaller and keeps gateway server failures on the crash path.) **What happened?** A user approves a tool review on a task card. The tool definition or connection changed after the review. The API answers 500 "Internal server error". Sentry records the expected rejection "Tool definition or connection changed; request a new review" as a server crash. Each retry records one more event. **Expected behavior** The API answers 409 with `code: "approved_tool_target_changed"` and the message "Tool definition or connection changed; request a new review". Sentry does not record a crash. **Steps to reproduce** 1. Make a connected tool that needs approval, and start a tool call so that a review card appears. 2. Change the tool connection (for example, its config) before you approve. 3. Click Approve on the task card (`POST /api/issues/:id/interactions/:interactionId/accept`). 4. See 500 "Internal server error" and a new Sentry event from `approveActionRequest`. **Paperclip version or commit** `master` at `482ec0aea9ab6f2684baaa1487f2c7a2eedf009e`. ## What Changed - `server/src/middleware/error-handler.ts`: the handler converts a `ToolGatewayHttpError` with a 4xx status to an `HttpError`. The new error keeps the status, the message, the gateway details, and the reason code as `code`. The original stack stays. - Gateway errors with a 5xx status do not change. They still go on the crash path, answer 500 "Internal server error", and go to Sentry. Thus provider text in those messages stays hidden. - `server/src/__tests__/error-handler-tool-gateway.test.ts`: new tests for the 409 mapping, for the unchanged 5xx path, and for an Express route that lets the gateway error escape. ## Verification - `cd server && npx vitest run src/__tests__/error-handler-tool-gateway.test.ts src/__tests__/error-handler.test.ts` passes: 21 tests. - Without the fix, the new tests fail with `expected 500 to be 409`. With the fix, they pass. - `src/__tests__/issue-thread-interaction-routes.test.ts` and `src/__tests__/json-parse-error-handler.test.ts` also pass: 175 tests in 4 suites with the two error handler suites. - I checked the runtime import graph of `services/tool-gateway.ts` (685 modules). It does not import `middleware/error-handler.ts`, so the new import makes no cycle. - I did not run the full server typecheck locally, because it ran out of memory in an 8 GB sandbox. CI must do this check. ## Risks - Low risk. Only gateway errors with a 4xx status change. Before, they became 500. Now they keep their own status. - Some routes outside the gateway router can now return gateway 4xx details in the `details` field. The gateway routes already return the same details through `sendGatewayError`. - `middleware/error-handler.ts` now imports `services/tool-gateway.ts`. Unit tests that import the error handler load more modules. The app already loads this module. - This PR does not fix the cause of some of these rejections. A routine OAuth token refresh changes values in the signed approval snapshot (token secret versions, the order of grant secret refs, and token expiry times). This can make a pending review fail. That fix changes `services/tool-gateway.ts` and comes in a separate PR. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used as a coding agent with tool use (shell, file edits, test runs, and GitHub API). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no documentation change is necessary) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
5877d751c0 |
feat(server): bind idle credential eligibility to incoming wake proof (#18639)
Reviewed and verified at
|
||
|
DottaandPaperclip
|
b8f8b904fa |
feat(routines): preserve automation ancestry through webhooks (#18772)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines let external services start agent tasks through a normal webhook. > - A repair routine must recognize failures from work that it created. > - The receiver added in #18714 put Sentry payload parsing and repair policy in core routine intake. > - This pull request replaces that receiver with verified creation provenance on normal routine calls. > - Sentry can filter the resulting tags before it sends a normal routine webhook. > - The benefit is reusable provenance and loop prevention without a provider-specific route. ## Linked Issues or Issue Description **Problem or motivation** A Sentry repair alert can start another repair when its own work fails. The dedicated receiver in #18714 solved this through a Sentry-specific payload contract. Ordinary routine webhooks also need a trusted way to retain the automation that created their tasks. **Proposed solution** Use the existing public `/fire` route and its current authentication. An optional source run credential proves a live Paperclip caller. Store the caller separately from webhook JSON. Follow recorded task creation links and export bounded ancestry tags on run-failure events. Let Sentry alert filters exclude repair-origin work. **Alternatives considered** Keep the dedicated Sentry receiver, accept sender-supplied origin IDs, or filter only the immediate routine. The first couples core intake to one provider. The second accepts unverified claims. The third misses child tasks, standalone follow-ups, and calls through another routine. **Roadmap alignment** This improves the shipped Scheduled Routines and Activity log capabilities. It does not add a new scheduler or incident system. Related work: #18714 is the implementation replaced here; #9727 adds source-task navigation and UI attribution; #10206 proposes exception-led routine dispatch. This PR uses existing task creation records and does not implement either of those broader proposals. ## What Changed - Remove the Sentry-only route, parser, authentication exception, and payload-specific tests and documentation from #18714. - Add optional `X-Paperclip-Source-Run-Token` support to normal routine webhooks. Validate the standard run JWT, company, instance, agent, live run, and cancellation state. Authenticated routine API calls also retain their validated caller. - Add nullable `routine_runs.created_by_run_id`. Keep the reference after source-run deletion so missing history becomes unknown. Separate callers during coalescing, including legacy open tasks. - Follow company-scoped routine receipts, creator runs, task parents, and legacy creation activity. Mark missing, foreign, cyclic, private, or excessive ancestry for safe alert filtering. - Add event-local `automation_origin_status`, `automation_routine_ids`, and `task_shareable` tags through the existing Sentry SDK hook. Cap ancestry at 32 reads and share a 750 ms deadline across metadata and ancestry lookup. Keep the SDK one-second wait and eight-lookup concurrency bound. Redact the source credential from HTTP logs and Sentry request headers. - Preserve unexpected database failures as server errors without rejection bookkeeping in an aborted transaction. Document 409, 415, and 422 responses for the normal webhook. - Keep `routineScopedRunId` as generic opt-in `X-Paperclip-Idempotency-Scope: routine` support. Default trigger receipts and timestamped HMAC replay protection retain their current scope. - Add [the Sentry repair tutorial](https://github.com/paperclipai/paperclip/blob/codex/routine-creation-provenance/docs/guides/board-operator/sentry-run-failure-repairs.md), API documentation, OpenAPI headers, and capability-map updates. ## Verification At commit `0f4a0051528edd1367a921a76a8050c3fd087872`: - `pnpm -r typecheck` and `pnpm build` passed for the full workspace. Migration numbering and safety checks passed. - The focused service, route, Sentry, redaction, and OpenAPI suites passed 228 tests in five files. The database Sentry resolver and OpenAPI invocation passed 33 tests in two files. These are separate invocations with some overlap. - The full routine HTTP suite passed all 15 tests. The authenticated API case failed before the caller-wiring fix and passed after it. The fixture loads the real route-owned services. No test deadline was raised. - The real `@sentry/node@10.71.0` suite passed all 12 tests. It uses an in-memory transport and sends no events to Sentry. It covers terminal failures, saved-message interruptions, event isolation, and source-token redaction. - Regression tests also cover forged, expired, foreign, and cancelled caller credentials; payload spoofing; inherited routine ancestry; legacy coalescing; deleted history; private work; cycles; query and time limits; database error causes; and routine-wide receipts. The new error and budget cases failed before their fixes. - `node scripts/check-maps.mjs` and `git diff --check` passed. The map check covers 35 areas and 192 capabilities. - The full local Vitest command was stopped for a required master merge after four chat shards passed 1,278 tests. No full local pass is claimed. An earlier combined HTTP run hit an existing malformed-ID timeout whose original cause is not established; the final full HTTP suite passes. An extra local run of the unchanged heartbeat scheduling file passed 12 of 13 tests and failed a readiness-status assertion. Its logs also reported a missing fixture user membership. The same 13-test file now passes in current-head CI. - All GitHub Actions checks pass on this commit. Three jobs initially lost their self-hosted runners. One server shard initially failed while deleting a company in an unchanged heartbeat test because of a `company_skills` foreign key. One retry passed all affected jobs without code changes. That server shard passed all 499 tests, and all eight browser shards passed. - Fresh Greptile review is 5/5 on this commit. All six review threads are resolved. The separate Paperclip Review check remains queued. Its bot reports that work is paused. A maintainer must resume the assigned review work to complete that check. No full green check state is claimed while this external gate is pending. - Live Sentry configuration and deployment are outside this PR's acceptance evidence. The tutorial describes setup, a controlled failure, and a repair-origin exclusion check for operators to run after deployment. ## Risks - Normal app webhooks retain the sender's full JSON in company-readable routine history and the execution task. Alert filters and task visibility are not content scrubbers. Operators must scrub sensitive event data in Sentry before forwarding it and keep connection tests in setup mode. - Existing users of `/sentry-run-failures` must move to `/fire` and add the documented Sentry filters before enabling repair intake. The old route is removed. Stored tasks and receipts remain. - The additive migration uses `ADD COLUMN IF NOT EXISTS`. It does not rewrite historical tasks or infer missing provenance. - Unknown or unshareable ancestry is excluded by the tutorial's alert rule. Missing history, an enrichment timeout, or more than five routine IDs can therefore suppress a repair webhook while retaining the Sentry error. - An external webhook without a verified source credential starts a new receiving-routine origin. Its JSON cannot establish upstream Paperclip ancestry. Sentry must not store a Paperclip run credential. - Routine-wide receipt reuse needs a stable sender key. A static Sentry custom header is not a per-event key. The tutorial keeps delivery deduplication separate from loop prevention. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-6 model family, with reasoning, repository access, code editing, and test execution. The session does not expose the exact model variant or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
5dee4993f1 |
fix(server): distinguish aborted HTTP responses in access logs (#18781)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use HTTP logs to investigate task updates and database contention. > - Node starts each response with status 200 before it sends any headers. > - The custom HTTP formatter printed that status when the client disconnected before a response. > - This made an aborted request look like a successful task update. > - This pull request labels incomplete responses as aborted and preserves error severity. > - Operators can distinguish HTTP delivery from database commit evidence. ## Linked Issues or Issue Description - Refs: #18753. - Related: #14369 removes writes from legacy comment attribution reads. This change addresses a separate logging defect. The missing-write incident in #18753 remains unconfirmed. We reproduced a misleading `PATCH /issues/:id 200` log when the client received no HTTP response. We did not reproduce an HTTP 200 delivered before the mutation committed. ## What Changed - Label premature connection closure as `request aborted before response` or `request aborted during response` in both success and error log paths. - Log aborts as warnings, including on routes whose successful polling logs are normally silent. Keep explicit errors and 5xx responses at error severity. - Keep private webhook paths and query values out of aborted-request logs. - Add real HTTP disconnect coverage and real PostgreSQL task-update tests with synthetic data. - Document that HTTP completion logs do not establish database commit or rollback. ## Verification - Baseline: the new HTTP disconnect test failed because the log said `PATCH /api/issues/synthetic-task 200` at info level, while the client received no response. - `node node_modules/vitest/vitest.mjs run server/src/__tests__/http-log-aborted-response.test.ts server/src/__tests__/http-log-redaction.test.ts server/src/__tests__/issue-update-lock-contention-postgres.test.ts` passed: 81 tests in 3 files. - `pnpm -r typecheck` and `pnpm build` passed. The build used a second clean clone at the exact PR commit. - GitHub CI completed at `e3e96d836119df5fa1c7f22615484265de5aa700`: 52 checks passed and two Storybook checks were skipped. Greptile returned 5/5 with no findings. - Local `pnpm test:run` passed all six chat shards (1,834 tests). Its general server stage then reported four failures. We stopped the run after preserving these results and comparing the failing cases. The later workspace and serialized stages were not completed locally. The full local suite is not green. - The four observed failures were the two OAuth concurrency cases in `tool-access-service.test.ts`, the bounded workspace conflict case in `workspace-runtime.test.ts`, and the `enableNativeRunner`-disabled stop-control case in `muse-control.test.ts`. - Comparisons used the same sanitized environment on unchanged master `2b14ec03a48f17606db53fd7fc0edc53bc930409` and this PR. Both OAuth cases pass in isolation on both revisions. The full tool-access file passes all 498 tests on the PR; master has a different 15-second timeout, with 497 tests passing. The workspace case passes on the PR but fails port-reservation setup on master. The Muse case passes on both. These results do not explain or clear the original broad-suite OAuth and Muse failures. - Tested on macOS arm64, Node.js 26.4.0, pinned pnpm 9.15.4, and disposable embedded PostgreSQL. The API fixture uses a synthetic process-adapter agent. No model provider runs and no production database or real credentials participate. - The PostgreSQL fixture verifies an actual lock waiter with `pg_blocking_pids`. A blocked issue update does not return 200 before release. After release, status, monitor policy, inline comment, and audit receipt are persisted before the completed response. - Holding the company row with `FOR NO KEY UPDATE` does not block these fixture writes. They return 200 with persisted data while the compatible parent-row lock remains held. - An injected database trigger failure returns 500 and leaves the issue, comment, and mutation audit unchanged. Two influence-attempt receipts remain under the existing documented cap policy. - The combined lock/disconnect/failure test verifies no delivered response, an aborted log, and rollback after an injected database failure. The injected failure is not evidence of the reporter's production root cause. - Source inspection at release `v2026.1023.0` (`1b4c803816e82b0265c8422e1a804327a7ed8c54`) finds the same unconditional success formatter and awaited issue-update transaction. That release was not runtime-tested. ## Risks - Log consumers that parse the literal method/path/status message must handle the aborted-request message. - Client disconnects during streaming or SSE responses also use the aborted warning. Expected disconnects may therefore produce more warnings. - A disconnected request may still commit after the client leaves. This change reports transport completion and does not cancel or retry mutations. - This PR does not close #18753. The original missing-write cause and a received-200-without-commit scenario remain unconfirmed. - Local full-suite failures remain unresolved despite green CI. Keep this PR in draft until that verification gap is accepted or resolved. > This is a targeted bug fix. ROADMAP.md does not list overlapping feature work. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, tool use, code execution, and parallel subagent investigation. - The exact runtime model ID and context window size were not exposed to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
27d35314be |
feat: connect experimental Cursor Cloud to native Runner (#18069)
## Thinking Path > - Paperclip helps people manage AI agents at work. > - Native Runner owns execution and task authority. > - Operators need an explicit native Cursor Cloud choice. > - Tools/files need company, task, session and run scope. > - This connects setup and lifecycle to Runner Core. > - Legacy agents retain their selected runtime/history. ## Linked Issues or Issue Description Depends on #18068. Managed-only API placement and API-owned Stop containment are in #18310. Refs #5664, #11803, #17299, and #17471. No duplicate native integration was found. ## What Changed - Add explicit native setup, Test, saved settings and dispatch with no-prompt account/catalog/callback checks. - Scope MCP to company, agent, task, session and run through authenticated HTTPS. Register verified artifacts/work products; preserve unpriced accounting, completion and budgets. - Connect Question/Plan continuation, cold handoff and admitted fresh sessions; recheck waiting admission under the task lock. - Present audited Stop intent while preserving physical containment proof; bound typed Stop metadata. Preserve title edits through preference/navigation readiness. - Select one live/history representation per run to prevent duplicate settled cards. Integrate upstream task/read-state/retry/schema contracts and repair two test fixtures. ## Verification Core #18068 merged normally as `3910352c923a`. Product head `5a60f541c982` has passing repository CI/build/browser checks, including required `ci / verify` and `ci / e2e`, plus a fresh exact-head Greptile 5/5 review with zero unresolved threads. Two non-required external app checks remain pending: Paperclip Review is queued waiting for its assigned agent; Superagent Security Scan is reviewing the diff. These are not claimed completed. The normal merge endpoint enforces the repository rules; no policy or status override is used. The dependent Top PR is addressing its fresh deferred-Stop review finding. Product `5a60f541c982` passed 270 UI and 20 selected Cloud/native/disk-full tests, UI/server typechecks, tokens and maps. Top’s clean normal merge passed tokens/maps/diff checks; its matrix is pending. Earlier receipts retain Core’s 47 Rust, 4 compiled-native retirement and 19 settlement controls, normal Runner build, and runtime 9af build/recursive typecheck. Reproduced loading/shutdown CI fixture failures passed 7/7 browser and 121/121 unit tests, including a broken-unsubscribe negative control. No timeout/oracle was relaxed. The actual-parent restack adopts 75 mainline paths, including schema/permission and generic runtime changes, beyond the prior two fixture repairs and client projection fix. The full difference from hosted `9af1a5de42a2` is therefore larger than four files; backend source also differs. The merged config-read checks match Core391, and the Cursor placement/runtime/relay/selection modules match frozen9af; selected affected regressions pass. Paid proof keeps its actual 9af source/image identity. The separately tested live/history card fix is not claimed deployed on that host. On 9af, original saved settings survived reopening and fresh no-prompt Test authenticated catalog/model/binding; no new Save is claimed. Fresh Linux admission passed. With `claude-sonnet-4-5`, Artifact downloaded exact 27 bytes and matched 121 canonical identities in complete S3-served history. First Stop’s strict quiet FAILED on a 10.373-second gap despite independent cancellation/no late artifact. One targeted unchanged-sampler retry retained all 24 samples: 35.076 seconds quiet / maximum gap 4.584 seconds, 67 ordered canonical identities, and separate CANCELLED/artifact-absent status after the original 180-second deadline. The pre-reload duplicate settled card is disclosed. Exact-ID cleanup for the three new provider agents is awaiting authorization, so full grades and the fresh Artifact’s post-cleanup download remain blocked. A historical deleted-provider artifact was newly downloaded on 9af; original F226 Question/Plan/reconnect and cleanup proof keep their identities. Retained failures: stale fixture 132/133; E8 48/135 with 87 reset-hook failures; Mac 17 SQL failures/skips; earlier title/startup/custody/Stop/deletion failures with unknown causes/bounded recoveries; `dea023` Stop’s warning/50 duplicate identities; pre-repair ACK and original Core CI failures. WSP’s 251-assertion, 7/8-suite attempt failed physical PostgreSQL teardown; the unchanged wrapper rerun passed 128. Retry alone establishes no cause or full local-suite pass. All paid attempts remain recorded; vendor dollar cost is UNKNOWN. ## Risks Experimental: Cursor independently acknowledges neither the effective model nor instruction digest before the first prompt. Explicit values are submitted and checked through behavior; unpriced provider dollar cost stays UNKNOWN. Callbacks need authenticated reachable HTTPS. Scope, revocation, budgets, completion and cleanup remain authoritative. Operator Stop intent is separate from physical containment. Saved legacy agents retain their selected runtime. Prior shared title-save/deletion failures remain disclosed. ## Model Used OpenAI Codex, GPT-6 family, assisted with reasoning, repository tools, code execution, browser verification, and delegated code review. The serving model’s exact API ID and context window are not exposed. No value is inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used and the limits of available model metadata - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or described the issue in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch describes the change and contains no internal ticket ID - [x] I have run the scoped local tests reported above; pending gates are stated separately - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented the risks - [x] All required CI gates pass on the current head - [x] Fresh Greptile is 5/5 with no unresolved findings on the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1e3ab690b0 |
fix(routines): prevent recursive Sentry run repairs (#18714)
Validate Sentry failed-run alerts before routine dispatch. Reject repair-generated provenance and private sources. Deduplicate source runs through stable primary-key receipts, including setup retries and suppressed deliveries. Verified with 166 focused tests, server typecheck and build, generated-reference checks, passing required CI, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a92f616cd0 |
fix(db): attribute bounded captures to execution owners (#18634)
## Thinking Path > - Paperclip manages agent work through shared server and database services. > - High query volume can spend main-thread time on SQL construction and result decoding. > - The existing bounded response capture counts rows and bytes, but it cannot name the operation that executed a query. > - Bundled callers can be missing, and measuring every parsed JSON value adds work to the diagnostic. > - This pull request carries a semantic execution owner into each measured query and bounds response sizing. > - Operators can rank query volume by owner before choosing a performance fix. ## Linked Issues or Issue Description Refs #17504, #17522, #17537, #17564. **What existing behavior does this improve?** The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic. **Subsystem affected** Database diagnostics and existing server execution boundaries. **Current behavior** The capture groups by table, operation, and builder caller. It cannot distinguish execution owners that reuse a prepared query. Bundled deployments need source maps for caller locations. The capture visits every returned value and serializes parsed JSON again to estimate its size. **Proposed behavior** Version 2 captures use fixed semantic owners for HTTP requests, delivery queues, execution phases, and budget work. They count completed calls, returned rows, errors, and completion wall time. They sample successful results 1, 17, 33, and so on, with fixed cell and string/byte limits. Complex values remain explicitly unmeasured. Semantic ownership works without source maps. **Reason and benefit** A short census can identify which operations issue the most queries or return the most rows. It preserves each query's execution owner through pooled connections, concurrent operations, and prepared-query reuse. It reduces diagnostic work without claiming per-query CPU attribution. **Breaking changes** Opt-in capture output moves from `entry.v1`/`summary.v1` to `entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace all-result byte estimates. Consumers must use the new fields and must not extrapolate them into a total. There is no database migration or application API change. ## What Changed - Added `withDatabaseOperation(owner, action)` to the DB package. The server owns semantic labels. Queries retain execution ownership in their completion closure. - Added fixed owners at existing HTTP, delivery, budget, and execution timing boundaries. Delivery work replaces the context of the producer that woke it. - Added complete call/row/error counters and completion wall durations within the existing bounded window. Durations include waits and are not CPU measurements. - Sampled and bounded primitive-value sizing. Complex and oversized values remain unmeasured. Known owners skip repeated stack extraction. - Preserved the deadline, group limit, paced output, query outcomes, and existing coverage exclusions. Updated the operator documentation. - Added real PostgreSQL owner-isolation and prepared-query tests, server scope tests, and plain bundled-runtime proof. - Moved projection-test and benchmark full-response sizing into an explicit test helper. Existing payload-size thresholds remain unchanged; these synthetic checks do not use the production sampler. ## Verification - Final-commit DB capture suites: 43 tests passed in 13.86 seconds against the actual PostgreSQL driver, including concurrent owners, prepared reuse, transaction failure, a dedicated client, sampling, and a query crossing the deadline. DB typecheck passed. - Server attribution and existing delivery/execution/budget suites: 53 tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5 tests passed. - Runtime bundle suite: 8 existing tests passed. The new no-source-map owner test passed after its fixture was corrected to use a driver thenable. An ordinary native Promise can bypass an overridden `.then`; the real PostgreSQL tests separately cover driver integration. - Observer-only experiment: three alternating pairs per case on Node 24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an already-materialized synthetic JSON result, median capture CPU fell from 82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell from 8.10 to 4.40 microseconds/call. Every result retained identity and all calls were counted. The new diagnostic intentionally measures less byte data; these are observer costs, not database latency or a hosted application speedup. Disabled-case timings were below one microsecond/call and too small for a performance conclusion. - Final lifecycle regression: 28 unit tests and DB typecheck passed. Closing the last capture disables and replaces owner storage; overlapping captures retain it. - Final-commit full recursive typecheck passed in 38.38 seconds and full build passed in 40.81 seconds. The build stamp matches `83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution tests also passed 5/5 before the test-helper-only migration. Full test acceptance will use the repository's parallel CI gates; the unsharded local command takes hours. - Verification failures retained: initial sandbox runs could not open local IPC/PostgreSQL listeners; permitted reruns exercised the real services. One old test expected JSON serialization and was updated to assert explicit unmeasured complex values. The first bundle fixture used the wrong promise boundary, as described above. A retained-callback regression also demonstrated that disabling and reusing the same AsyncLocalStorage instance could resurrect an old owner; replacing the instance after the last capture closes fixes that case. - The first CI run found older projection tests consuming the previous full-response fields. Those consumers now use the full-response test helper, preserving their byte thresholds. The two complete affected route suites plus helper tests passed 115/115 in 73.08 seconds. A temporary full-row query negative control then failed the unchanged 2,048-byte limit at 5,000,812 bytes and was restored; five other service suites passed 34/34 and the affected chat cases passed 3/3. All three previously failing CI shards passed on the corrected head. A local skill-list benchmark smoke check completed 255 captured queries with no in-flight queries or overflow; this was integration verification under concurrent test load, not a timing benchmark. - Two final-head runner jobs lost communication with GitHub before reporting a test result. One targeted retry of those jobs and their dependent aggregate passed. All 53 final-head checks passed, with 3 intentional skips; previously passing suites were preserved. - Hosted capture remains pending until the reviewed revision is deployed and a bounded window is explicitly configured. Reproduce the focused checks with Node 24 and the repository pnpm version: ```sh pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs pnpm -r typecheck pnpm build ``` ## Risks - Owner context is active while a valid future capture window is armed, up to 30 minutes before accounting starts. No-window and expired calls take the direct path. Diagnostics still add bounded work while enabled. - A nearest nested owner is reported. Broad `http.request` and unattributed groups remain visible; this does not identify every service callsite or SQL shape. - The existing census excludes tagged/raw explicit-execute/streaming paths and internal protocol commands. Its counts are not PostgreSQL transaction totals. - Completion wall time includes queue/database/network/scheduling waits. Concurrent durations overlap. Sampled bytes exclude complex values, large strings, and values beyond the sizing limits. - Activation is startup-only. Operators must choose the window after artifact preparation, allow rollout/warm-up time, and remove the expired setting afterward. ## Model Used OpenAI Codex, GPT-6 family, with code execution, repository inspection, and parallel agents. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
006d5691a6 |
feat(server): support volume-free web server handoff (#18518)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
eb12cfc96b |
perf(github): activate connection work from events and deadlines (#18395)
Reviewed and verified at
|
||
|
|
ec3b558277 |
fix(server): schedule accepted native response recovery (#17696)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs store accepted results and their finalization decisions. > - A completed run can still owe its public answer after an interruption. > - Periodic recovery scans for that answer even when no new work exists. > - This pull request wakes recovery from committed changes and saved retry deadlines. > - The existing owner still verifies the result, source, and current publication authority. > - Settled or dormant response work no longer needs its own periodic scan. ## Linked Issues or Issue Description Refs #17690 (merged). This is slice 48 of the sleep migration stack, now based on master. Related: #17661 handles GitHub terminal publication; this change handles native accepted-response presentation. Governed-review presentation remains a separate duty. **What happened?** Native finalization recovery repeatedly checks already accepted results for a missing answer. Removing that check without complete notifications would lose responses when evidence arrives late, an agent resumes, or admission settles after execution. **Expected behavior** Recover the answer from its saved result when a supported change makes it eligible. Retain real retry deadlines. Stop querying when the answer is selected or cannot currently be published. Reconstruct work after restart. **Steps to reproduce** 1. Commit a native accepted result with no selected response comment. 2. Run finalization recovery while its source is unavailable or locked. 3. Restore that source or release the lock. The answer must appear once without repeating provider execution. ## What Changed - Honor recorded non-create decisions and skip-comment policy in both discovery and recovery. Apply durable conversation/result/continuation guards before recovery locks. Preserve genuine unanswered and legacy recovery. - Scope routine response wakes to the affected issue/run. Coalesced input names its active run; pending-effect settlement names its issue. Sixteen existing global authority/configuration hooks use one explicit company invalidation boundary. No new transaction wrappers or tables are added. - Add an accepted-response worker using the existing delivery coordinator and scoped deadline helper. Preserve saved retry deadlines and pending-effect fences under the owner's lock. - Share the finalization notification boundary. Wake on finalization settlement, exact accepted-result/final-message evidence, agent availability, retry admission, and existing chat authority changes. - Keep lifecycle policy in services. A narrow pre-write status hook lets lifecycle retain ownership of its transaction and status transitions. - Register the worker during enabled maintenance startup. Remove its authority subscription on shutdown. Periodic replay skips this duty on both committed and fresh-commit paths; ordinary finalization and explicit recovery retain their behavior. - Add a partial index and expression statistics for unpresented native runs. Migration 0412 supports concurrent replay and adds no queue table. - Add real PostgreSQL and lifecycle coverage. Preserve the implementation and producer audit in the plan document. ## Verification - Additional real Photon callback acceptance on the published production code disproves the security warning: ten active assertions cause zero response-recovery hints/SELECTs, while three concurrent assertions during an actual attention-to-active recovery cause exactly one wake/SELECT. The existing locked status guard already enforces this boundary. No production change was needed; the resolved finding now has a successful current-head security check. The regression will be retained with the next slice. - Eligibility repair source `892f36e5f66ba1af123034936a7160c64c3ab883` passes **1489 distinct server cases**: 1139 in six complete recovery/finalizer/monitor/summary/workspace suites (81.02 seconds), plus 350 selected chat cases (297.68s). Another 1463 chat cases are outside the pattern; the complete 1813-case file is not claimed. All eleven new regressions are included in that count. Five dedicated fresh PostgreSQL 17 databases migrate through unchanged 0412, with no fixture-source adaptations. Database build, server types/build, boundaries and full-SHA migration ordering pass. All seven test source hashes match. - The exact published baseline first fails eight behavioral assertions while the three genuine-answer cases pass. The same corrected test source passes all eleven after repair: recorded none/reuse remain suppressed; five ineligible history families take zero recovery row locks across repeated issue hints; missing/legacy/create-null answers still publish once; fresh periodic finalization defers presentation to its worker. Earlier fixture setup and variable/compiler failures remain explained in the audit, rather than erased by later success. - Repair source `2fc8b0614aaf9ac7337045de4ced7325f95edc45` passes **1821 server cases**: 1496 in 13 complete producer/lifecycle/authority suites (110.63 seconds), plus 325 selected chat cases (278.99 seconds). Coverage includes all native-session cases, queue admission, status decisions, membership/board access, the complete GitHub workflow, accepted-response lifecycle and real authority restoration. The other 1477 chat cases are outside this selection; the complete 1802-case file is not claimed for this repair run. - Actual PostgreSQL regressions first reproduce a coalesced-input hint missing its run scope and two routine hint paths producing 12 response-history SELECTs each after three notifications. The same assertions pass with run-scoped coalescing and zero response-history SELECTs for unscoped routine hints. Actual membership restoration still publishes the waiting response once. Assertion time limits are unchanged; this measures real driver SQL and persisted producer work. - Eleven dedicated fresh PostgreSQL 17 databases migrate through 0412. One embedded-only wake-queue fixture import temporarily uses its own external database and restores byte for byte. All 14 selected source files match pre-run hashes. This qualifies external-database behavior rather than reproducing embedded CI locally. - Master advanced independently with #18126's Pi archive retry fix. Normal merge `2a535bc61259ef99387369e8b32498f774e5ba2c` preserves both complete binary patches. All accepted server/database/UI/test/CLI trees stay identical; the three independent runner installer/manifest files are retained. All 13 installer tests pass, including stalled header/body cancellation. Combined server types/build, boundaries and full-SHA migration order pass. A subsequent normal merge of #18128 at `e8b78cce5bb557b086bee9ca930eeaab1d09b244` retains its three test-only file changes with both complete patches preserved. The changed HTTPS backfill case passes on a fresh PostgreSQL fixture (1 selected case; 178 outside the pattern), and combined types/build, boundaries and migration order pass again. All 14 repair source hashes remain unchanged. The independent browser fixture changes are preserved; no local browser run is claimed. - The earlier source `9feec170ef8174011fc3c5d3d6d76f0a0f39fea0` passed the 4171-case full baseline before this scope repair. That scope is preserved in the audit rather than represented as a current full rerun. All 0412 database bytes remain unchanged: original concurrent index/statistics/ANALYZE, prior executable history, journal entries and retained snapshots. Five snapshots 0408 through 0412 remain. The populated 0411-to0412 upgrade preserved 410 prior receipts and 38 companies, appended one receipt and none on replay; index/statistic checks passed. - The earlier P2 security failure, Greptile's former conditional dependency on #17702, and the subsequent actual Ponytail eligibility findings are recorded in `doc/plans/2026-10-25-native-response-presentation-sleep-audit.md`. This PR now includes the issue-scoped pending-effect notification fix itself, so it is independently correct before #17702. Fresh published-head CI, Greptile 5/5 and a clean security scan remain required. No deployed-sleep, external-wake or fleet-savings claim is made. ## Risks - The writer contract is process-local. Independent SQL writers and other replicas are not covered by these notifications. - Unpresented backlog reconstruction still has memory and time cost proportional to candidate count. Pages contain at most 100 rows; execution uses the shared fair batch. Selected history is excluded. Health-only signals do not trigger reconstruction. - Publication policy remains with its current owner. Digest, company/run binding, source authorization, selected-answer suppression, and ownership fences remain required. - The rest of native recovery and governed-review presentation still run separately. This PR does not relax Cloud sleep eligibility or add external wake coverage. - The stack parent has merged and its actual ancestry is reconciled. This publication still needs fresh current-head CI and review. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution, and PostgreSQL tests. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
DottaandPaperclip
|
5f0da562db |
fix(server): wake chat lifecycle milestones from committed changes (#17606)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a26d217e94 |
fix(server): coalesce Cloud membership synchronization writes (#17890)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud requests synchronize a human user's profile and company membership. > - Concurrent requests repeated the same writes before the debounce could record success. > - Upserts also updated unchanged rows and caused avoidable tuple work. > - This pull request shares identical pending writes and guards unchanged updates. > - Each request still reads current authority. Bounded profile lookup also reaches older memberships. > - The benefit is less database work during request bursts with the same actor results. ## Linked Issues or Issue Description **What happened?** A burst of 20 trusted Cloud requests issued 20 membership upserts and 40 profile plus membership tuple updates for one unchanged user. The completed-write debounce did not share in-flight work. **Expected behavior** Identical in-flight requests share persistence work. Distinct contexts retain their order. Failures remain retryable. Each request resolves its own current company and role authority. **Steps to reproduce** Run the synthetic PostgreSQL comparison below at baseline `db70c5acd92f50fa37fb4f0dea3da0e7685089a2`. The harness imports the real resolvers and counts actual writes with database triggers. Related work: #17663 covers adjacent Cloud authorization and projection behavior. This change preserves fresh imported memberships, owner-elevation settings and stale administrator removal. ## What Changed - Serialize synchronization per database client and user. Share only the last identical queued context. Preserve A/B/A transitions and retry after partial failure. - Guard user and membership updates with null-safe value comparisons. Retain default-grant repair when `RETURNING` is empty. - Keep authority reads outside the write debounce. - Resolve exact principal IDs through the existing index, then use at most one company-scoped alias query. This prevents both the old 200-row profile cutoff and repeated membership-page scans. Exact IDs take precedence over namesake aliases. - Add concurrency tests, PostgreSQL tests, a reproducible benchmark and behavior documentation. - Accept both Git 2.55 and 2.56 diagnostic messages in an existing workspace safety assertion, after full CI exposed that portability failure. No expiration behavior changes. ## Verification - Five focused suites passed, 63 tests total. These include Cloud actor concurrency, transient database retry, company provisioning, real PostgreSQL synchronization and the user profile route. - The auth-to-profile regression returned 404 before the lookup repair and 200 after it. The final fixture has 601 memberships and tests raw and normalized non-UUID IDs, missing slugs, alias ordering and exact-ID priority. A counterfactual with the old paging repair performs four membership queries for a missing slug; the final code performs two, each returning at most one row. Cross-company access returns 403 before lookup. - Local PostgreSQL 17.11 benchmark uses two warmup pairs and five alternating measured pairs. Each pair starts a fresh debounce scope. All 20 complete actor results match on every sample. - Burst median: 51.07 ms before, 25.71 ms after. Ranges: 47.21–63.25 ms and 24.17–29.32 ms. Membership statements: 20 to one. Tuple updates: 40 to zero. - Reproduce: `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-cloud-membership-sync.ts --baseline-ref=db70c5acd92f50fa37fb4f0dea3da0e7685089a2 --docker-postgres=17`. - Real PostgreSQL tests compare all four alias fields with the route's own JavaScript canonical slug output across 76 fixtures. Exhaustive Unicode lowercase inspection validates the special SQL mappings. - A real SQL_ASCII database reproduces the previous en-dash alias 404. The final whole-character replacements precede ASCII-only translation. Four alias fields and ten multibyte edge cases pass, including rejection of a false euro-to-k alias. The disposable database is migrated normally, then removed. - Server TypeScript passed with no diagnostics. - PostgreSQL tests used the normal migrations with a temporary Docker startup override because this host's native PostgreSQL semaphore allocation is exhausted. No tracked test infrastructure changed. - The first full CI run had an unrelated CLI company-import/export health-start timeout. Review also identified repeated profile scans; this follow-up repairs that finding. A later run passed 3,882 tests in server shard 4/12 and failed one existing assertion because Git 2.56 changed its error wording for a missing gitfile target. The safety behavior still refused expiration and preserved the workspace. The assertion now accepts both valid messages; the actual Git/PostgreSQL regression passes locally. Final head `3a644e600ab0ae7315f9828b1ea9dffdef407e4a` passed [full CI](https://github.com/paperclipai/paperclip/actions/runs/40537153522), including the previously failing shard. There are 55 successful checks/statuses, two skipped and one neutral. Fresh Greptile and Paperclip reviews are both 5/5 on this exact head, with zero unresolved threads. GitHub reports mergeable and CLEAN. - Full repository typecheck and build passed on the integrated final source. The local full test command was stopped after native PostgreSQL initialization failed repeatedly, producing 16 failures and skipped database suites. It is incomplete, not a pass. Focused real-database suites used disposable Docker PostgreSQL. Current-head CI runs the full gates separately. ## Risks - Coalescing is local to one process and database client. Other replicas can still take conflict locks. - `updated_at` now records actual profile and membership changes. Access lists sorted by it can show a different order. Profile resolution no longer depends on the first 200 entries. Exact principal IDs take precedence over namesake aliases. The single alias fallback still scans company candidates; no constant-CPU claim is made. - Default role grants retain their existing additive semantics. This change does not define a new demotion policy. - Local synthetic service timings exclude HTTP and network latency. No hosted deployment or production speedup is claimed. - No migration, startup job, polling change or deployment is included. ## Model Used OpenAI Codex, configured model `gpt-6.1-sol` with xhigh reasoning, tool use and code execution. Independent blast-radius review also used `gpt-6-astra` and `gpt-6.1-sol`. The runtime context window was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
9f3a5456ac |
perf(server): bound run reads in authentication and conversation recovery (#17663)
Read compact run identity during authentication and compact turn state during conversation recovery. Preserve fresh checks, durable steering and reply evidence, locking, and complete stored payloads. Validate 184 targeted PostgreSQL and lifecycle tests, workspace typecheck, server build, full CI, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
79b2bf78dd |
Create agent records when onboarding names are submitted (#17005)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core onboarding selects an agent and connects its provider account. > - The current flow creates the agent record after authentication. > - Setup needs a stable agent identity as soon as the user submits its name. > - This PR saves a paused setup record on name submission and finishes the hire on that same record. > - The existing sandbox authentication and approval paths continue to apply. ## Linked Issues or Issue Description **Subsystem affected** Core agent lifecycle, shared validators, and onboarding UI. **Current behavior** Onboarding creates the database record at the final hire step. Earlier authentication and saved setup have no agent record. **Proposed behavior** Create a paused agent with a setup hold when the user submits its name. Do this before adapter selection or authentication. Update that same record after adapter selection. Reuse its ID on retries and saved setup. Finish that record through the existing hire path. **Reason and benefit** Give setup a stable identity without coupling this refactor to computer provisioning or a new execution backend. **Breaking changes** Incomplete setup now leaves an inactive agent record. Existing clients can still create and hire agents without the new optional fields. The sandbox authentication flow and API key support remain in place. **Additional context** Requested Core-only refactor. Related work: #15422 changes runner defaults. This PR preserves those choices. No duplicate early-record refactor was found in the issue and PR search. ## What Changed - Save the agent when the user submits its name in both onboarding entry points. Use a provisional adapter until the user chooses one. Reopen name-only drafts at adapter selection. - Add retry-safe setup creation and same-record hire completion. Keep approval checks at final hire. - Save and restore setup configuration in both onboarding entry points. Show Finish setup for paused setup agents. - Restrict setup changes to authorized board users. Block generic resume until setup finishes. - Add lifecycle, authorization, connection, and UI coverage. Update the specification and feature maps. ## Verification - Latest focused server run: 126 lifecycle and AI-connection tests passed after the Apex fixes. An earlier broader run passed 301 lifecycle, instruction, adapter-validation, and AI-connection tests. Earlier authentication, retry, cross-company, and direct-create regressions also passed. - Name-save verification: 336 tests across the wizard, naming dialog, New agent, action-button, and wizard-step suites passed. Coverage includes retry identity, repeated submission, closing during a save, saved-model restoration, and a disabled process adapter. - Shared, server, and UI TypeScript checks passed. The latest UI package check and direct server TypeScript check passed. The server package preparation command could not finish locally because Cargo is unavailable; full build preparation runs in CI. Map checks, token gates, and `git diff --check` passed; each pushed diff was scanned for credentials and PII. - All six onboarding browser tests passed together with installed Chrome against a fresh throwaway instance after the Apex fixes. They include numbered duplicate names, first-time Copilot without a connected account, and Grok’s saved model. They verify early persistence in both entry points, closing and reopening a name-only draft, same-record adapter selection, sandbox sign-in, login reload, and failed-connection gating. Provider responses are simulated; Core persistence is real. - Regression coverage includes incomplete Copilot creation versus final validation, Kimi settings through redacted responses, stale local drafts, company switches during creation, saved environments, and provider changes with personal/shared old bindings. - Apex found five name-only setup gaps: numbered names, early Copilot account validation, the Grok model default, same-name draft ownership, and provisional runner restoration. The fixes preserve server names, defer implicit account binding until hire, set Grok’s model, avoid name-based reuse of unfinished records, and restore Auto for pending adapter choices. UI, database, and browser regressions cover these cases. - Standard Greptile found a Grok provider mismatch in the new adapter save. The save now uses ACPX with the Grok profile. Both the 129-test dialog/setup run and a real-API browser regression passed after the fix. The first browser attempt stopped on an ambiguous empty-roster button selector; it now scopes the click to the toolbar. - Moving the save point exposed a saved-model reset and timing assumptions in the same-name and portal fixtures. The source choice now preserves the restored model, and tests cover same-name recovery without a second hire. - Initial CI exposed direct-create compatibility, concurrent setup effects, fixture assumptions, and a missing-fields assumption in the provider-validation guard. These were fixed and verified locally. Review findings were addressed with authorization, recovery, credential-persistence, and restored-configuration regressions. - The final desktop capture passed after host resource cleanup. A repeat first failed before Core started because macOS had exhausted shared-memory IDs. Standalone PostgreSQL initialization reproduced the same error. Releasing only unattached control segments from this task’s stopped test databases allowed the unchanged revision to pass. No Core or system setting changed. - Local database tests required port access outside the sandbox. Missing declared development dependencies were supplied outside the branch. No live provider sign-in was tested locally. - [Full monorepo CI](https://github.com/paperclipai/paperclip/actions/runs/39522434236) is green on `82b82fec189a172a885d4dd565ae2f4a18c73efc`. Standard Greptile scored 5/5 with zero new comments (check `118969424903`). The subsequent Apex review also scored 5/5 with zero new comments (check `118972866331`) on that same commit. All review threads are resolved. - The separate Superagent Security scan returned neutral/inconclusive because its pinned merge-base checkout failed or timed out. GitHub returned 404 for check and suite rerun requests. This is not a successful security scan; the CI workflow and other security checks passed. ## Risks - Users can leave inactive setup records behind, including after name submission alone. These records cannot run until hire completes. - Setup completion touches hiring and approval boundaries. Tests cover retries, company isolation, creator permissions, and approval snapshots. - An uncertain credential-save result retains its new secret reference for retry. A confirmed client rejection removes only credentials staged by that attempt. - No schema migration or VM provisioning change is required. Agent-computer integration is out of scope. ## Model Used OpenAI Codex, based on GPT-6, with repository search, code editing, and tool execution. The exact deployment ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Apex Greptile is also 5/5 on the same commit as standard review and green CI - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
505e1a5582 |
feat(cases): add persistent workflows with same-task agent handoffs (#16959)
## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
92b238137b |
Connect and configure GitHub Copilot through Apps (#15705)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps and AI Connections store credentials and govern access to agent runtimes. > - Users need a supported saved-token connection and config values that survive save and reopen. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will connect Copilot through the normal Apps and AI Connections path. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701 and #15703. Supersedes setup work in #15562. Base: #15703. Next: #15706. ## What Changed - Require configuration update authority before Copilot credential-bearing model discovery; default to the saved environment and preserve explicit selection for editors. - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Add GitHub AI token connections with encrypted storage, ownership, grants and audit logging. - Keep repository access separate from Copilot runtime authentication. - Probe the selected environment without sending a model prompt and preserve authenticated model selection through save/reopen. - Keep fresh execution admission consistent with normal Copilot setup; persist task model choices without changing provider or permission policy. - Refresh authenticated models after reconnecting the same saved Copilot account, including recovery from a previous authentication error. - Apply the additive provider-default constraint migration through the normal generated migration workflow. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. **Remaining:** current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks Bad credentials, missing entitlement, runtime installation and unavailable models must remain distinct failures. A connection must not adopt ambient credentials. The constraint migration changes accepted provider values, with no credential-storage migration. Current-master NewAgent fixture extraction is retained. Copilot cases use that shared support and test both passive discovery and explicit reconnect under the same setup lifecycle. The onboarding retry regression shares the component suite’s saved-connection, authenticated-model, draft, agent-list, hire and render lifecycle fixtures. Keeping it beside the existing onboarding failures makes adoption versus duplicate hire observable under the same lifecycle; this is the concrete cohesion reason for the existing large test file. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d9bcf4dfe8 |
fix(auth): let Cloud stack admins manage experimental settings (#16907)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People use experimental settings to opt into product features. > - Cloud stack Admins can manage their company, but cannot save an experiment. > - Requiring an Owner role adds unrelated privileges to this workflow. > - This pull request gives trusted Cloud stack Admins permission to update experiments. > - Other instance administration and platform settings keep their existing checks. ## Linked Issues or Issue Description **What happened?** A person with the Cloud stack Admin role receives “Instance admin access required” when they enable an experimental setting, such as Fast responses. **Expected behavior** Cloud stack Admins can save editable experimental settings. This permission does not grant general instance-admin authority. Members, Support users, and agents cannot change experiments. Cloud-managed and operator-hidden settings remain protected. **Steps to reproduce** 1. Sign into a Cloud staging workspace as a stack Admin. 2. Open Settings → Experimental. 3. Enable Experimental fast responses. 4. Observe the permission error instead of a saved setting. **Additional context** Related: #10343 introduced computed Owner elevation. #15748 added Fast responses and the company Connections settings page. The roadmap and existing PRs were checked. This is a small authorization fix for the existing settings path. ## What Changed - Derive experiment-management authority from the current trusted Cloud stack Admin role on each request. - Apply that authority only to the experimental settings PATCH route. Keep general settings, environment defaults, and task drain restricted to instance admins. - Preserve existing hidden-setting checks, managed feature restrictions, and activity records. - Add role and role-downgrade regressions, allowed and denied route cases, staging coverage, and updated OpenAPI and access documentation. ## Verification - Before the fix, the Admin fast-response request returned 403 and the new role assertions failed. The first sandboxed HTTP run could not bind local sockets; the regression run outside that sandbox reached the expected authorization failures. - Focused Cloud actor, settings route, managed-overlay, and health tests: 149 passed. - `node scripts/check-maps.mjs` and `git diff --check` pass. - `pnpm -r typecheck` and `pnpm build` pass. - Greptile reviewed commit `7c33b90066121f03183579d2fae2650755e025e4` at 5/5. Its one style finding is fixed and resolved. - The full local `pnpm test:run` did not pass. It reported two early collection failures before workspace build outputs were ready and a 15-second timeout in the unchanged AgentMail recovery test. The run was stopped after those failures while the full CI suite continued. No full local-suite pass is claimed. - The first current-head CI attempt failed the unchanged Runner test “preserves a real listening process and its marker when a health RPC times out” with `runner_remote_process_identity_unavailable` / `process_identity_changed`. The test passes in a local isolated run. This recovery does not establish the CI failure's cause. [Failed job](https://github.com/paperclipai/paperclip/actions/runs/39332989655/job/118336142261). The failed shard passed on its single rerun. The failure cause remains unconfirmed. All CI checks on commit `7c33b90066121f03183579d2fae2650755e025e4` are now green. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/39332989655). - No deployed staging behavior is claimed. Manual acceptance after deployment: as a stack Admin, enable Fast responses, reload, and confirm Connections appears. Confirm Members and Support users still cannot save experiments. ## Risks - This intentionally extends experimental settings write access to Cloud stack Admins. It does not persist an instance-admin role or expand other instance controls. - Production retains its existing managed feature policy. Staging retains its editable experiment allowlist and protected execution and owner-elevation settings. - No database migration or UI change is required. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser inspection. The runtime does not expose an exact model snapshot or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac9459bae3 |
feat: configure company agent observability (#16874)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Companies use several agent harnesses and execution environments. > - Operators need one place to select where native agent logs and traces go. > - An inference proxy would add a dependency to the agent request path. > - This pull request supplies native OpenTelemetry settings to managed Codex and Claude Code processes. > - Company secrets protect destination credentials. Export failure does not change the task result. ## Linked Issues or Issue Description **Subsystem affected** Company settings, shared contracts, database, server, adapters, and runner. **Problem or motivation** An operator cannot configure a company destination for native agent logs and traces. Manual harness configuration is hard to apply across local and remote execution. **Proposed solution** Add one company destination: Braintrust or a custom OTLP HTTP/protobuf endpoint. Apply a transient snapshot when execution preparation starts. Keep credentials out of API reads, task environment manifests, recovery files, and command arguments. Replace a warm process when its export settings change and preserve its conversation. **Alternatives considered** Multiple destinations can use a user-operated collector for fan-out. This change does not add a collector, an inference proxy, or transcript scraping. **Roadmap alignment** This extends company settings, secret management, and support for multiple harnesses. The maintainer requested this implementation. Searches for observability and Braintrust found no duplicate company destination feature. Related infrastructure tracing work in #16236 has a separate content-free contract. ## What Changed - Add strict schemas, revision checks, encrypted secret storage, activity records, and board-only settings routes. - Add explicit synthetic logs and traces tests with a bounded HTTP request and sanitized results. - Supply native Codex configuration and Claude Code environment values through local and remote launch paths. - Preserve Codex login refresh through the existing managed credential link. Retain unsaved drafts across background reads. Require complete custom header re-entry when the endpoint origin changes. - Separate process export compatibility from conversation identity. Isolate generated Codex configuration and preserve resumable sessions, including remote home refresh after export changes. - Advance the immutable Pi/Copilot source attestations for shared environment and sidecar code changes. Keep their provider payloads unchanged and retain prior identities for decoding. - Redact destination credentials throughout launch diagnostics and HTTP logs. - Add the production settings form, Storybook states and journey, documentation, and feature/access map entries. ## Verification - Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks exported to Braintrust. API readback and the Braintrust UI showed the test records. A real remote Daytona Codex task exported through the legacy `exec` path. - Live tests exercised content capture off, export disabled, warm conversation reuse, and a destination that accepted connections but never replied. Agent work succeeded and export requests stopped at the configured three-second timeout. The UI connection test returned “Test payload accepted”; the saved API key stayed masked after reload. - All current-head CI test lanes, typechecks, packaging/build checks, and required aggregates passed on `71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)). Greptile completed on that exact commit at 5/5, with zero unresolved review threads. The additional optional Paperclip Review check remains queued waiting for its assigned agent. - Local workspace typecheck and build passed. Focused verification included 239 combined runner tests, 789 API/native-session tests, 107 Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the affected Rust suites. Migration, token, map, Storybook catalog, and Storybook build checks passed during acceptance. These checks ran at the relevant implementation or merge boundary; the current-head CI is the complete final gate. - Managed AI credential projections now receive enabled and disabled company export settings; genuine caller-owned Codex homes remain untouched. Two regressions reproduced the skip before the repair. All 276 engine/lifecycle tests and adapter-utils typecheck pass afterward. The launched TOML retains provider/MCP settings, credential continuity, and redaction. - Settings saves, removal, and the real company-deletion owner acquire secret mutation ownership before database locks. PostgreSQL NOWAIT probes cover all three paths; concurrent save/rotation/removal/deletion requests settle correctly. The affected suites passed 137 tests, followed by all 32 service/connection cases after expanded deletion coverage. Server and Codex adapter typechecks pass. - Codex source profiles use the effective CLI argument list. A spawned fixture CLI reproduced both ignored-argument failures; all three precedence/fallback cases now pass. - Real tar staging reproduced remote conversation loss in both API-key and subscription modes before the repair. Both now pass: changing managed settings preserves sessions, archives, native indexes, and refreshed subscription credentials. A corrupt upload preserves the prior config and conversation. - Preserved failure evidence: initial CI caught stale Pi/Copilot source attestations, a missing test-harness helper, and packaging fixture/probe mismatches; those were corrected and covered by focused checks. Superseded heads had intermittent attachment-composer timeouts despite successful uploads; [the original trace and screenshot remain available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203). The same attachment case passed on the final head: [25/25 tests in shard 3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051). - A connection-fault fixture also exposed a stale pooled socket after its hang case; closing fixture responses isolated scenarios and the full affected suite passed. - The broad local test run was not clean: database resource failures, timeouts, an older cached module graph, and a stale optional-resource manifest assumption were recorded. Focused checks passed after the documented fixes. This does not establish a clean full local pass. - Native remote Codex, remote Claude, and the full ACPX Codex bridge remain unqualified. The direct Codex resume test does not establish full bridge acceptance. ## Risks - Native exporters perform network requests from each execution environment. A server connection test does not prove remote reachability. - Prompts, assistant responses, and tool content are enabled by default when supported. Native caps and harness-specific controls still apply. - Caller-owned Codex homes are unsupported for automatic configuration and remain untouched. - Native harnesses and their tool processes can inspect export credentials. The settings form and documentation advise using ingestion-scoped credentials. Retained headers remain bound to their saved scheme, host, and port; changing origin requires fresh values or removal. - Migration 0357 adds a company-scoped row and secret reference. It is safe to reapply. Secret removal and company deletion use the existing secret lifecycle. - Slow or unavailable destinations must not block task settlement. Export settings take effect at the next execution preparation boundary. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and browser verification. The exact serving model ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
51c81d0129 |
feat(sentry): link run errors to task and run pages (#16803)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use Sentry to inspect run and server failures. > - Reports identify the run and task, but the existing links open API records. > - Operators need a direct path from an error to the normal task and run pages. > - This change adds navigation links through the shared server Sentry hook. > - Saved company and task records determine the links. A failed lookup preserves the error report. ## Linked Issues or Issue Description Related work: #16300 added saved run API references. **What existing behavior does this improve?** Sentry event context for run failures, Stop timeouts, and server errors from an authenticated agent run. **Current behavior** Reports show task and run UUIDs. The existing run URL opens an API record. The operator must find the normal task page separately. **Proposed behavior** Add `task_url`, `run_url`, and `task_identifier` tags when saved run metadata is available. Add `taskUrl` and `runPageUrl` to the event context. Use the company prefix and saved task identifier in normal board URLs. **Reason and benefit** An operator can open the task or run directly from the error report. ## What Changed - Add a shared Sentry hook for events with a known run identity. - Read only the saved company prefix, task identifier, and run identity. Keep the lookup within the recorded company and agent. - Include authenticated run identity on request errors. Use the existing run identity on Stop errors. - Bound optional lookups to one second and eight unfinished reads. Preserve the original event on failure. - Keep the existing API references and error grouping. - Document the fields and add SDK, timeout, and company-boundary coverage. ## Verification - Passed: `pnpm -r typecheck` and the final server TypeScript check. - Passed: `pnpm build`. - Passed: 332 tests across the affected server suites, plus 25 final link and real Sentry SDK tests after review fixes. The SDK test checks outgoing event envelopes for run failures, request errors, and Stop errors. It checks that unrelated errors receive no links. The database test verifies saved metadata and company boundaries. It passes with embedded PostgreSQL and with an explicitly configured disposable test database. - Passed in CI on `bec0cdac853efdef71041d59a66261941827653e`: the full test matrix, typecheck, build, Runner checks, browser suites, and the real Sentry SDK contract. All 56 active checks pass. Two Storybook jobs are skipped. - The full local `pnpm test:run` attempt first failed because PostgreSQL library links were missing. The package setup script repaired the install. The affected database suites then passed. The repaired full local run was stopped after all CI test jobs passed; it did not complete locally. - Greptile: 5/5 on the final commit. All three review threads are resolved. - Read-only browser inspection confirms that Sentry displays context URL values with an Open link control. - No live test event was posted. Validation uses the real SDK with an in-memory transport. ## Risks - A slow or unavailable database can prevent optional links. It does not discard the error. At most eight unfinished lookups can remain active. - Missing or unsafe public origins omit the links. URLs longer than the tag limit remain in context. - This change applies to newly captured events. It does not update existing Sentry events. - The change adds no schema, permission, or first-party Telemetry contract change. ## Model Used OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning, with tool use and code execution. The exact context window is not exposed in the session metadata. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e6dcc6b095 |
Connect personal Muse through secure native execution and Vault OAuth (#16448)
Add company-scoped Muse native control, OAuth, wake receiver and recovery. Recheck current authority after waits and before effects or receipt replay. Preserve task-read scope, assignment expiry, independent holds, nonblocking reassignment and truthful cleanup provenance. Retain source trust on progress comments and verify current Muse V8 authority through real signed-JWT HTTP requests. Preserve the upstream trimmed ordinary authentication lookup. Refs #16448. Preserve captured canonical queue recovery and concurrent environment preparation while retaining the scoped Muse V8 exclusion. Normal migration 0354 and existing Muse SQL remain unchanged. Fresh current-head Greptile review is 5/5 with the project-write finding addressed. Required CI and independent technical review gate the merge; maintainer override is reserved for formal approval rules. Recheck current Muse project-write authority after all receipt/task/run/admission waits, before creation or replay. Real PostgreSQL regressions reproduce four late effects/replays on the old route; the repaired three-file treatment passes 354 cases with no skips, and the scoped server typecheck passes. Ordinary/Dot behavior is preserved. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
63fbb714e6 |
fix(server): release idle tracking after aborted static UI files (#16685)
## Thinking Path > - Paperclip runs work for AI agents. > - Hosts must prove that accepted work has finished before idle sleep. > - HTTP request tracking keeps unfinished writes active after a client leaves. > - Static file middleware has no returned completion promise. > - An aborted UI download can leave its request active forever. > - This PR completes only a matched file response and keeps other work tracked. ## Linked Issues or Issue Description Refs #15522. Found while qualifying #16679. A search found no duplicate static-download idle fix. **What happened?** Closing a client during a static UI download leaves one tracked request active. The response has closed, but the file middleware never calls the wrapped response end method. The instance cannot report idle after that request. **Expected behavior** A closed, matched static file response must not leave a permanent work hold. Unfinished application writes must remain tracked. A missing file must keep the normal fallback and error paths. **Steps to reproduce** 1. Serve a large local file through the static UI middleware with idle admission enabled. 2. Start the download over HTTP. 3. Disconnect the client after its first data chunk. 4. Observe that the tracked request remains active with the original middleware. **Paperclip version or commit** Reproduced on the behavior in `d30648dee89720358271c276b9fc28cf83f18be7`. This branch starts at `cfa9dce295970a85b3d8ee14cab0eccf50bc72e2`. ## What Changed - Mark the completion of a known read-only response without dropping pending middleware promises. - Wrap static UI file serving. A successful file match is required before a closed response can complete its tracking. A late match rechecks an already closed response. Missing files keep the normal fallback. - Use the wrapper for hashed assets, other static UI files, and public development files. Preserve caller headers and cache options. - Test both HTTP abort timings, nested routers, conservative mounted-application holds, parent writes, mutating GET fallbacks, and callback-only work. - Update the idle contract and its verification map. Correct the paused-message recipe from hosted qualification: an explicit Pause cancels older queued work, so test new accepted input after the pause and await the normal resume lifecycle. ## Verification - The original two real stream-abort regressions fail with the original Express static behavior. - Three review regressions fail on `e260f4694afab8447b0cb203f439cb3ccad254e8`: a client closing before file lookup, an unfinished parent in a mounted application, and an unfinished parent in a root without installed handler tracking. Fixed sources were restored byte for byte after the comparison. - Two focused suites: 16 passed. They cover nested routers, both abort timings, hidden parent writes, late missing-file fallbacks, contents, HEAD, conditional responses, and caller headers. - Module and map checks pass. Map checks are structural only. - `pnpm -r typecheck` and `pnpm build`: passed on `f242ee739344484995d3addf81635512370778c8`. - Full CI matrix: 54 successful checks and 2 intentional Storybook skips. - Apex: 5/5 on the exact current head, with zero unresolved threads. - Full local maintained test groups are still running before final handoff. - The initial local broad run was stopped before changing source for the P1 review findings. Two file-mode fixture failures in that run came from the scratch runner's inherited `0077` mask; the same suite passes with the normal child test mask `0022`, without source or host configuration changes. The replacement run uses `0022` for test children and command-scoped `init.defaultBranch=master` for existing Git fixture assumptions. - This PR has not been deployed to a hosted environment. ## Risks Only a matched file served by the admitted application with installed handler tracking can complete this way. Routers share that scope. Mounted Express applications and roots without handler tracking keep the existing conservative response hold; this change does not claim their hidden writes are tracked. An unfinished tracked parent promise still blocks sleep after a file client disconnects. Redirects, missing files, application callbacks, and other GET requests retain their guards. No authentication, company scope, provider cleanup, or scheduling gate is removed. No schema or API response changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source editing, and shell tools. The session does not expose a more specific serving model ID or its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
4de6335c3b |
perf(server): trim saved run JSON from agent authentication (#16577)
## Thinking Path > - Paperclip coordinates AI agents that work for companies. > - Agent JWT authentication checks each request against its saved run. > - Saved runs can contain large prompts and result payloads. > - The first run lookup needs only identity fields, cancellation state, and the conversation flag. > - Reading all saved JSON transfers data that this lookup does not use. > - This PR selects the required JSON fields in PostgreSQL and preserves the existing authorization decisions. ## Linked Issues or Issue Description **What existing behavior does this improve?** The initial saved-run lookup in agent JWT authentication. **Subsystem affected** Server authentication and database queries. **Current behavior** The lookup transfers the full context and result JSON on each request. It uses only the cancellation state and conversation flag from those values. **Proposed behavior** Select the required JSON fields in PostgreSQL. Keep the run, company, and agent predicates. Read current state on every request. **Reason and benefit** A synthetic lookup with a 96 KiB prompt and 4 KiB result returns 102,520 JSON bytes before this change and 134 bytes after it. This reduces data transferred by this query. **Breaking changes** None intended. Identity fields, write revocation, read access, and cancellation response codes retain their current behavior. Related: #9280 addresses authentication round-trip counts. This PR only reduces the data returned by the existing initial run lookup. ## What Changed - Extract the initial run lookup and select only the JSON fields that authentication consumes. - Add real PostgreSQL tests for legacy JSON shapes, fresh cancellation state, payload size, and run/company/agent scope. - Add a reproducible benchmark that loads the original query from an explicit baseline commit. - Document the lookup and benchmark. ## Verification - Baseline: `06689c1b3410000bcd65ff40cb469e600209c4db`. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Final server typecheck: passed. - Final-head GitHub checks: all green at `2be1c38e6bf51616b69993cf84cee1eef9657517`. Apex: 5/5, zero unresolved threads, conflict-free. - `pnpm check:module-boundaries`: passed. The standalone synthetic benchmark now uses the existing root benchmark directory and resolves Drizzle through the server workspace. - Focused HTTP middleware and database tests: 29 passed on real Homebrew PostgreSQL 16.14. The SQL test compares 270 combinations of status, cancellation JSON, and conversation JSON. It also checks fresh state and company/agent/run scope. - Broad native verification: 873 passed across the HTTP middleware, SQL projection, run diagnostics, and native executor suites. - The default local full command completed its general server lane with 12,090 passed and 8,885 skipped tests. It could not finish the root workflow: the host could not start the bundled PostgreSQL; a diagnostics assertion matched a directory name; and Git fixtures retained the host default branch. These failures remain in the verification record. All 68 failed files now pass on replay with disposable Homebrew PostgreSQL, an isolated checkout, and command-scoped Git defaults. The 68-file replay passed 2,588 tests and left two email skill-resolution failures. Moving the checkout outside an unrelated ancestor skills directory resolved those last failures: all 44 tests in that file passed. Product code and test assertions were unchanged for these host adjustments. Benchmark command: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-agent-authentication.ts --baseline-ref=06689c1b3410000bcd65ff40cb469e600209c4db ``` The final benchmark ran at treatment commit `2be1c38e6bf51616b69993cf84cee1eef9657517` with a clean worktree. The local run used PostgreSQL 16.14 and the maintained test constructor seam to start disposable Homebrew clusters. The bundled PostgreSQL binary could not start on this host. Both versions use the same schema and predicates. Each case has two warmup rounds and ten measured rounds with alternating order. Random content prevents unrealistic compression. Complete authentication decisions match. | Context / result | Before JSON bytes | After JSON bytes | Before median | After median | | --- | ---: | ---: | ---: | ---: | | 1 KiB / 4 KiB | 5,240 | 134 | 0.157 ms | 0.157 ms | | 96 KiB / 4 KiB | 102,520 | 134 | 0.331 ms | 0.140 ms | These are local query round trips, including result transfer and decoding. The small control has no material latency gain. The large case improves by about 2.4x. Both plans read the same single row. The primary result is reduced response size. This does not measure HTTP latency, server CPU savings, or fleet-wide cost savings. Full samples and plans are retained with the verification evidence. ## Risks - JSON types must retain their meaning. The real database tests compare the consumed authorization contract with the original projection, including SQL null, JSON null, scalars, arrays, objects, string booleans, and cancellation states. - The change covers the first JWT run lookup. Later run identity capture keeps its existing full-run reads and transaction locks. Those reads remain a separate cost. - The change adds no schema migration or authorization cache. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, and shell tools. This session did not expose a more specific serving model ID or its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
514d8738cf |
fix: keep the JWT run-mismatch audit when the claimed run is missing (#13183)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Rejected agent JWTs whose run header disagrees with the signed claim are audited into `activity_log` so operators can see spoof attempts > - The audit insert writes the claimed run id into `run_id`, which is a real foreign key to `heartbeat_runs`; a bogus claim violates the FK and the catch swallows it, so the audit row is silently lost exactly when it matters most > - This pull request verifies the claimed run exists before trusting it into the FK column; the claim itself stays in `entityId` and `details` > - The benefit is a reliable audit trail for rejected credentials ## Linked Issues or Issue Description Refs #10498 — the anonymous fall-through half was fixed on master already (every failed-credential branch now returns an explicit 401/403/422); this closes the residual FK-swallowed audit defect. ## What Changed - `server/src/middleware/auth.ts`: `auditAgentJwtRunHeaderMismatch` looks up the claimed run id and populates `runId` only when the row exists; `entityId` and `details.claimRunId` carry the claim either way. - `server/src/__tests__/agent-auth-middleware.test.ts`: a mismatch whose claimed run does not exist still produces the audit row with `runId` omitted. ## Verification - `pnpm --filter @paperclipai/server vitest run src/__tests__/agent-auth-middleware.test.ts` — 17 tests pass. - Before: a run-header mismatch whose claimed run id was absent from `heartbeat_runs` violated the `activity_log.run_id` FK, and the catch swallowed it — no audit row. After: the row lands with `runId` null and the claim preserved in `entityId`/`details`. ## Risks - Low risk. One extra indexed select on a rare rejection path; the audit payload is unchanged apart from `runId` now being conditional on existence. ## Model Used - Anthropic Claude — SWE-2 Max agent via Devin CLI, tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Waseem Ilyas <1478353+Waseemilyas@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3a440d28c4 |
fix(server): read existing grants before seeding role defaults (#16365)
`insertMissingPrincipalGrants` now reads the existing permission keys first and inserts only missing grants. In the steady state no INSERT runs, so cloud-tenant auth resync no longer waits on row locks held by concurrent grant edits until the statement timeout (Sentry PAPERCLIP-BACKEND-3S). Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
b8ffeb973a |
feat(runner): add experimental Grok Bot agent participation (#16219)
## Thinking Path > - Paperclip manages AI agents that perform work for a company. > - The shared Runner owns task admission, authority, receipts and finalization. > - An existing Grok Bot needs to receive assignments and return results through that Runner. > - The Bot also needs governed access when it starts work from its own conversation. > - This PR adds agent-purpose MCP access and routine webhook dispatch. > - It reuses the external-agent Rust Runner and keeps Grok capabilities explicit. > - The provider stays experimental until the real-client managed journeys pass. ## Linked Issues or Issue Description **Agent or provider** Grok Bot, distinct from the Grok Build coding runtime in #13882. **Why this adapter is useful** Users can assign work to an existing Bot, receive progress, questions, documents and files in Paperclip, and let the Bot initiate work under its agent permissions. **How the agent is invoked** A webhook-only routine delivers bounded references. Dedicated named-agent OAuth and MCP provide inbound calls. Setup follows [Grok routines](https://cursor.com/help/grok-bot/routines) and [custom plugins](https://cursor.com/help/grok-bot/connect-plugins). The current desktop app needs its Bot-generated Webhook URL navigation link when routine-row clicks fail. The GitHub search found related Grok Build runtime work, but no existing Grok Bot agent provider. This extends the roadmap’s existing agent, Runner, governance and artifact foundations. ## What Changed - Add default-off Grok Bot participation through the shared real Rust external backend, preserving native provider identity guarantees and distinguishing session closure from remote termination. - Bind agent OAuth to company, agent, client, gateway and authorizer; recheck permissions and task-bound execution authority without manufacturing a native identity. - Persist routine configuration, dispatch intents, authenticated acknowledgement and recovery holds. Store URL/key together in encrypted secret storage, use guarded public egress, and never automatically replay ambiguous HTTP delivery. - Reuse native tools, durable waiting, completion assessment and review. Add task-bound file grants through existing artifact storage; suppress proactive echo dispatch and synchronous result wake loops. - Enforce the current native deliverable work-mode policy at file-ticket issuance and use. Add operator retries for confirmed rejected wakes, with retained receipts, stable retry intent and current-authority checks. Recovery locks bindings before routine rows. Thread Stop resolves the selected run and its agent. - Cancel queued and preparing Stop targets before assignment creation, restore saved setup after a failed request and successful retry, and publish the Grok provider descriptor schema. Advance the versioned Copilot source declaration for the shared schema change; its qualification remains pending. - Filter task summaries with the shared privacy policy, preserve feedback admission when wake delivery is blocked, reconcile settled delivery receipts without restoring work authority, and bind every Stop control to its displayed run. Recovery records distinguish dispatch uncertainty from a revoked session. - Add resumable guided setup, credential replacement, consent, management, observed work status and stop/recovery controls, plus an interactive Storybook journey using the production task thread, question form and report card. Grok’s own apps remain available; Paperclip workspace and third-party tools are deferred. ## Verification - Five focused protocol, authorization, person MCP and Runner files: 192 tests pass. The Grok assigned-work case executes the actual Rust Runner, saves a document, uploads/deduplicates a file and reaches ordinary finalization. - Proactive intake/deduplication and renewed-session recovery pass. These use scripted clients and synthetic storage; they are not real-provider acceptance. - Repository typecheck, production build, token gates, map checks and Storybook build/catalog checks pass. The local full-suite campaign was stopped after macOS shared-memory exhaustion repeatedly prevented embedded PostgreSQL startup. No local aggregate pass is claimed. The first clean-worker CI run identified avatar fixture/configuration, flag defaults, feature-card ordering and OpenAPI route-coverage gaps; all four focused regressions pass after correction. Those regressions pass. The latest clean-worker CI run also found a Copilot source-hash declaration mismatch after the shared descriptor update; the corrected declaration passes all 28 integrity tests. The next review corrections pass 144 rendered UI tests, 21 OpenAPI tests and 82 Runner contract/integrity/history tests. The latest database fixtures were blocked at startup by host shared-memory exhaustion; clean-worker CI must execute their new regressions. Full current-head CI now passes: 54 successful checks and four intentional skips. The database regression uses the PostgreSQL driver result shape; file-policy, rejected-feedback and lock-order cases pass on clean workers. Current-head Greptile is 5/5 with no unresolved findings. - Rendered setup steps and outbound-only verification were walked through in Storybook. Simulation cannot enable Done without an authenticated-reply state. - Actual client OAuth, continued access, reconnect and one routine wake with an exact chat acknowledgement were observed separately. This candidate’s live authenticated challenge, assigned/proactive workflows, Linux managed artifact, restart and cleanup are not yet qualified. The provider stays disabled by default. - The separate Cloud ingress change forwards only exact task-bound upload/download paths and preserves host identity and header stripping. - Current Cloud policy reserves Test for pre-merge app/harness experiments; Staging accepts only master-derived revisions. Real-client qualification needs a supported, authorized deployment target and remains outstanding. ## Risks One append-only migration adds routine/dispatch state and extends shared bindings and file tickets. The unshipped migrations were regenerated after upstream consumed their original numbers. Rolling back code retains the durable rows; revoke grants and inspect unknown remote activity before resuming dispatch. Grok cannot confirm remote termination, stream its complete transcript or report spending. Plugins are account-wide and do not isolate credentials between Bots. Real-client managed acceptance remains required before enabling the experimental provider. ## Model Used OpenAI Codex based on GPT-6, with reasoning, code execution and browser/native UI tools. The exact deployment model ID and context-window size are not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9cc979116e |
fix(server): stop reporting queries that outlive the database shutdown (#16243)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server reports unexpected route errors to Sentry through the
shared error handler.
> - An orderly shutdown drains the HTTP listener, and then it ends the
database pool.
> - `server.close()` completes when the sockets close. A route handler
whose client disconnected, or whose socket the drain timeout closed, can
still run.
> - The next query of that handler fails with postgres.js
`CONNECTION_ENDED`, because this process ended the pool. The error
handler reports this as a 500 crash.
> - This pull request marks the pool as closing just before it ends. The
error handler then skips the error sinks for that one error code during
shutdown.
> - Public MCP calls run in their own internal Express app, so the flag
is also shared with that app.
> - The benefit is that a normal restart does not create false Sentry
crash reports. Real errors are still reported.
## Linked Issues or Issue Description
No public GitHub issue exists. This is the bug description.
**What happened?**
A staging server restarted. During the restart, `GET
/api/companies/:companyId/heartbeat-runs` failed in the per-row run
privacy check (`canActorReadHeartbeatRun`). The error was `Failed query:
select "id" from "issues" ...`. Its cause was `write CONNECTION_ENDED
<host>:5432`. postgres.js raises this code only for a query that starts
after the process called `sql.end()`. The error handler sent the error
to Sentry as a crash.
**Expected behavior**
A query that runs after the shutdown ends the pool is an expected part
of the shutdown. The server must not report it as a crash. If the
response is still open, the server answers `503`. The server must
continue to report all other errors.
**Steps to reproduce**
1. Start the server, and send a slow request (for example, a large
heartbeat run list).
2. Close the client connection, or let the 5 second drain timeout close
it, while the handler still runs.
3. Send SIGTERM. The shutdown drains HTTP and ends the pool.
4. The handler's next query fails with `CONNECTION_ENDED`. Before this
change, the error goes to Sentry.
## What Changed
- `server/src/database-retry.ts`: Add `isEndedDbPoolError`. It finds the
postgres.js `CONNECTION_ENDED` code anywhere in the error `cause` chain.
`isTransientDbConnectionError` now uses the same chain walk. Its
behavior does not change.
- `server/src/index.ts`: The orderly shutdown sets
`app.locals.paperclipDatabaseClosing = true` immediately before it ends
the database pools.
- `server/src/middleware/error-handler.ts`: When the flag is set and the
error is `CONNECTION_ENDED`, the handler answers `503 { error: "Server
is shutting down" }`, or it ends the response if the headers are already
sent. It does not call Sentry or crash telemetry. This follows the
existing client-abort (`ECONNRESET`) case.
- `server/src/services/public-mcp/capabilities.ts` and
`server/src/app.ts`: `createMcpApiDispatch` takes the main app's locals.
Each per-call dispatch app reads `paperclipDatabaseClosing` through a
live getter, so a public MCP call that outlives the pool is handled the
same way (Greptile review finding).
- `server/src/__tests__/error-handler.test.ts`: Add tests for the 503
path, the headers-sent path, `CONNECTION_ENDED` outside shutdown (still
reported), and `CONNECTION_CLOSED` during shutdown (still reported).
- `server/src/__tests__/public-mcp-dispatch-shutdown.test.ts`: Send an
ended-pool error through the real MCP dispatch, both during shutdown
(503, not reported) and outside shutdown (reported).
## Verification
- `cd server && vitest run
src/__tests__/public-mcp-dispatch-shutdown.test.ts
src/__tests__/error-handler.test.ts
src/__tests__/cloud-tenant-transient-db-retry.test.ts`: 25 tests pass.
- With only the error-handler change reverted, the two new error-handler
shutdown tests fail. With only the dispatch change reverted, the MCP
shutdown test fails. The negative-control tests pass in both cases.
- I could not run the full `pnpm -r typecheck`, `pnpm test:run`, and
`pnpm build` locally, or `public-mcp.test.ts`, which needs built
workspace packages. The sandbox has a 4 GB memory limit. CI runs the
full gates.
## Risks
- Low risk. The skip applies only to `CONNECTION_ENDED`, and only after
the shutdown starts to end the pools. postgres.js raises that code only
for a pool that this process ended. Network and database failures
(`CONNECTION_CLOSED`, `CONNECTION_DESTROYED`, and others) are still
reported.
- `CONNECTION_ENDED` outside shutdown (for example, a dedicated client
used after it ended) is still reported as a crash.
- The MCP dispatch app gets only the shutdown flag from the main app's
locals. It does not get any other locals.
- No schema, API contract, or UI change.
## Model Used
Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool
use (repository inspection, shell, test execution) and the Sentry and
GitHub connectors.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Devin FoleyandPaperclip
|
a0651c96f8 |
Add database-free process liveness probes (#16101)
Expose process liveness without database or credential queries while preserving hostname and runtime identity guards. Deep health and sleep/wake readiness checks retain their existing behavior. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f0e12c8519 |
Add managed Slack setup with an independent own-app path (#15679)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Slack connects a person to an agent through an installed app and a verified identity. > - Independent installations must remain usable without Paperclip Cloud or manager credentials. > - Approved Cloud deployments can use Slack manager grants to create and install each agent's app. > - Both paths should share credential storage, identity checks, recovery, and the existing runtime. > - This pull request adds gated managed setup and keeps independent app setup under Advanced. > - It also makes ongoing connection settings and personal test messages easier to use. ## Linked Issues or Issue Description **Subsystem affected** Slack setup, connection settings, tenant credential storage, and the Cloud connector handoff. **Problem or motivation** Approved deployments need an installation flow without pasted configuration tokens. Open-source users must retain their independent Slack app flow. A connection manager also needs a simple way to check a bot from Settings. **Proposed solution** Use the managed path when the instance is configured to support it. Choose agent and Connect are the two visible stops. Continue starts authorization or installation. Use your own app remains under Advanced before creation. Save personal manager grants by company, Paperclip user, Slack user, workspace, and manager app. Reuse only verified identities for test messages. Unknown identities must complete a private Slack connection challenge before linking and receiving the test DM. **Alternatives considered** A shared bot would change each agent's Slack identity. Requiring Slack CLI or a hosted service for self-hosted users would remove independent installation. **Roadmap alignment** This maintainer-requested change improves Connected Apps and experimental Agent Chat. It adds provisioning and management controls to the existing Slack runtime. **Additional context** The companion Cloud broker profile is a separate deployment prerequisite. Managed setup remains gated. Existing-app adoption is deferred. ## What Changed - Extract Slack registration, provider requests, completion, customer-owned setup, managed setup, and personal manager grants into focused services. Preserve the compatibility facade and independent removal cleanup. - Add company-scoped manager grants and a registration reference through an additive, idempotent migration after current master. Keep runtime credentials separate from app-registration secrets. - Add authenticated manager authorization, sealed handoff, serialized refresh and revocation, app creation, managed installation, and installer-bound OAuth fallback. - Preserve creation intent, block automatic uncertain recreation, save credentials before downstream checks, and validate app, workspace, bot, scopes, and installer identity. - Add the two-stop production wizard and ordered stories. Keep the own-app, manual, existing-app, and legacy-draft paths. - Simplify Settings. Open Slack configuration targets the saved app. The test-message action has its own row. Remove the old manual-avatar section, channel-message section, and broken permissions link. - Add paginated account discovery and a private connection challenge for unknown accounts. Expired invitations stop waiting and offer a new private invitation. Save send receipts before provider dispatch and preserve confirmed delivery when activity logging fails. - Reinstall the saved managed app on explicit reconnect, replace revoked credentials, and resume completion. Persist reconnect intent and duplicate-request receipts without creating another app. - Update provider browser tests, recovery tests, secret redaction, setup documentation, and the agent prompt. Preserve current master's durable verification scheduling. ## Verification - Full workspace typecheck and repository build passed on `56d389d47a`. The subsequent reconnect retry fix in `288364eada` also passed UI typecheck and UI build. - 114 focused Slack registration cases and 13 production managed-wizard cases passed, including fresh-token reconnect, retry after installation failure, and duplicate reconnect requests. - All nine affected Slack browser cases passed against the production UI and a disposable fixture server. They cover managed setup, automatic setup, own-app selection, refresh, recovery, mobile behavior, and connection settings. - Eight focused real-database test-message cases passed. They cover a private ownership challenge, saved grant reuse, account pagination, revoked and conflicting links, workspace mismatch, disabled DMs, revoked membership, audit failure, ambiguous dispatch, and secret canaries. - UI token gates and the production Storybook build passed. - Greptile reviewed exact head `288364eada5d9c114fad991ff634612639d06672` at 5/5 with no actionable findings or unresolved review threads. - The full repository test suite and current-head CI are running. Their final results will replace this line before merge. - Earlier live feature previews created two bots in the authorized Paperclip Slack workspace and reused one personal manager grant. Local managed Continue created and installed a bot without a configuration token or a separate Add to Slack action. Avatars, welcome DMs, signed callbacks, and direct replies were observed. A Settings test DM also arrived in the signed-in user's Slack account. - Earlier staging thread qualification required manual publication recovery for one bot. Current-head live qualification remains required before enabling the production default. Fixture results do not establish Slack approval behavior in other workspaces. ## Risks - Personal manager tokens can create apps. They stay in scoped vault entries. The manager client secret stays at the broker. Company, actor/session, origin, and app identity checks apply throughout setup. - The additive migration must also work on qualification instances that applied earlier branch migrations. Table, column, index, and constraint creation is idempotent. - Creation and message timeouts can leave uncertain outcomes. Saved request IDs prevent automatic duplicate dispatch. - Unknown Slack identities must receive and complete their private challenge while signed in. Account selection alone does not grant Paperclip permissions. Existing revoked or conflicting links are not overwritten. - A development-only manager transport uses an explicitly configured private credential file for local qualification. Production ignores this transport; no manager credentials are bundled with open source. - Managed rollout requires Slack approval, the companion broker deployment, configured secrets, and final live qualification. Removing a connection does not delete its Slack app. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, and browser tools. The session does not expose the exact serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
3ec7309d91 |
feat(voice): add experimental Speko task connection runtime (#15727)
First PR: runtime. UI: #15728. Qualification: #15729.
## Thinking Path
> - Paperclip manages agent work through tasks and governed connections.
> - Chat connections already bind external conversations to those tasks.
> - Speko handles voice and telephony, while the selected Paperclip
agent does the work.
> - A call needs durable session authority, ordered follow-ups, and safe
result delivery.
> - This PR adds an explicit voice transport to the existing connection
runtime.
> - The UI and qualification suites follow in linked PRs. Speko stays
experimental.
## Linked Issues or Issue Description
**Problem or motivation**
Phone and browser voice conversations cannot use the chat connection
task lifecycle. A separate voice runtime is needed to preserve task
authority, retries, and reply history.
**Proposed solution**
Add company-scoped voice sessions, signed callbacks, durable replies,
inbound intake, and an agent-facing outbound-call tool. Reuse existing
tasks, queues, secrets, and publications.
**Roadmap alignment**
This extends the experimental chat connector entry points under Agent
Chat and Connected Apps. The project owner requested this feature.
Refs #11418 for the original provider catalog proposal. This
implementation uses the newer chat connection infrastructure rather than
an MCP catalog-only integration. Refs #15721 for remote OpenCode
startup.
## What Changed
- Add additive voice schema, shared contracts, session APIs, and
provider callbacks.
- Bind browser, inbound, and outbound sessions to an endpoint, task, and
selected agent.
- Add low-trust guest intake in an explicitly selected sandbox and keep
ordinary approval rules.
- Submit task work through session-scoped tools and push approved
replies through Speko POST /v1/calls/{id}/messages.
- Distinguish provider acceptance from confirmed spoken playback.
Reconcile reports without downloading recordings.
- Add protocol, migration, persistence, provider, and
delivery-diagnostic tests.
## Verification
- Master
|
||
|
Nicky LeachandPaperclip
|
2dd9811e8b |
Own agent hiring and termination in a lifecycle module (#15631)
## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description **Problem or motivation** A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. **Proposed solution** Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. **Alternatives considered** A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. **Roadmap alignment** This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2b4a1d7073 |
feat(dot): add guided invitations and persistent agent connections (#15581)
Add guided external-agent invitations with live Dot connection checks and ongoing access until revoked. Keep expired credentials invalid and enforce permissions for private avatar uploads. Preserve Runner recovery and remote launch boundaries. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
428b617fc9 |
test(evals): qualify native question resume paths (#15616)
## Thinking Path > - Paperclip lets agents pause tasks for human answers and continue the work. > - Native questions can yield a run or pause inside the provider's running turn. > - Those paths have different run identities and different forms. > - The documentation-placement test requires a semantic three-turn journey. > - A valid provider question therefore needs separate behavioral qualification. > - This pull request adds an opt-in two-answer journey with exact source and resume checks. > - Qualification exposed a Vite startup crash in idle-handler traversal, so the branch also distinguishes Connect mount paths from Express Route objects. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E evaluation of native task questions and answer delivery. **Current behavior** The semantic documentation case rejects the provider adapter's optional Other field before answering. Its three-run expectation also cannot prove a provider question that resumes within the same run. **Proposed behavior** Keep that semantic test intact. Add a separate opt-in case that classifies each recorded question by authoritative identities. Accept the optional Other field only for a verified provider question. Require both user answers, the correct continuation, and one saved final document. Related: #15564 added strict lifecycle checks. #15522 introduced idle-request tracking; this PR corrects its Connect/Vite compatibility after reproducing the startup crash. #14591 changes ACP question drafts and provider support; this PR has no overlapping files with that change. ## What Changed - Add the explicit-only `question-resume` suite for native Codex and ACPX Claude. Reuse the existing neutral choice-then-text task request. - Bind each question to its company, task, agent and run. Require an applied semantic tool receipt or the exact provider request identity. - Require a provider answer to continue the same run. Require a semantic answer to start one new run with the matching interaction and source-run wake fields. - Preserve question forms and both real board answers through the final checkpoint. Reject missing inputs, duplicate answers, extra runs and substituted identities. - Keep the existing lifecycle, output, task ownership and budget checks. Limit the new case to one attempt and one to three runs. - Add mutation calibrations and document the separate qualification claims. Keep production guidance, scheduling, provider code and the historical semantic case unchanged. - Fix startup with Vite middleware: treat Connect string mount paths separately from Express Route objects when traversing idle-request handlers. A real-Vite regression verifies startup and retained async work through an idle hold. ## Verification - 81 focused continuation, question and calibration tests passed before review. After the reference-preservation correction, all 50 question/resume tests pass, including two new negative cases that failed against the previous grader. - Final eval support suite passes: 1,921 Vitest tests and 128 Node tests, with one intentional skip. - Eval and workspace typechecks pass. Workspace build passes. - Catalog discovery finds only the two declared opt-in cells. Existing continuation remains 23 cells; default paid scope is unchanged. - Original [campaign 37831726391](https://github.com/paperclipai/paperclip/actions/runs/37831726391), source `1ac4b2873466896ff2892f49be82904927b6c57d`, failed during server startup before any test or provider run. Its original `transient_infrastructure` result and `cleanup: not_started` are preserved. Local reproduction identifies the Connect string route traversal crash; the question grader was never reached. - Setup-corrected [campaign 37833872898](https://github.com/paperclipai/paperclip/actions/runs/37833872898) measures source `9781a772389d41421df55759004676e1b905e512` with trusted workflow `8e59efc50b162126f33816841e06509634eba65c`. The workflow bytes are unchanged from the first dispatch. Same single selected native Claude cell, one attempt, at most three runs, ten-minute cell deadline, and 1,000-cent company/agent hard stops. Original result **PASS**, 24/24 checks and cleanup pass. Exactly three successful native Claude runs: two applied Paperclip `request_human_input` questions and one finishing run. Both real board answers persist, exact interaction/source-run response wakes match, one revision-1 task document contains Afternoon and the supplied reference, and final task state is done with no active lock, retry, recovery, monitor, pending interaction or child task. No model retry. - Current review head `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e` differs from the measured source only in the question/resume grader and its tests (11 insertions, 3 deletions). The saved document must contain the complete submitted reference, including its prefix and punctuation. Separate provider-free replay of the retained successful campaign passes all 24 continuation/lifecycle checks under this stricter grader. The original result and its measured source remain unchanged; no new provider run was made. - Observed paths were **semantic → semantic**. The provider built-in question/optional Other path was not exercised by this new live attempt. Its form acceptance has retained-original calibration; same-run answer/resume has deterministic positive/negative calibration. This neutral-task pass alone does not qualify the provider path; the subsequent explicit bridge trial below is separate evidence. Neither trial regrades the old Claude failure or establishes a causal behavior/performance comparison. - Subsequent explicit provider-path [campaign 37841107684](https://github.com/paperclipai/paperclip/actions/runs/37841107684) measures the final PR source `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e`, using trusted workflow `2a5f65c9ae501b49b2a38ce7edd04209b806c9d0`. Exact cell: `continuation.runner-acpx-claude.local.provider-question-bridge`, Claude `claude-sonnet-5`, suite fingerprint `fc77203bf85fd99b06ebc53cd52982186c0ae0dc369fcc38a9ec3c737e9a31e5`. **Original PASS, 15/15 checks, cleanup passed, one actual provider run and zero eval rerolls.** The built-in question produced one choice plus its optional Other companion. The browser selected the requested reference, left Other blank, and submitted. Exact runtime request/response IDs and the selected option match the saved interaction; the same paused native run created one revision-1 document and finished Done with no pending interaction, lock, retry, recovery, monitor or child task. Independent evidence and runtime-receipt audits pass; marked initial/final screenshots were visually checked. [Original public report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37841107684-1/index.html). - This separate provider-directed trial qualifies one choice-answer bridge and traversal of an empty optional Other field. It does not establish natural tool selection, a typed Other answer, two consecutive built-in questions, free-text-only input, restart recovery or general reliability. No production, prompt or grader change was made for this dispatch. The old failure and the separate neutral semantic-path result retain their original grades. - Preserve within-run friction: one `write_task_document` input-schema rejection and two HTTP 409 “Document does not exist yet” responses preceded a successful `write_document` call. These are three unsuccessful tool calls inside the same run, not three extra provider runs; no flawless document-tool behavior is claimed. Billing reports token usage but remains unpriced/incomplete, so actual charges are unknown and local/hosted runtime is unmetered. Both company and agent hard stops were 1,000 cents, with one attempt, one selected cell, concurrency one and a ten-minute cell deadline. - Billing records all three model runs with token usage, but no priced dollar receipt (`unpriced`, `complete: false`). Actual charges are unknown; local/hosted runtime is unmetered. Numeric zero reported cost does not mean free. - Startup repair: real-Vite reproduction passes; 35 targeted idle-tracking tests pass, 34 database-dependent checks skip because embedded PostgreSQL is unavailable locally. Workspace typecheck and build pass again after the repair. - Full local repository database tests were not repeated because embedded PostgreSQL was unavailable in the preceding workspace verification. Full Linux CI now passes on the final review head. - Final-head [CI run 37837804284](https://github.com/paperclipai/paperclip/actions/runs/37837804284) passes. Complete current-head audit: 53 successful check-runs, two intentional Storybook skips, and separate Snyk success. The ready transition’s contributor and security checks also pass (security scan: no findings). [Fresh review](https://github.com/paperclipai/paperclip/pull/15616#issuecomment-6068092601) is 5/5 on `3c08cb7d26e7f9ac16469eeb07d1f31d828ec02e`; zero unresolved threads and no merge conflicts. ## Risks This is a new behavioral definition. The neutral two-question trial records semantic-tool use; the separate provider-directed trial records one built-in question. They do not establish consistent tool selection, documentation placement, default hiring, remote environments, crash recovery or general reliability. The separate explicit bridge trial covers one provider choice, an empty optional Other companion and same-run completion. Typed Other, two consecutive built-in questions and free-text-only provider input remain unqualified. The observed document-tool schema rejection and two 409 responses are retained as a separate follow-up, without attributing their cause. The historical case and its original grades remain intact. The reference question must still be text-only; the oracle does not turn an arbitrary extra question into an optional companion. Missing or unfamiliar identity evidence fails closed. The only production change distinguishes Connect string mount paths from Express route objects during idle-handler traversal. Unknown route objects still fail closed. No API, schema, migration or workflow change is included. ## Model Used OpenAI Codex, GPT-6-based, with tool use and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
3d0e74e743 |
Keep proven external connection failures out of Sentry (#15590)
Apply the reviewed change for Keep proven external connection failures out of Sentry. Validation: local typecheck/build and focused regression tests, passing exact-head CI, Greptile 5/5, and independent source review. The PR records full-suite evidence and any local environment limitations. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
3367b75ccc |
fix: fence accepted work and cleanup before idle sleep (#15522)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A host can stop an idle instance to reduce unused compute. > - Zero active runs do not prove that requests, saved work, or cleanup are complete. > - A client can disconnect while its request still writes, and failed cleanup can remain only in memory or on disk. > - This pull request holds new admission and checks accepted work, cleanup, and persisted work under an owned drain. > - The host gets an empty report only while those checks remain valid. ## Linked Issues or Issue Description Refs #13413. That companion PR uses the same owned-hold vocabulary for runtime services. This PR covers HTTP requests, scheduler work, accounting, cleanup, and the instance work inventory. It does not include the preview gateway changes. **What existing behavior does this improve?** The instance task-drain API reports process counters. It does not prove that a host can safely stop the instance for idle sleep. **Current behavior** A quiet run set can coexist with an unfinished request, cleanup after a completed run, future work, or a failed accounting write. **Proposed behavior** Provide a bounded owned idle hold. Block new ingress, track accepted handler promises, inspect durable and local work, and return `none` only when the same hold stays quiet through the checks. Keep normal deployment drains compatible. **Reason and benefit** Hosts can identify eligible idle instances without treating a disconnect or failed cleanup write as completed work. ## What Changed - Add `purpose: "idle"`, a bounded TTL, unique owners, and owner-checked release to task drain. Existing holds cannot be replaced by another API request. - Gate HTTP ingress before parsers, auth, webhooks, and MCP. Gate new WebSocket upgrades. Track async handlers in nested Express routers and error middleware until they settle, even after the response or client disconnect. - Count accepted live-event WebSocket authentication through settlement, even after disconnects. Count detached built-in agent, managed-home and runtime-service startup reconciliation after readiness. - Keep health and control mutations tracked. Count control-request authentication separately from the read-only report, including concurrent user/company/membership writes. - Pause new scheduler admissions during idle holds. Count work already in flight, including database backups, and reject scans whose work generation changes. Periodic backups block sleep without a host wake schedule. - Inspect accounting and orphan-cleanup spool directories without skipping temporary or malformed entries. Retain orphan tokens through queue splices, flush failures, and buffer overflow. Keep failed usage capture counted until its database failure fence is written. - Check persisted work across companies in a bounded read-only transaction. Include deferred agent-file cleanup and saved watchdogs whose watched issues are complete. Enabled plugins and unsupported retained work remain blockers. - Require the exact idle owner and completed startup tracking before returning an empty report. Keep reports free of tenant details. - Document the hosting protocol, retry behavior, conservative blockers, and the remaining external provider-stop race. ## Verification - Current head: `8d9a599b732c2047b8c671d4799065d7c90a3567`, rebased on master `941a3fa991aeb97eb1ac390c65b7973b5f6de1ad`. Both heartbeat helper extractions are preserved. GitHub confirms no merge conflicts. - Focused heartbeat renderer/run-log, drain, control-auth, admission, route and PostgreSQL inventory checks: 234 tests passed in 10 suites after the rebase. - The POST task-drain contract includes the expected `409` conflict response. The 84 OpenAPI and instance-settings route tests and server typecheck passed after that final documentation fix. - Final accepted-upgrade/startup regression run: 104 tests passed in five suites, including success and failure after readiness or disconnect. Final server typecheck also passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm install --frozen-lockfile` and the PR diff secret scan passed. No dependency or lockfile changes are added by this PR. - The prior verification covered HTTP disconnects and early responses, async error handlers, concurrent authentication writes, signed bootstrap, saved watchdogs, backup promises, orphan cleanup, spools and failed accounting fences. Those tests passed. - The last full local `pnpm test:run` stopped in the general-server phase with 16,658 passing tests and five skill/connector fixture failures caused by an ancestor workspace skill directory. That full local run preceded this rebase and has not been repeated for the import conflict. Full current-head CI passed: 53 successful checks and two conditional skips, with no failures. - All eight review findings are fixed and their threads resolved, including accepted upgrade authentication, detached startup writes and the POST conflict contract. Current-head Apex review is 5/5 with no new findings; all eight review threads remain resolved. - No live provider stop or production change was performed. ## Risks - The hosting controller must use the owned protocol and hold external admission through its final validation and provider stop. A legacy drain cannot authorize idle sleep. During an idle hold, new requests receive 503 with `Retry-After: 1`; the host must handle queueing or retry before enabling this path. - This is a single-process protocol. An unexpected restart after the last validation can race an external provider stop. The host must serialize deploy/wake/sleep operations and bind the validation to the instance it stops. Multiple replicas need shared fencing. - Some retained state conservatively prevents sleep, including every enabled plugin. This PR does not promise that every inactive instance becomes eligible. - The HTTP adapter uses Express 5 router layers. Real Express tests cover nested routes, errors and disconnects. New routes must register before tracking is installed. Detached work must have durable state or explicit work tracking. - Unrecoverable in-memory cleanup debt keeps the instance awake until reconciliation. This change does not make such debt survive an unplanned process crash. No schema migration or provider configuration change is included. ## Model Used OpenAI GPT-6 via Codex, with reasoning, repository inspection, tool use and code execution. The exact deployment variant/model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; the full local fixture limitation and clean full CI result are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d66acb7ac1 |
feat: automate Slack bot app setup and installation (#15413)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connectors give each agent a customer-owned bot and task-backed conversations. > - Manual Slack setup requires app creation and copying durable credentials. > - Operators need a shorter setup that an assisting agent can use safely. > - This pull request creates the app through Slack's Manifest API and installs it through OAuth. > - Durable registration state supports recovery without creating another app. > - A four-screen wizard, automatic avatar upload, and OAuth account linking reduce setup work. > - Connector settings and per-turn tool guidance support daily use after installation. ## Linked Issues or Issue Description **Subsystem affected** Native Slack bot setup, company secret storage, chat connector management, and agent tool guidance. **Problem or motivation** New Slack bots require manual app creation and copying a signing secret and bot token. Interrupted setup can create duplicate apps. The setup and management screens contain unnecessary controls. Agents also need guidance for native questions, files, thread replies, and governed Slack actions. **Proposed solution** Use a temporary app-configuration access token to create a customer-owned app. Save durable secrets in the vault. Bind OAuth to the initiating actor, company, endpoint, registration revision, scopes, and configured origins. Preserve manual and existing-app recovery. Link the installing user's account, send a welcome DM, and advance from saved server evidence. Keep request-URL recovery instructions available if automatic connection detection waits. **Alternatives considered** The Slack CLI adds installation requirements. Socket Mode changes transport. A shared Paperclip-owned app changes app ownership. These alternatives are outside this change. **Roadmap alignment** This extends existing chat connectors and secrets capabilities. Related public work: #14037 and #13954 cover Slack MCP prerequisites and user OAuth. No duplicate bot-registration PR was found. ## What Changed - Share one reviewed manifest builder between automatic registration and manual setup. - Add replay-safe migration 0318 and company-bound registration state with vault references and uncertain-creation recovery. - Add registration, installation, callback, and resume APIs with short-lived, single-use OAuth state. - Save installation credentials before downstream checks and preserve bot identity constraints. - Reduce automatic setup to four screens. Keep advanced app details, manual recovery, and existing-app setup. - Upload the agent avatar with the Paperclip dark background. Link the OAuth installer's account and send setup DMs. - Show agent and connector-owner avatars. Simplify settings, access, and conversation screens. - Discover joined Slack channels and enable them by default. Start a task from a bare mention and admit same-thread follow-ups. - Refresh Slack tool guidance each turn. Add native-form, file, approval, and delivery regressions plus manual model probe definitions and sanitized acceptance records. - Update deployment/database docs, OpenAPI, redaction, removal cleanup, production Storybook stories, and provider browser tests. - Merge current master and move the registration migration after its latest migration without rewriting published commits. The completed Slack success view intentionally has a single centered **Done** action and no **Save & exit**, as explicitly requested by the product owner. `DESIGN.md` records this exception; unfinished setup steps retain the aligned wizard footer. ## Verification - Passed after the master merge: repository typecheck, full build, Storybook build, design-token gates, module-boundary gates, and migration generation. - Passed: all 352 focused Slack deterministic tests and all 14 affected provider browser tests. Browser tests use controlled provider fixtures and a separate throwaway instance. - Passed on current head `c5d01e0e2`: the complete GitHub test matrix (general server, chat, all workspaces, serialized server, and Runner), all eight browser shards, typecheck/release registry, build, canary dry run, security checks, and policy gates. There are 52 passing checks and no pending or failing checks. - Greptile completed on the exact current head with 5/5 and no actionable findings or open review threads. - Local repair verification passed 93 focused tests, including same-app reinstall after revocation and rejection of consent started before revocation, the AgentMail browser journey, and repository typecheck. Local build and Storybook build also passed. The redundant local full-suite rerun was stopped after the complete current-head CI matrix passed. - Real Slack setup and agent replies were exercised in the authorized isolated test drive during the setup iteration. - The ten additional model probes were attempted with legacy `codex_local`, `gpt-5.6-sol`: five passed, two failed, and three were partly verified. Native runtime is not qualified. See `server/src/services/connectors/slack/evals/2026-10-08-acceptance.md` for evidence and limits. - Passing model probes cover native forms, downloaded file bytes, bare mentions with thread replies, explicit posts/reactions, and saved approval denial. - The controlled uncertain-write probe found wrong delivery-check IDs. The canvas fallback attempt used an invented tool name. Search pagination/native search, a private-source denied-tool receipt, and distinct board/webhook origins remain unqualified. Reviewer path: enable Chat connectors, start Slack chat setup, select an agent, enter an app-configuration access token, and approve Slack installation. Send a message to the bot and confirm that setup advances to success. Inspect settings and allowed channels. See `doc/connections/SLACK-AUTOMATIC-SETUP.md` for deployment and recovery. ## Risks - Slack app creation has no provider idempotency guarantee. A timeout after dispatch stays uncertain until the operator checks Slack. - OAuth needs a stable public HTTPS board origin. Webhook ingress may use a separate configured HTTPS origin. Workspace policy can delay installation. - Migration 0318 can replay safely on instances that applied the earlier development migration. - OAuth installation now links the installer to the initiating Paperclip user. Identity checks and company access rules still apply. - Joined channels now enable bot responses by default. Linked-user authorization and per-action approval rules still apply. - Model behavior has the documented delivery-check and canvas fallback failures. A passing CI run does not establish that every model probe passed. - Removing the connection does not delete the customer's Slack app. No new first-party telemetry is added. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser verification. The runtime does not expose a more specific authoring model ID or context-window size. The live bot probes used OpenAI `gpt-5.6-sol` through `codex_local` in legacy mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dd777f4b73 |
feat(dot): complete onboarding and expand governed Runner capabilities (#15414)
## Thinking Path
> - Paperclip manages AI agents, their work, and their permissions.
> - Paperclip Runner supplies the same admitted tool authority to each
provider.
> - The Dot provider in #15402 needs reliable onboarding and useful
agent capabilities.
> - An idle Dot could not start work, assign a human, read skills, or
produce a workspace artifact.
> - Pairing also relied on a second configuration save before ordinary
admission could work.
> - This pull request adds governed idle admission and shared Runner
tools, and completes pairing atomically.
> - Operators can test Dot with fresh data while keeping normal company,
approval, budget, and run ownership checks.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner, dedicated Dot MCP access, OAuth onboarding,
experimental settings, and empty worktree startup. This PR builds on
merged provider PR #15402. It reuses the merged MCP gateway from #14846
and assistant connection work from #14933 and #15380.
**Problem or motivation**
An idle Dot could see assignments but could not act on a conversation
request until someone created a task first. Its Runner catalog could not
assign tasks to humans or read pinned skills and workspace files.
First-time OAuth discovery and pairing also needed browser fixes, and a
completed pairing did not persist its binding reference on the agent.
**Proposed solution**
Keep Dot within the existing Runner. Admit a visible agent-authored
intake task for idle requests. Add people, human assignment, cross-task,
skill, and optional sandbox workspace tools through shared authority.
Relay assigned app calls through the configured MCP gateway. Add lease
renewal and follow-up references. Save pairing and its configuration
revision atomically. Show prerequisites and provide a complete copy
prompt.
**Roadmap alignment**
This extends the experimental provider in #15402. It uses the existing
governed gateway, task model, skills, artifact path, and native Runner.
It adds no separate execution subsystem.
## What Changed
- Add a standalone OpenAI Dot agent choice with an independent
experimental opt-in. It works with the general Runner option off.
Require Assistant connections (MCP), authenticated sign-in, and public
HTTPS for pairing.
- Use Dot’s own option for company package import. Allow an unpaired Dot
configuration to save after external billing acknowledgement; task
admission still requires pairing. Prepare the shared dev binary for
Dot-only opt-in.
- Label saved Dot agents as OpenAI Dot. Use prerequisite-check copy in
setup and runtime configuration.
- Route Dot creation directly to pairing after explicit external billing
acknowledgement. Hide local CLI, model, and harness setup for Dot.
Preserve the shared Runner implementation and canonical API type. Other
Runner providers still require their general opt-in.
- Add an empty worktree option with fresh signing keys and no production
data copy.
- Fix public-client OAuth negotiation, discovery compatibility, and
optional separate browser authorization origin.
- Add one-use pairing consent preview, clear copied setup instructions,
and atomic binding persistence and cleanup.
- Add idle request admission with stable request IDs and normal
scheduling, permissions, budgets, and task ownership.
- Add identity and people discovery, human task assignment and
reassignment, and authorized cross-task comments and documents.
- Read assigned skill files from pinned manifests. Relay assigned app
calls through the merged gateway without exposing credentials.
- Add an off-by-default workspace bridge. Constrain paths and writes.
Run commands in a deny-by-default OS sandbox with no network or injected
credentials. Reserve mutations before effects and never blindly repeat
uncertain work.
- Add rolling lease renewal, task pagination, bounded operation limits,
and deduplicated follow-up references without comment bodies in
webhooks.
- Add an off-by-default attachment reading setting. Restrict reads to
files on the current assigned task. Verify size and hash, cache bounded
verified copies per run, paginate text or binary bytes, and recheck live
authority before returning.
- Keep file grants operator-owned. Reject agent self-grants across
configuration routes. Preserve attachment consent in create/import
forms. Close generic API file bypasses while retaining current-run
response snapshots and permitted uploads.
- Keep provider limits explicit. Do not inherit a Dot binding,
attachment permission, or workspace permission when hiring another
agent.
- Include the required Markdown format in cross-task document writes and
validate the API title limit. Verify real creation and revision
persistence.
- Restrict command execution to Linux bubblewrap with descendant
containment. macOS retains workspace file tools and artifact publishing,
while refusing command calls. Explain the platform limit in setup.
- Exclude Paperclip instance state from workspace files, uploads,
artifact publication, and sandbox commands. Protect nested directories
and case variants. Fail closed when the directory protection scan
exceeds 4,096 directories.
- Add static UI compression for slow public tunnels. Document setup, the
complete tool inventory, and qualification limits.
## Verification
- Merge preparation on `00ca2c75b` integrates merged base #15402 and
master `fc6304dfe`. The ancestry commit preserves the reviewed follow-up
source tree. The subsequent security fix excludes instance state from
file tools, uploads, artifact publication, and sandbox commands. It
preserves private task authorization and task monitors. It regenerates
the combined tool catalog, seeded catalog digest, and protocol manifest.
Workspace typecheck, full build, and UI token gates pass. All sixteen
real Dot broker cases and 93 company import cases pass after the review
fixes and creator-attribution test correction. The exported
human-assignment catalog and Unicode page boundaries are also fixed. All
ten catalog tests and nine workspace/skill bridge tests pass. All 35
workspace bridge and authority tests passed after the instance-state
fix, including real macOS commands in that intermediate version. The
subsequent document and descendant-containment fixes pass 44 focused
tests across bridge, authority, and setup UI, with six Linux command
cases skipped on macOS. Real cross-task documents pass the route
validator and persist two revisions. macOS refuses command execution and
does not advertise the tool. Full workspace typecheck and build, changed
server/UI typechecks, and token gates pass again on the final commit.
Greptile rates final head
|
||
|
DottaandPaperclip
|
a9a20fb5c6 |
feat(security): add read-only customer-success inspection APIs (#15405)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents need a persistent identity and a verified active run for governed access. > - Customer-success inspection needs broad reads without tenant writes or secret access. > - Ordinary board login and database credentials give more authority than this task needs. > - This pull request adds a dedicated inspection API and strict managed-run authority. > - Cloud owns short grants, human approval, replay protection, and audit records. > - The benefit is inspectable access that an operator can disable immediately. ## Linked Issues or Issue Description Refs: #15352. This change reuses the persistent Ed25519 identity from that PR. **Subsystem affected** Server authentication, pure resource readers, and the shared wire contract. **Problem or motivation** One internal Paperclip agent must inspect customer onboarding work. It must not receive owner login or database credentials. Reads must not create customer sessions, memberships, activity, or read receipts. **Proposed solution** Add disabled-by-default run authority and versioned tenant inspection endpoints. Require strict instance-bound managed-run JWTs on the home instance. Require exact-operation, single-use Cloud permits on tenants. Execute a reviewed company-scoped catalog in read-only transactions. Cloud applies seven-day stack-age eligibility and human exceptions. **Roadmap alignment** This is access support for Cloud deployments and governed agent identities. Bot creation, scheduling, scoring, and reports are separate work. The maintainer requested this implementation. ## What Changed - Reuse existing public identity reads and managed private-key injection. Reject unprovisioned keys, paused agents, ended runs, legacy signatures, and wrong instances. - Mount `/api/customer-success/v1` before actor/session synchronization. Verify Cloud permits and consume them centrally before reading. - Add explicit company-scoped database readers and bounded instruction, skill snapshot, run log, workspace, and asset reads. Preserve existing redactions and file protections. - Add protocol, security, database immutability, and managed-agent qualification tests. Add deployment and rollback documentation. ## Verification - Full `pnpm -r typecheck` and `pnpm build` passed. Server typecheck passed after review fixes. - The broad local `pnpm test:run` recorded 14,277 passes and four failures in unchanged suites: two timeouts and two PR-metadata mock assertions. All three affected suites passed on isolated reruns (36 tests). The complete CI matrix passes at the final head, including every test lane, typecheck, build, runner checks, canary dry run, and the security scan. - Focused inspection, JWT, and existing identity tests pass. The catalog test compares every public database table before and after reads. - Inspection and route-contract tests: 22 passed. The coordinated test runs a real managed process agent against separate home/customer PostgreSQL databases and a PostgreSQL broker over HTTP. It proves wake through the existing controller, bounded binary file reads, single challenge consumption across replicas, concurrent grants with a two-connection pool, scoped SQL audits, append-only runtime auditing, one-year retention, and unchanged tenant data/files. - Run the coordinated test with `PAPERCLIP_INSPECTION_CLOUD_DIST` pointing at the sibling Cloud build. Normal unit runs skip that optional private integration. - Final-head Greptile is 5/5 with no unresolved findings. - No production deployment or customer inspection occurred. ## Risks - This adds an authentication boundary. Keep both feature flags disabled until coordinated staging and canary qualification. - Cloud support must deploy after this API. Unsupported tenants fail closed. There is no owner-login or database fallback. - Existing redactions remain the content boundary. Arbitrary pasted secrets in readable prose or files may remain. - Remote files and suppressed provider traces remain unavailable. Wake can cause normal startup/background writes; test those separately. - Disable Cloud policy first during rollback. Preserve existing identity material and Cloud audit history. ## Model Used OpenAI GPT-6 (Codex), with reasoning, code execution, and browser testing. The session does not expose a more specific deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks and isolated reruns; broad-run flakes are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |