From 12f70d0d4cbfe7cac4d1830a03669bf9a6658cda Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 10 Nov 2026 12:17:50 -0600 Subject: [PATCH] Keep Slack form custody and receipt transfer durable (#19825) ## Thinking Path > - Paperclip manages work for AI-agent companies. > - Chat connections deliver provider messages and user answers. > - Slack requires a response within its original three-second deadline. > - Forms and their delivery receipts must survive retries and tenant restarts. > - This change uses the existing connector queue and durable form owners. > - Current authorization, exact receipt checks and normal ACK rules stay in force. > - The tenant can settle useful work while keeping the connection enabled. ## Linked Issues or Issue Description Refs #19019 and #19348. Companion Cloud PR: paperclipai/paperclip-cloud#1250. ## What Changed - Preserve form grants, sessions, encrypted evidence, terminal receipts and company-authorized Retry. - Check initial form enrollment before the provider send. Keep a typed preparation wait only while an exact current enrollment obligation exists. - Refund the attempt only when the claimed attempt and credential lease still match and provider invocation has not started. Possible or accepted sends retain uncertain-delivery custody. An ordinary error with the same text still uses the normal five-attempt limit. - Notify the existing publication worker when normal enrollment commits. Reconstruct pending work from durable state after a lost reply or owner restart. - Page settled history before proving idle safety. Keep unfinished, malformed and foreign receipts visible beyond the first 1,000 rows. - Keep Slack callback, answer, receipt and ACK rules in their provider owners. Support body-bound Cloud idle capabilities when a complete catalog exceeds the header budget. - Add focused regression coverage, an enabled quiet/reactivation recipe and explicit fixture port checks. - Keep SDK evidence tests tied to a normally built reviewed reference, with a separate strict refusal for actual unreviewed CI outputs. ## Verification App head: `6284d6881b5dc38a9ee149bcbde35e69a67dd8ca`, including actual master `dff8bed007e01c32f3826144b893ec80603d1335`. Companion Cloud head: `f83b38187dc7e6b2e219fe865dc16d268cc66fdb`, including actual Cloud master `b1dcc3421a91efffee631a61536cf48f357c22fc`. Cloud #1250 is merged at `1d07d81f23d4627ab7b6dcf49176b4128a2f8f9b` after exact-head passing checks and complete Greptile 5/5. Current exact-head CI typecheck, build, tests and aggregate verification are green; the complete edited Greptile review is 5/5 and all 13 review threads and nested comment pages are resolved. - Both latest enrollment findings were reproduced against the preceding product. The pure readiness control returned no wait for the first enrollment. The real-PostgreSQL control observed a provider call while that enrollment was still staged. - The repaired source passes three whole files: 21 readiness controls, 80 real-PostgreSQL milestone controls and 37 tenant-actor middleware controls. All 138 pass with zero excluded or TODO cases. The SQL case reconstructs the worker through seven waits, preserves the attempt budget, recovers a controlled lost enrollment reply, and commits one provider-port call and one exact grant. Replaying the selected attempt does not duplicate them. Authorization, SDK and enrollment-response ports are controlled in that SQL case. - The same files cover ordinary five-attempt exhaustion, accepted and possible sends, stale ownership, malformed bindings, attention states and current actor restrictions. One synthetic mismatched-binding parameter uses generation 2 against current generation 1; it is not evidence for an older-generation control. - The normal server-package typecheck passes at the measured source. It runs the normal Runner TypeScript, protocol, SDK and declaration prerequisites and all seven main App programs. The separate declaration configuration is also recorded. A worker union-narrowing error was found, fixed with an explicit callable/null branch, and verified by that normal compiler. No cast or test relaxation was used. - The normal merge of current master preserves all 78 owned paths and 15,783 unrelated master entries and modes. The three measured test files and their changed feature paths are unchanged. New upstream DB code adds only the normal nonunique concurrent native-session index and its migration classification. Shared and adapter-utils changes alter unrelated prompt or denial wording. The 138-case proof carries for those unchanged behaviors; it is not reported as a rerun after the merge. The normal Shared/SDK ensure, DB build, adapter-utils build and complete server-package check also pass at the current merged head. Their fresh inventory contains 8,143 outputs; canonical SQL and journal copies match. This build did not apply the new index migration. - Earlier real-PostgreSQL retained-history coverage reproduced the missing proof with 1,005 settled rows. Its repair retains the existing two-second/64-query inspection budget, current endpoint bound and exact owner snapshot. The 46 source-bound controls cover unfinished and invalid tails, exact cross-page action references, Retry, rollback and lost receipt responses. - Earlier enabled quiet/reactivation coverage produced ordinary useful work, kept configuration enabled, accepted fresh signed work plus a duplicate, and produced exactly one second useful reply. Its two App-worker windows lasted at least 30 seconds with zero measured PG writes, connection or outbound activity after settlement. That evidence stays tied to its measured revision. A missing worker EXIT marker limits that earlier proof; parent evidence does not replace it. - Earlier three-second callback, terminal ACK, current-question race and complete body-size controls remain source-bound. The 500-owner/1,000-binding body measured 313,875 bytes with a 618-byte JWT; signature tampering failed closed. - Final managed-provider acceptance now passes all five fresh Node boots: A/A, A/AB, B/A rejection, B/AB recovery, B/B final. Two independently configured, challenged, verified, linked and enrolled Slack identities exercise ACK and retirement debt separately. Normal provider factory/materialization delivers the configuration through a controlled external stack-provider boundary. The registered opening returns HTTP 200 in 21 ms within the original 3-second deadline. Two answers persist, normal ACK is accepted, exact retirement and lost acceptance/GC replies recover, and the original canonical snapshot remains unchanged without duplicate provider sends or submissions. - The separate complete committed-preflight case passes. A committed `update_attempted` response is lost before SDK entry; normal service-owner reconstruction after the original lease/deadline recovers the same useful update and one answer without a second trigger. This reconstructs service owners within one App process; the separate five fresh Node boots cover process restart and managed signing-key rotation. - Assertion-time PG socket reports balance 15 opens/15 closes across the five boots and 5/5 in preflight, peak 2 and zero live sockets or unexpected attempts. Worker terminal EXIT markers were missing and are not replaced by parent EXIT markers. These are domain/recovery and assertion-time connection proofs, not a new whole-worker terminal quiet measurement. All known owners/groups closed; independent SQL audits were empty; only successful fixture databases were removed. Earlier failed fixture databases/logs remain retained. No product guard, deadline or assertion was weakened. Exact runtime source and normal outputs are unchanged by the three test files in the current follow-up, so these source-bound proofs carry without repeating those journeys. - The preceding App CI ran 11,312 support cases: 11,289 passed, three SDK-evidence cases failed, and 20 external cases skipped. Normal CI dependency setup resolved six stale Slack patch-hash references and produced an unreviewed lock. The public SDK proof correctly refused that lock; the tests incorrectly assumed the CI output was a reviewed reference. The three-file test repair keeps the protected lock policy, source/compiler/parser/Zod pins, scanner and redaction rules unchanged. It builds an owned reviewed reference using the existing normal Shared/SDK producers for the positive seal test, while a separate negative requires actual unreviewed-build privacy refusal and no retained SDK bytes. It does not relabel the CI build as certified or admit a new lock hash. - All four whole SDK support files pass locally: public type 30, evidence 16, public-input synchronizer 8 and lock preparation 10, totaling 64 with zero excluded/TODO cases in 28.73 seconds. The genuine fresh reference build/positive seal and separate invalidated-reference privacy refusal passed within their original 120-second bounds. Original installed dependencies, SDK outputs/cache and runtime source stayed unchanged; normal global setup produced only its own Runner JavaScript outputs. PostgreSQL was denied throughout, all four worker markers were observed, and the owned process group/private test root closed. The local canonical-lock negative mutates only its own reference; fresh remote CI also passes the actual resolved-CI-lock branch: all 16 evidence and 30 public-type tests pass, including the genuine reviewed-reference positive and the strict actual-unreviewed-build refusal. The upstream synchronizer file runs its current 10 cases in CI. - The first complete CI attempt on this head also exposed two separate 30-second timeouts: one real reporter-artifact test and the `agent-test-environment` module-loading hook. Full original logs remain preserved. Their exact test, sanitizer and evidence source blobs are unchanged from the preceding head; inspection found no deterministic setup cause. After the original run finished, one retry of only the failed jobs passed on the same synthetic merge `168851e210671ebe97e7a778e8cf891807e80502`: Runner support reports 11,304 passes and 20 external skips; serialized shard 6 reports all 395 passes, including all 67 route bodies. No source, timeout or assertion changed. This proves recovery; the original causes remain unexplained. - The final pre-merge tree audit uses actual master `17136a6c25ef6cb7c816a29140a4c48c2c75fb31` (refreshed after the earlier `85dec046` audit). The expected normal squash preserves all 81 owned paths and 15,816 unrelated master entries and modes. Only OpenAPI has edits from both sides, and its complete content composition preserves the new project route and token help alongside this PR's API entries. The shared inbound selection changes preserve the same candidate set; new concurrent indexes, Native/workspace behavior and shutdown changes remain upstream behavior. The scoped form proof uses process agents and does not qualify the new Native/sandbox branches. The latest question-schema change still accepts the explicit version-1 payload exercised here. The last master delta also changes managed AI/Native credential refresh and writeback; those branches are outside the measured process-agent form paths. Existing 8,143 local outputs remain tied to their measured revision; no new whole-tree output equality or deployed suspension is claimed. - Reproduce the public path with `doc/testing/slack-cloud-qualification.md`, the companion Cloud checkout, Node 24/22 and fresh owned fixture databases. Aggregate local `pnpm -r typecheck`, `pnpm test:run` and `pnpm build` were not rerun as one command set. Current exact-head required CI and the complete Greptile 5/5 review now pass; the full edited summaries and all paginated threads were read. Local worker quiet does not prove deployed host or Neon suspension. Photon and Discord remain separate work; this PR completes the described Slack slice. ## Risks Use compatible App and Cloud readers before transferring custody. Signatures prove authenticity; current authorization and exact ownership still control execution. Preserve old targets, secrets and terminal facts until their normal owners release them. Keep the managed seal identity stable while custody remains. The required Slack adapter patch remains pinned. Upstream's concurrent index uses the normal top-level migration path. ## Model Used OpenAI `gpt-6.1-sol` implemented and tested the changes. A GPT-6-based Codex managing agent reviewed and integrated them. The host does not expose the managing agent's exact model ID or context-window sizes. The agents used code execution and test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, its version and tool capabilities - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public related issues or PRs - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run the local tests listed above and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green for the current head - [x] Greptile is 5/5 with no open findings for the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- doc/idle-sleep-safety.md | 12 +- doc/testing/slack-cloud-qualification.md | 66 +++ patches/@chat-adapter__slack@4.39.0.patch | 100 +++- .../__tests__/chat-run-milestone-work.test.ts | 157 +++++- .../cloud-control-task-drain.test.ts | 64 ++- .../slack-cloud/controlled-codex-cli.cjs | 19 + .../slack-cloud/enabled-quiet-host.mjs | 400 ++++++++++++++ .../normal-scoped-useful-final.mjs | 24 + .../helpers/slack-cloud/quiet-prewire.cjs | 153 ++++++ .../helpers/slack-cloud/read-owned-scopes.mjs | 8 + .../slack-cloud/run-enabled-quiet-app.mjs | 44 ++ .../helpers/slack-cloud/run-enabled-quiet.py | 216 ++++++++ .../slack-cloud/scoped-cli-prewire.cjs | 17 + .../slack-cloud/transport-observer.cjs | 27 + server/src/__tests__/idle-admission.test.ts | 13 + .../instance-settings-routes.test.ts | 45 +- .../src/__tests__/slack-cloud-forms.test.ts | 330 ++++++++++++ .../__tests__/slack-cloud-owner-work.test.ts | 13 +- .../slack-enabled-quiet.postgres.test.ts | 361 +++++++++++++ .../slack-form-publication-readiness.test.ts | 116 ++++ .../slack-form-retirement-work.test.ts | 510 ++++++++++++++++++ server/src/http/body-limits.ts | 1 + server/src/middleware/cloud-control.ts | 5 +- .../src/middleware/cloud-tenant-actor.test.ts | 5 +- server/src/middleware/idle-admission.ts | 4 +- server/src/routes/chat-channels.ts | 5 + server/src/routes/instance-settings.ts | 13 +- server/src/routes/openapi.ts | 24 + server/src/services/chat-channels.ts | 59 +- .../chat-channels/idle-history-proof.ts | 28 +- .../chat-channels/interactions/callbacks.ts | 40 +- .../chat-channels/publications/failures.ts | 12 +- .../chat-channels/publications/preparation.ts | 7 + .../chat-channels/publications/worker.ts | 20 + .../services/chat-channels/runtime-types.ts | 1 + .../chat-channels/slack/ack-receipt.ts | 67 +++ .../slack/ack-retirement-receipt.ts | 60 +++ .../services/chat-channels/slack/adapter.ts | 21 + .../slack/cloud-callback.test.ts | 105 ++++ .../chat-channels/slack/cloud-callback.ts | 56 ++ .../chat-channels/slack/cloud-consumer.ts | 58 ++ .../chat-channels/slack/cloud-enrollment.ts | 2 + .../chat-channels/slack/cloud-event.ts | 8 + .../chat-channels/slack/cloud-forms.ts | 268 +++++++++ .../chat-channels/slack/cloud-ingress.ts | 21 +- .../slack/configured-coverage.test.ts | 23 + .../slack/configured-coverage.ts | 66 +++ .../chat-channels/slack/form-ack-work.test.ts | 30 ++ .../chat-channels/slack/form-ack-work.ts | 201 +++++++ .../slack/form-canonical-state.test.ts | 26 + .../chat-channels/slack/form-delivery.ts | 85 +++ .../slack/form-enrollment-identity.test.ts | 39 ++ .../slack/form-enrollment-identity.ts | 27 + .../slack/form-idle-history-sql.ts | 47 ++ .../chat-channels/slack/form-idle-history.ts | 85 +++ .../chat-channels/slack/form-normal-ack.ts | 37 ++ .../chat-channels/slack/form-open-callback.ts | 43 ++ .../chat-channels/slack/form-opening-proof.ts | 63 +++ .../slack/form-owner-rearm.test.ts | 21 + .../chat-channels/slack/form-owner.ts | 320 +++++++++++ .../chat-channels/slack/form-publication.ts | 82 +++ .../slack/form-retirement-work.ts | 120 +++++ .../slack/form-retirement.test.ts | 88 +++ .../chat-channels/slack/form-retry.ts | 87 +++ .../chat-channels/slack/form-session.ts | 448 +++++++++++++++ .../slack/form-signature-refresh.ts | 29 + .../slack/idle-history-owner-proof.test.ts | 30 ++ .../chat-channels/slack/idle-incoming.test.ts | 64 +++ .../chat-channels/slack/idle-incoming.ts | 175 ++++++ .../chat-channels/slack/receipt-authority.ts | 11 + .../services/chat-publication-errors.test.ts | 13 + .../src/services/chat-publication-errors.ts | 4 + server/src/services/cloud-runtime-identity.ts | 51 +- .../src/services/github-connection-events.ts | 48 +- .../services/idle-incoming-wake-capability.ts | 17 +- server/src/services/idle-incoming-wake.ts | 20 + .../paperclip-cloud-connector.test.ts | 57 ++ .../src/services/paperclip-cloud-connector.ts | 94 +++- tests/runner-e2e/source-sdk-evidence.test.ts | 53 +- .../runner-e2e/source-sdk-public-type.test.ts | 25 +- .../runner-e2e/source-sdk-reviewed-fixture.ts | 72 +++ 81 files changed, 6189 insertions(+), 97 deletions(-) create mode 100644 doc/testing/slack-cloud-qualification.md create mode 100755 server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs create mode 100644 server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs create mode 100644 server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs create mode 100644 server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs create mode 100644 server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs create mode 100644 server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs create mode 100644 server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py create mode 100644 server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs create mode 100644 server/src/__tests__/helpers/slack-cloud/transport-observer.cjs create mode 100644 server/src/__tests__/slack-cloud-forms.test.ts create mode 100644 server/src/__tests__/slack-enabled-quiet.postgres.test.ts create mode 100644 server/src/__tests__/slack-form-publication-readiness.test.ts create mode 100644 server/src/__tests__/slack-form-retirement-work.test.ts create mode 100644 server/src/services/chat-channels/slack/ack-receipt.ts create mode 100644 server/src/services/chat-channels/slack/ack-retirement-receipt.ts create mode 100644 server/src/services/chat-channels/slack/cloud-callback.test.ts create mode 100644 server/src/services/chat-channels/slack/cloud-callback.ts create mode 100644 server/src/services/chat-channels/slack/cloud-consumer.ts create mode 100644 server/src/services/chat-channels/slack/cloud-forms.ts create mode 100644 server/src/services/chat-channels/slack/configured-coverage.test.ts create mode 100644 server/src/services/chat-channels/slack/configured-coverage.ts create mode 100644 server/src/services/chat-channels/slack/form-ack-work.test.ts create mode 100644 server/src/services/chat-channels/slack/form-ack-work.ts create mode 100644 server/src/services/chat-channels/slack/form-canonical-state.test.ts create mode 100644 server/src/services/chat-channels/slack/form-delivery.ts create mode 100644 server/src/services/chat-channels/slack/form-enrollment-identity.test.ts create mode 100644 server/src/services/chat-channels/slack/form-enrollment-identity.ts create mode 100644 server/src/services/chat-channels/slack/form-idle-history-sql.ts create mode 100644 server/src/services/chat-channels/slack/form-idle-history.ts create mode 100644 server/src/services/chat-channels/slack/form-normal-ack.ts create mode 100644 server/src/services/chat-channels/slack/form-open-callback.ts create mode 100644 server/src/services/chat-channels/slack/form-opening-proof.ts create mode 100644 server/src/services/chat-channels/slack/form-owner-rearm.test.ts create mode 100644 server/src/services/chat-channels/slack/form-owner.ts create mode 100644 server/src/services/chat-channels/slack/form-publication.ts create mode 100644 server/src/services/chat-channels/slack/form-retirement-work.ts create mode 100644 server/src/services/chat-channels/slack/form-retirement.test.ts create mode 100644 server/src/services/chat-channels/slack/form-retry.ts create mode 100644 server/src/services/chat-channels/slack/form-session.ts create mode 100644 server/src/services/chat-channels/slack/form-signature-refresh.ts create mode 100644 server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts create mode 100644 server/src/services/chat-channels/slack/idle-incoming.test.ts create mode 100644 server/src/services/chat-channels/slack/idle-incoming.ts create mode 100644 server/src/services/chat-channels/slack/receipt-authority.ts create mode 100644 tests/runner-e2e/source-sdk-reviewed-fixture.ts diff --git a/doc/idle-sleep-safety.md b/doc/idle-sleep-safety.md index 8cf330272b2..84a53f96f46 100644 --- a/doc/idle-sleep-safety.md +++ b/doc/idle-sleep-safety.md @@ -5,6 +5,16 @@ instance-wide `idleSleepSafety` report for instance administrators and signed Cloud control callers. Ordinary task drains keep their existing behavior. Company members and agents cannot read this report. +Cloud callers carrying full idle catalogs use `POST /api/instance/task-drain/status` +with the same read query and `{ "idleCapabilities": ... }` JSON body. A short +`task-drain:read-body` control JWT binds the exact raw-body SHA256. The existing +10 MiB JSON limit, fatal UTF8 decoding, strict catalog parsing, stack/action/expiry +checks and exact byte digest all apply before consuming the single-use request ID +or installing any capability. Embedded catalogs and body digests cannot be mixed. +Unsigned bodies and browser authority alone cannot authorize this Cloud-only read. +GET remains compatible; POST drain start and DELETE stop are unchanged. + + Instance administrators and signed Cloud control callers may add `idleSleepDiagnostics=1` to an owned safety request for fixed-key blocker diagnostics. This explicit one-shot read evaluates all current predicates in @@ -48,7 +58,7 @@ and `Retry-After: 1`. Health checks and authenticated task-drain control remain available. Ordinary deploy drains do not enable this HTTP gate. The host must handle retry/wake at its ingress before enabling automatic sleep. Health and control mutations remain tracked through completion, including -signed bootstrap writes. Only the task-drain read skips request tracking. +signed bootstrap writes. Only the exact GET task-drain read and POST task-drain/status body read skip request tracking. Its authentication finishes under a separate work token before entering the report, so it counts concurrent writes without blocking on its own scan. diff --git a/doc/testing/slack-cloud-qualification.md b/doc/testing/slack-cloud-qualification.md new file mode 100644 index 00000000000..46198df67f9 --- /dev/null +++ b/doc/testing/slack-cloud-qualification.md @@ -0,0 +1,66 @@ +# Local Slack App/Cloud qualification + +This opt-in qualification uses the normal App owners, a companion Cloud checkout, +real PostgreSQL, and controlled Slack/Codex protocol boundaries. It does not call a +model or Slack, deploy a tenant, or prove Neon suspension. Ordinary tests skip this +one case unless the explicitly named fixture URL and owned state file are present. +The runner supplies the state file; do not construct a receipt or a trusted key set. + +Use Node 24 for App, Node 22 for Cloud, and Python 3.9 or newer. Install and build +both checkouts through their normal repository commands first. In the companion +Cloud checkout, `npm run build:test` must emit its own current source. App needs its +normal workspace prerequisites and server typecheck (`pnpm --filter +@paperclipai/server typecheck`). The runtime helper does not install dependencies, +compile a substitute program, copy another checkout's output, or start PostgreSQL. + +Create one fresh, caller-owned App/Cloud test database pair on the same loopback +PostgreSQL cluster. The names must be `sleep2_slack_form__app_test` and +`sleep2_slack_form__cloud_test`. Provide the matching `/postgres` admin URL. +The URLs must use `postgres://`, an explicit port, and test-only credentials. Never +point the helper at a production database. Normal migration owners apply the full +retained executable migration history; this helper neither substitutes a schema +nor drops databases. + +Run from the App checkout with these environment variables set: + +```sh +export PAPERCLIP_CLOUD_REPO_ROOT=/path/to/owned/cloud-checkout +export PAPERCLIP_SLACK_NODE24=/path/to/node24/bin/node +export PAPERCLIP_SLACK_NODE22=/path/to/node22/bin/node +export PAPERCLIP_SLACK_TEST_DATABASE_URL=postgres://paperclip@127.0.0.1:55448/sleep2_slack_form_local_app_test +export SLACK_EVENT_TEST_DATABASE_URL=postgres://paperclip@127.0.0.1:55448/sleep2_slack_form_local_cloud_test +export PAPERCLIP_TEST_POSTGRES_ADMIN_URL=postgres://paperclip@127.0.0.1:55448/postgres +python3 server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py /path/to/fresh-output-directory +``` + +The runner starts the actual Cloud owner host and exactly one whole normal Vitest +case. The controlled CLI uses only the minted current-run token to check out its +admitted task, post its attributed reply, and complete the task through ordinary +APIs. Its canonical CLI final message lets normal presentation reuse that comment. +The controlled Slack endpoint must receive the exact useful text and provider ID; +a database publication or synthetic receipt cannot replace that observation. + +Each phase keeps the endpoint and connection enabled. After normal work settles, +the case waits for actual used App-worker PostgreSQL sockets to close with the +normal 60-second idle default, inside the existing 95-second settlement budget. +An already-required useful/query call proves the PG write counter is live. It +samples scalar counters immediately before and after the original 30-second window, +with the end sample taken before later inspection queries. The window requires +zero PG writes, connect attempts/connections/closes, live/peak PG sockets, outbound +requests and non-PG connection attempts. It then admits fresh signed work plus its +duplicate, requires a second distinct useful result, and repeats the same window. +No query text, wire payload, token, or private key is recorded by the counters. + +The plaintext same-worker hooks do not establish a cross-process or OS-wide socket +census. Missing Vitest-worker EXIT evidence is reported explicitly, even when its +final PG counters and sampled process-group absence are known. Real detached useful +helpers and version probes require their own entry and EXIT 0 evidence. The runner +joins owned groups and closes the normal Cloud/App owners before returning. It +retains failed evidence and does not mark a failed attempt green after a retry. + +Keep the output directory private: it contains diagnostic logs and source/owner +identity receipts. State containing synthetic fixture credentials stays in a +mode-0700 temporary root and is not copied into the report. A caller may inspect +and remove successful fixture databases only after all owners close. This test +does not qualify managed JWKS distribution/rotation, Photon, Discord, Cloud's own +PostgreSQL quiet, deployed sleep, or physical suspension. diff --git a/patches/@chat-adapter__slack@4.39.0.patch b/patches/@chat-adapter__slack@4.39.0.patch index 55a55b02f10..e44eaeb7ba1 100644 --- a/patches/@chat-adapter__slack@4.39.0.patch +++ b/patches/@chat-adapter__slack@4.39.0.patch @@ -1,5 +1,4 @@ diff --git a/dist/index.js b/dist/index.js -index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909a36c143d 100644 --- a/dist/index.js +++ b/dist/index.js @@ -2154,6 +2154,7 @@ @@ -117,7 +116,37 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 }; } catch (error) { this.handleSlackError(error); -@@ -4183,7 +4203,7 @@ +@@ -4157,8 +4177,10 @@ + this.handleSlackError(error); + } + } +- async updateModal(viewId, modal) { +- const view = modalToSlackView(modal); ++ async updateModal(viewId, modal, options) { ++ const metadata = options?.contextId ? encodeModalMetadata({ contextId: options.contextId, privateMetadata: modal.privateMetadata }) : void 0; ++ const view = modalToSlackView(modal, metadata); ++ if (options) view.external_id = options.externalId; + this.logger.debug("Slack API: views.update", { + viewId, + callbackId: modal.callbackId +@@ -4167,6 +4189,7 @@ + const result = await this._client.views.update( + await this.withToken({ + view_id: viewId, ++ ...(options ? { hash: options.hash } : {}), + view + }) + ); +@@ -4174,7 +4197,7 @@ + viewId: result.view?.id, + ok: result.ok + }); +- return { viewId: result.view?.id }; ++ return options ? { viewId: result.view?.id, viewHash: result.view?.hash } : { viewId: result.view?.id }; + } catch (error) { + this.handleSlackError(error); + } +@@ -4183,7 +4206,7 @@ * Upload files to Slack and share them to a channel. * Returns the file IDs of uploaded files. */ @@ -126,7 +155,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 const bufferResults = await Promise.all( files.map(async (file) => { try { -@@ -4205,7 +4225,9 @@ +@@ -4205,7 +4228,9 @@ (result2) => result2 !== null ); if (fileUploads.length === 0) { @@ -137,7 +166,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 } this.logger.debug("Slack API: files.uploadV2 (batch)", { fileCount: fileUploads.length, -@@ -4219,15 +4241,158 @@ +@@ -4219,14 +4244,157 @@ const result = await this._client.files.uploadV2(uploadArgs); this.logger.debug("Slack API: files.uploadV2 response", { ok: result.ok }); const fileIds = []; @@ -191,7 +220,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + ); + } + return { fileIds, messageId }; - } ++ } + async paperclipResolveFileUploadReceipt(fileIds, threadId) { + if (!Array.isArray(fileIds) || fileIds.length === 0 || fileIds.length > 20 || new Set(fileIds).size !== fileIds.length || fileIds.some((fileId) => typeof fileId !== "string" || !/^F[A-Z0-9]{1,254}$/.test(fileId))) { + const error = new Error("Slack file upload receipt is invalid"); @@ -296,11 +325,10 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + (messageId) => rest.every((messageIds) => messageIds.has(messageId)) + ); + return sharedMessageIds.length === 1 ? sharedMessageIds[0] : null; -+ } + } async editMessage(threadId, messageId, _message) { const message = await this.resolveMessageMentions(_message, threadId); - const ephemeral = this.decodeEphemeralMessageId(messageId); -@@ -4622,6 +4787,8 @@ +@@ -4622,6 +4790,8 @@ this.logger.debug("Slack: starting stream", { channel, threadTs }); const token = await this.getToken(); const streamer = this._client.chatStream({ @@ -309,7 +337,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 channel, thread_ts: threadTs, ...options?.recipientUserId && { -@@ -4667,6 +4834,72 @@ +@@ -4667,6 +4837,72 @@ } }; const fallback = { message: null, mode: "native", nativeRendered: false }; @@ -382,7 +410,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 const updateIntervalMs = options?.updateIntervalMs ?? 1e3; let fallbackSent = ""; let lastFallbackEditAt = 0; -@@ -4709,17 +4942,11 @@ +@@ -4709,17 +4945,11 @@ return; } try { @@ -403,7 +431,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 } switchToFallback(error); await flushFallback(force); -@@ -4736,12 +4963,15 @@ +@@ -4736,12 +4966,15 @@ return; } try { @@ -424,13 +452,26 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 structuredChunksSupported = false; this.logger.warn( "Structured streaming chunk failed, falling back to text-only streaming. Ensure your Slack app manifest includes the agent/assistant feature and the assistant:write scope", -@@ -4749,22 +4979,28 @@ +@@ -4749,22 +4982,28 @@ ); } }; - for await (const chunk of textStream) { - if (options?.signal?.aborted) { - break; +- } +- if (typeof chunk === "string") { +- renderer.push(chunk); +- await flushCommitted(); +- } else if (chunk.type === "markdown_text") { +- renderer.push(chunk.text); +- await flushCommitted(); +- } else { +- await sendStructuredChunk(chunk); +- } +- } +- renderer.finish(); +- await flushCommitted(true); + try { + for await (const chunk of textStream) { + if (options?.signal?.aborted) { @@ -445,29 +486,18 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + } else { + await sendStructuredChunk(chunk); + } - } -- if (typeof chunk === "string") { -- renderer.push(chunk); -- await flushCommitted(); -- } else if (chunk.type === "markdown_text") { -- renderer.push(chunk.text); -- await flushCommitted(); -- } else { -- await sendStructuredChunk(chunk); -- } ++ } + renderer.finish(); + await flushCommitted(true); + } catch (error) { + // Iterator/renderer/mention lookup failures also follow an irreversible + // accepted native prefix; never classify them as safe whole-send retry. + rethrowNativeFailure(error); - } -- renderer.finish(); -- await flushCommitted(true); ++ } if (fallback.mode === "fallback") { if (options?.stopBlocks || this.feedbackButtons) { this.logger.warn( -@@ -4775,7 +5011,7 @@ +@@ -4775,7 +5014,7 @@ this.logger.debug("Slack: fallback stream complete", { messageId: fallback.message?.id }); @@ -476,7 +506,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 return fallback.message; } const stopBlocks = [ -@@ -4784,21 +5020,27 @@ +@@ -4784,21 +5023,27 @@ ]; let result; try { @@ -507,7 +537,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 return fallback.message; } const messageTs = result.message?.ts ?? result.ts; -@@ -5636,7 +5878,8 @@ +@@ -5636,7 +5881,8 @@ const response = await fetch(responseUrl, { method: "POST", headers: { "Content-Type": "application/json" }, @@ -517,3 +547,17 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 }); if (!response.ok) { const errorText = await response.text(); +diff --git a/dist/index.d.ts b/dist/index.d.ts +--- a/dist/index.d.ts ++++ b/dist/index.d.ts +@@ -1179,8 +1179,9 @@ + openModal(triggerId: string, modal: ModalElement, contextId?: string): Promise<{ + viewId: string; + }>; +- updateModal(viewId: string, modal: ModalElement): Promise<{ ++ updateModal(viewId: string, modal: ModalElement, options?: { hash: string; externalId: string; contextId?: string }): Promise<{ + viewId: string; ++ viewHash?: string; + }>; + /** + * Upload files to Slack and share them to a channel. diff --git a/server/src/__tests__/chat-run-milestone-work.test.ts b/server/src/__tests__/chat-run-milestone-work.test.ts index 40d3ce01195..1fec371eed3 100644 --- a/server/src/__tests__/chat-run-milestone-work.test.ts +++ b/server/src/__tests__/chat-run-milestone-work.test.ts @@ -1,9 +1,9 @@ import { documentService } from "../services/documents.js"; import { createChatMilestoneFixtures } from "./helpers/chat-milestone-fixtures.js"; import { randomUUID } from "node:crypto"; -import { eq } from "drizzle-orm"; +import { eq, sql } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; -import { agents, agentWakeupRequests, caseHandoffs, cases, chatConversations, chatEndpoints, chatPublications, completionContracts, createDb, nativeRunFinalizations, nativeRunResults, statusDecisions, statusDecisionEffects, workAssessments, subscribeDatabaseWork, heartbeatRuns, issueComments, issueQuestionResponseDeliveries, issueThreadInteractions, issues } from "@paperclipai/db"; +import { applyPendingMigrations, agents, agentWakeupRequests, caseHandoffs, cases, chatConversations, chatEndpoints, chatPublications, completionContracts, createDb, nativeRunFinalizations, nativeRunResults, statusDecisions, statusDecisionEffects, workAssessments, subscribeDatabaseWork, heartbeatRuns, issueComments, issueQuestionResponseDeliveries, issueThreadInteractions, issues } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js"; @@ -25,6 +25,11 @@ import { NATIVE_STATUS_ARBITER_POLICY_VERSION, type NativeStatusDecision } from import { claimQueuedNativeReviewRun } from "../services/native-runtime/native-review-dispatch.js"; import { createChatPublicationResolution } from "../services/chat-channels/publications/resolution.js"; import { createChatPublicationWorker } from "../services/chat-channels/publications/worker.js"; +import { chatCredentialMutationLease } from "../services/chat-credential-mutation-lease.js"; +import { createSlackFormPublicationStage } from "../services/chat-channels/slack/form-publication.js"; +import { stageSlackFormEnrollment, reconcileSlackFormEnrollment } from "../services/chat-channels/slack/form-owner.js"; +import { parseSlackFormBinding } from "../services/chat-channels/slack/cloud-forms.js"; +import { chatActions } from "@paperclipai/db"; import { CAPABILITIES } from "../services/chat-channels/provider-catalog.js"; import { createChatPublicationFailures } from "../services/chat-channels/publications/failures.js"; import { logger } from "../middleware/logger.js"; @@ -46,7 +51,8 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu beforeAll(async () => { if (!external) temporary = await startEmbeddedPostgresTestDatabase("chat-milestones-"); - db = createDb(external ?? temporary!.connectionString, { maxConnections: 6, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); + else await applyPendingMigrations(external); + db = createDb(external ?? temporary!.connectionString, { maxConnections: 4, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); fixtures = createChatMilestoneFixtures(db); }, 60_000); const fixture = (companyId?: string) => fixtures.create(companyId); @@ -765,6 +771,151 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu expect((await publications()).map(row => row.idempotencyKey)).toContain(`run:${targetId}:queued:${f.endpointId}`); await quiet(); }); + it("parks the first managed form through seven waits and worker restart, then commits one send and its exact grant", async () => { + const identity = { instanceId: "publication-readiness-instance", environment: "test" as const }; + const [originalEndpoint] = await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)); + const binding = parseSlackFormBinding({ version: 1, instanceId: identity.instanceId, companyId: f.companyId, + endpointId: f.endpointId, connectionId: originalEndpoint.connectionId, publicId: "p".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); + const { credentialFingerprint: _fingerprint, eventBindingId, ...base } = binding; + const enrollment = { ...base, bindingId: eventBindingId }; + // Initial fixture configuration has a real pending enrollment, not an + // enrolled owner or fabricated grant/transport receipt. + await db.update(chatEndpoints).set({ publicId: binding.publicId, capabilities: CAPABILITIES.slack, setup: sql`${JSON.stringify({ runtimeGeneration: 1, + slackCloudInstanceId: identity.instanceId, slackCloudSubject: f.userId, + slackCloudEvents: { runtimeGeneration: 1, bindingId: eventBindingId, teamId: binding.teamId, appId: binding.appId } })}::jsonb` }).where(eq(chatEndpoints.id, f.endpointId)); + await db.update(chatConversations).set({ externalConversationId: "slack:CTEST", externalThreadId: "slack:CTEST:1770000000.000001" }).where(eq(chatConversations.id, f.conversationId)); + const ownerId = await db.transaction(tx => stageSlackFormEnrollment(tx, { enrollment, subject: f.userId, environment: identity.environment })); + const source = await run(f, { status: "failed", errorCode: "test_run_failed" }); + const interaction = await issueThreadInteractionService(db).create({ id: f.issueId, companyId: f.companyId }, { + kind: "ask_user_questions", continuationPolicy: "wake_assignee", sourceRunId: source.id, + title: "Choose", payload: { version: 1, title: "Choose", questions: ["first", "second"].map(id => ({ + id, prompt: `Choose ${id}`, selectionMode: "single" as const, required: true, options: [{ id: "yes", label: "Yes" }], + })) }, + }, { agentId: f.agentId, runId: source.id }); + const prompt = (await publications()).find(row => row.idempotencyKey === `interaction:${interaction.id}:${f.endpointId}`)!; + const issued = (await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))) + .filter(row => ["question_form_open", "question_form_submit"].includes(row.kind) && row.payload.publicationId === prompt.id); + expect(issued.map(row => row.kind).sort()).toEqual(["question_form_open", "question_form_submit"]); + const read = async () => (await db.select().from(chatPublications).where(eq(chatPublications.id, prompt.id)))[0]!; + const post = vi.fn(async () => ({ id: "1770000001.000001" })); + const failures = createChatPublicationFailures(db, { + FailedChatRetryPublicationReadError: class extends Error {}, MAX_ERROR_TEXT: 1000, + getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null, + }); + // These are the same controlled authorization/SDK ports as the adjacent + // receipt-race control. Attempt claims, credential leases, enrollment, + // publication receipt and grant transactions are real PostgreSQL owners. + const makeWorker = () => createChatPublicationWorker(db, ({ + CAPABILITIES, settleGitHubAutomaticPublication: async () => false, + ensureDurablePublicationTransport: async publication => publication, + tryProcessTeamsFilePublication: async () => false, + endpointRecord: async () => ({ endpoint: (await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!, + assignedAgentName: "Maya", connectionName: "Chat", applicationId: "test", credentialSecretRefs: [] }), + runtimeContextForRecord: () => ({ generation: 1, localEpoch: 0, credentialFingerprint: "test", version: "test" }), + hasCommittedTaskControlCompletion: async () => false, nonDirectDestinationAllowed: () => true, + runOwnershipMilestoneSupersessionReason: async () => null, progressSupersededByFastResponse: async () => false, + runProgressSupersededByPublishedInteraction: async () => false, currentTaskControlPayload: async publication => publication.payload, + withCredentialMutationLease: chatCredentialMutationLease(db), + closeProgressToReplace: async () => null, interactionResolutionPublicationToReplace: async () => null, + interactionPromptPublicationToReplace: async () => null, runPublicationToReplace: async () => null, + inboundWakePublicationToReplace: async () => null, taskStatusPublicationToReplace: async () => null, + claimPublicationTransportAuthorization: async () => ({ endpoint: (await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!, + conversation: (await db.select().from(chatConversations).where(eq(chatConversations.id, f.conversationId)))[0]!, authorizationActionId: null, preparedText: null }), + prepareSlackFormPublication: createSlackFormPublicationStage(db), postSafePublication: post, + commitTaskControlCompletion: async () => {}, receiptReactionCompletionRunId: async () => null, + finalizePublicationFailure: failures.finalizePublicationFailure, redactError: String, + } satisfies Partial[1]>) as unknown as Parameters[1]); + for (let wait = 0; wait < 7; wait++) { + const before = Date.now(); + // A new worker reconstructs from the same durable publication each time. + await makeWorker().processSelectedPublication(await read()); + expect(post).not.toHaveBeenCalled(); + const parked = await read(); + expect(parked).toMatchObject({ state: "retry", attempts: 0, providerMessageId: null, + redactedError: "form_publication_enrollment_pending", payload: prompt.payload }); + expect(parked.nextAttemptAt!.getTime()).toBeGreaterThanOrEqual(before + 2000); + expect(parked.nextAttemptAt!.getTime()).toBeLessThanOrEqual(Date.now() + 2000); + await vi.advanceTimersByTimeAsync(2000); + } + let logicalEnrollments = 0; + const protocol = vi.fn(async () => { if (++logicalEnrollments === 1) throw new Error("Controlled enrollment reply lost after external acceptance"); return { binding }; }); + const reconcile = () => reconcileSlackFormEnrollment(db, { actionId: ownerId, identity, protocol, + resolveBotToken: async () => "xoxb-controlled-enrollment" }); + await expect(reconcile()).rejects.toThrow("Controlled enrollment reply lost"); + expect(post).not.toHaveBeenCalled(); + expect((await db.select().from(chatActions).where(eq(chatActions.id, ownerId)))[0]!.result?.phase).toBe("staged"); + await reconcile(); + expect((await db.select().from(chatActions).where(eq(chatActions.id, ownerId)))[0]!.result?.phase).toBe("enrolled"); + const selected = await read(); + await makeWorker().processSelectedPublication(selected); + expect(post).toHaveBeenCalledTimes(1); + expect(await read()).toMatchObject({ state: "published", attempts: 1, providerMessageId: "1770000001.000001" }); + const grants = (await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))).filter(row => row.kind === "slack_form_grant"); + expect(grants).toHaveLength(1); + expect(grants[0]!.payload).toMatchObject({ binding, subject: f.userId, environment: identity.environment, + grant: { publicationId: prompt.id, interactionId: interaction.id, providerMessageId: "1770000001.000001" } }); + // Lost local receipt response/reconstruction cannot re-claim the old exact attempt. + await makeWorker().processSelectedPublication(selected); + expect(post).toHaveBeenCalledTimes(1); + expect((await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))).filter(row => row.kind === "slack_form_grant")).toEqual(grants); + }); + + it("an untyped pre-send error with the same diagnostic still exhausts its ordinary five-attempt budget", async () => { + const failures = createChatPublicationFailures(db, { FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null }); + const [row] = await db.insert(chatPublications).values({ companyId: f.companyId, endpointId: f.endpointId, + conversationId: f.conversationId, issueId: f.issueId, idempotencyKey: `untyped-preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 5 }).returning(); + await expect(failures.finalizePublicationFailure({ ...row, attempts: 4 }, new Error("form_publication_enrollment_pending"), + false, null, undefined, "not_started")).resolves.toBe(true); + expect((await db.select().from(chatPublications).where(eq(chatPublications.id, row.id)))[0]).toMatchObject({ state: "failed", attempts: 5, nextAttemptAt: null }); + }); + + it.each([ + ["not_started", false, "retry", 4], + ["possible", false, "delivery_unknown", 5], + ["not_started", true, "delivery_unknown", 5], + ] as const)("typed readiness at attempt five respects invocation=%s accepted=%s", async (providerInvocation, providerAccepted, state, attempts) => { + const failures = createChatPublicationFailures(db, { FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null }); + const [row] = await db.insert(chatPublications).values({ companyId: f.companyId, endpointId: f.endpointId, + conversationId: f.conversationId, issueId: f.issueId, idempotencyKey: `typed-preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 5 }).returning(); + await expect(failures.finalizePublicationFailure({ ...row, attempts: 4 }, new Error("form_publication_enrollment_pending"), + providerAccepted, null, undefined, providerInvocation, + { kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2000 })).resolves.toBe(true); + const [settled] = await db.select().from(chatPublications).where(eq(chatPublications.id, row.id)); + expect(settled).toMatchObject({ state, attempts, providerMessageId: null, payload: row.payload }); + if (state !== "retry") expect(settled.nextAttemptAt).toBeNull(); + }); + + it.each([ + ["not_started", false, "retry"], + ["possible", false, "delivery_unknown"], + ["not_started", true, "delivery_unknown"], + ] as const)("settles local preparation context %s with accepted=%s as %s", async (providerInvocation, providerAccepted, state) => { + const [claimed] = await db.insert(chatPublications).values({ + companyId: f.companyId, endpointId: f.endpointId, conversationId: f.conversationId, + issueId: f.issueId, idempotencyKey: `preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 1, + }).returning(); + const failures = createChatPublicationFailures(db, { + FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, + invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null, + }); + const before = Date.now(); + await expect(failures.finalizePublicationFailure({ ...claimed, attempts: 0 }, + new Error("form_publication_enrollment_pending"), providerAccepted, null, undefined, providerInvocation)).resolves.toBe(true); + const [settled] = await db.select().from(chatPublications).where(eq(chatPublications.id, claimed.id)); + expect(settled).toMatchObject({ state, attempts: 1, providerMessageId: null, payload: claimed.payload }); + if (state === "retry") { + expect(settled.nextAttemptAt?.getTime()).toBeGreaterThanOrEqual(before + 2000); + expect(settled.nextAttemptAt?.getTime()).toBeLessThanOrEqual(Date.now() + 2000); + expect(settled.redactedError).toBe("form_publication_enrollment_pending"); + } else expect(settled.nextAttemptAt).toBeNull(); + }); it.each([ ["retry", null], ["failed", null], ["delivery_unknown", "cancel"], ["delivery_unknown", "mark_delivered"], ] as const)("rechecks resolved prompt ownership after delivery %s (%s)", async (state, resolution) => { diff --git a/server/src/__tests__/cloud-control-task-drain.test.ts b/server/src/__tests__/cloud-control-task-drain.test.ts index 46c2962edc5..985c7188325 100644 --- a/server/src/__tests__/cloud-control-task-drain.test.ts +++ b/server/src/__tests__/cloud-control-task-drain.test.ts @@ -1,6 +1,6 @@ import { connectionEventWakeRoutes } from "../routes/connection-event-wake.js"; import { registerConnectionEventWake } from "../services/connection-event-wake.js"; -import { generateKeyPairSync, sign } from "node:crypto"; +import { createHash, generateKeyPairSync, sign } from "node:crypto"; import express from "express"; import request from "supertest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; @@ -382,3 +382,65 @@ describe("cloudControlMiddleware", () => { expect(res.status).toBe(200); }); }); + + +describe("body-bound Cloud idle status assertion", () => { + const body = Buffer.from(JSON.stringify({ idleCapabilities: { deadlineCatalog: "domain_deadlines_v1" } })); + const digest = (value: Uint8Array) => createHash("sha256").update(value).digest("hex"); + const token = (raw: Uint8Array, extra: Record = {}) => controlAssertion({ action: "task-drain:read-body", claims: { bodySha256: digest(raw), ...extra } }); + const read = (compactJws: string, rawBody?: Uint8Array) => verifyCloudControlAssertion({ compactJws, rawBody, + expectedAction: "task-drain:read-body", env: ENV, now: NOW }); + beforeEach(() => resetCloudControlReplayFenceForTests()); + it("installs the complete closed body only after its exact-byte binding and rejects replay", () => { + const jws = token(body); + const claims = read(jws, body); + expect(claims.idleCapabilities).toEqual({ deadlineCatalog: "domain_deadlines_v1" }); + expect(Object.isFrozen(claims.idleCapabilities)).toBe(true); + expect(() => read(jws, body)).toThrow(/already been used/); + }); + it("rejects changed or missing bytes without burning the valid exact request", () => { + const jws = token(body); + expect(() => read(jws)).toThrow(/body binding/); + expect(() => read(jws, Buffer.concat([body, Buffer.from(" ")]))).toThrow(/digest changed/); + expect(read(jws, body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("rejects malformed UTF8 before consuming the same nonce", () => { + const requestId = "invalid-utf8-status-read"; + const malformed = Buffer.concat([Buffer.from('{"idleCapabilities":{"deadlineCatalog":"'), Buffer.from([0xff]), Buffer.from('"}}')]); + expect(() => read(token(malformed, { requestId }), malformed)).toThrow(TypeError); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it.each([ + ["unknown catalog", '{"idleCapabilities":{"deadlineCatalog":"unowned"}}'], + ["extra field", '{"idleCapabilities":{"deadlineCatalog":"domain_deadlines_v1"},"extra":true}'], + ["array body", '[{"idleCapabilities":{"deadlineCatalog":"domain_deadlines_v1"}}]'], + ["malformed JSON", "{"], + ])("rejects %s before consuming the same nonce", (_name, text) => { + const requestId = "rejected-status-body"; + const raw = Buffer.from(text); + expect(() => read(token(raw, { requestId }), raw)).toThrow(); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("rejects mixed embedded capabilities and malformed digest without consuming a nonce", () => { + for (const extra of [{ idleCapabilities: { deadlineCatalog: "domain_deadlines_v1" } }, { bodySha256: "invalid" }]) { + resetCloudControlReplayFenceForTests(); + const requestId = "mixed-status-body"; + expect(() => read(token(body, { requestId, ...extra }), body)).toThrow(/body binding/); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + } + }); + it("enforces the unchanged ten MiB JSON limit before consuming the same nonce", () => { + const raw = Buffer.alloc(10 * 1024 * 1024 + 1, 32), requestId = "oversized-status-body"; + expect(() => read(token(raw, { requestId }), raw)).toThrow(/body binding/); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("does not turn body-read authority into start or stop authority", () => { + for (const action of ["task-drain:start", "task-drain:stop"] as const) { + const jws = token(body); + expect(() => verifyCloudControlAssertion({ compactJws: jws, expectedAction: action, rawBody: body, env: ENV, now: NOW })).toThrow(/authorize/); + expect(read(jws, body).action).toBe("task-drain:read-body"); + expect(() => verifyCloudControlAssertion({ compactJws: controlAssertion({ action, claims: { bodySha256: digest(body) } }), + expectedAction: action, rawBody: body, env: ENV, now: NOW })).toThrow(/Body|body/); + } + }); +}); diff --git a/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs b/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs new file mode 100755 index 00000000000..87c115216ac --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs @@ -0,0 +1,19 @@ +#!/usr/bin/env node +// Controlled provider executable, NOT the real Codex model or service. +const assert = require("node:assert/strict"), fs = require("node:fs"); +require("./scoped-cli-prewire.cjs"); +assert.equal(process.versions.node.split(".")[0], "24", "The qualification Node24 executable is required"); +const version = process.argv.length === 3 && process.argv[2] === "--version"; +fs.appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify({phase:"controlled-cli-entry",pid:process.pid,ppid:process.ppid, + execPath:process.execPath,argv:process.argv,execArgv:process.execArgv,versionProbe:version,realModelProvider:false})+"\n"); +if (version) { console.log("paperclip-controlled-codex-jsonl-fixture (no real model)"); } +else { + assert.ok(process.argv.slice(2).includes("exec"), "Only actual normal Codex exec invocation accepted"); + assert.ok(process.env.PAPERCLIP_RUN_ID && process.env.PAPERCLIP_API_KEY, "Normal run token required"); + // Consume normal prompt stdin without executing it or leaking it to logs. + (async () => { + let size=0; for await (const chunk of process.stdin) { size+=chunk.length; assert.ok(size<=2*1024*1024,"Controlled prompt bound"); } + console.log(JSON.stringify({type:"thread.started",thread_id:`controlled-${process.env.PAPERCLIP_RUN_ID}`})); + await import("./normal-scoped-useful-final.mjs"); + })().catch(error=>{console.error(error);process.exitCode=1;}); +} diff --git a/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs b/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs new file mode 100644 index 00000000000..5dca71db88a --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs @@ -0,0 +1,400 @@ +import assert from "node:assert/strict"; +import { createHash, generateKeyPairSync, randomBytes, randomUUID } from "node:crypto"; +import { once } from "node:events"; +import { writeFile } from "node:fs/promises"; +import { createServer } from "node:http"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import path from "node:path"; +import { realpath, readFile } from "node:fs/promises"; + +const cloudRoot = await realpath(process.env.PAPERCLIP_CLOUD_REPO_ROOT ?? ""); +const cloudPackage = JSON.parse(await readFile(path.join(cloudRoot, "package.json"), "utf8")); +assert.equal(cloudPackage.name, "@paperclip/cloud-harness"); +const { createIdleIncomingWakeCapability } = await import(pathToFileURL(path.join(cloudRoot, "dist/sleep/incoming-wake-capability.js")).href); +const { default: pg } = await import(pathToFileURL(path.join(cloudRoot, "node_modules/pg/lib/index.js")).href); +const { PostgresCloudHarnessRegistry } = await import(pathToFileURL(path.join(cloudRoot, "dist/registry/postgres.js")).href); +const { PostgresConnectorStore } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/postgres.js")).href); +const { ConnectorBroker } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/broker.js")).href); +const { SLACK_FORM_OPERATION_BYTES } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-forms.js")).href); +const { ConnectorEventDeliveryWorker, createConnectorEventNotifier } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/event-delivery.js")).href); +const { providerRegistryFromEnvironment } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/providers.js")).href); +const { handleTenantSlackForm } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-form-http.js")).href); +const { handleTenantSlackEvent } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-event-http.js")).href); +const { InMemoryProviderSecretStore } = await import(pathToFileURL(path.join(cloudRoot, "dist/providers/neon.js")).href); +const { FakeCloudStackProvider } = await import(pathToFileURL(path.join(cloudRoot, "dist/providers/fake.js")).href); +const { CloudStackProvisioner } = await import(pathToFileURL(path.join(cloudRoot, "dist/provisioner/provisioner.js")).href); +const { createRuntimeIdentitySignerFromEnv } = await import(pathToFileURL(path.join(cloudRoot, "dist/runtime/runtime-identity-assertion.js")).href); +assert.equal(process.versions.node.split(".")[0], "22"); +assert.equal(process.argv.length, 3); +assert.ok(process.env.SLACK_FORM_FIXTURE_SCOPES?.length); +const stateFile = process.argv[2]; +assert.ok(stateFile && path.isAbsolute(stateFile)); +const databaseUrl = process.env.SLACK_EVENT_TEST_DATABASE_URL; +const database = new URL(databaseUrl ?? ""); +assert.equal(database.protocol, "postgres:"); +assert.equal(database.hostname, "127.0.0.1"); +assert.match(database.port, /^[0-9]+$/, "Explicit database port required before Pool or migrations"); +const databasePort = Number(database.port); +assert.ok(Number.isInteger(databasePort) && databasePort >= 1 && databasePort <= 65535); +assert.match(database.pathname, /^\/sleep2_slack_form_[a-z0-9_]+_cloud_test$/); +const pool = new pg.Pool({ connectionString: databaseUrl, max: 2 }); +const registry = new PostgresCloudHarnessRegistry(pool); +const store = new PostgresConnectorStore(pool); +const secrets = new InMemoryProviderSecretStore(); +const signing = generateKeyPairSync("ed25519"), sealing = generateKeyPairSync("x25519"), control = generateKeyPairSync("ed25519"); +const controlPublic = { ...control.publicKey.export({ format: "jwk" }), kid: "slack-form-journey", use: "sig", alg: "EdDSA" }; +const signer = createRuntimeIdentitySignerFromEnv({ + CLOUD_HARNESS_RUNTIME_IDENTITY_SIGNING_JWK: JSON.stringify({ ...control.privateKey.export({ format: "jwk" }), kid: controlPublic.kid, use: "sig", alg: "EdDSA" }), + CLOUD_HARNESS_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [controlPublic] }), +}, { required: true }); +assert.ok(signer?.signControlAssertion); +const fixture = { + companyId: randomUUID(), endpointId: randomUUID(), connectionId: randomUUID(), applicationId: randomUUID(), agentId: randomUUID(), + actorId: `form-owner-${randomUUID()}`, instanceId: `form-stack-${randomUUID()}`, publicId: randomBytes(32).toString("base64url"), + teamId: "TTEST", appId: "ATEST", botId: "UBOT", userId: "UOPERATOR", channelId: "CTEST", threadTs: "1788.100000", + botToken: "xoxb-controlled-journey-only", signingSecret: randomBytes(16).toString("hex"), controlNonce: randomBytes(32).toString("base64url"), + signPrivate: signing.privateKey.export({ format: "jwk" }).d, sealPrivate: sealing.privateKey.export({ format: "jwk" }).d, + controlJwks: { keys: [controlPublic] }, +}; +const failures = [], calls = [], views = new Map(), triggers = new Map(), postedMessages = new Map(); +const faults = { dropImmediateHint: false, loseTerminalReply: false, loseAckReply: false }; +let tenantOrigin = process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN, broker, worker, stopping = false, wakerStarts = 0, droppedHints = 0, lostTerminalReplies = 0, lostAckReplies = 0; +const json = (res, status, body) => { res.writeHead(status, { "content-type": "application/json", connection: "close" }); res.end(JSON.stringify(body)); }; +const read = async req => { + const chunks = []; let size = 0; + for await (const chunk of req) { size += chunk.length; assert.ok(size <= 2_100_000); chunks.push(Buffer.from(chunk)); } + return Buffer.concat(chunks); +}; +const loopback = value => { const url = new URL(value); assert.equal(url.protocol, "http:"); assert.equal(url.hostname, "127.0.0.1"); assert.equal(url.pathname, "/"); return url.origin; }; +tenantOrigin = loopback(tenantOrigin); +fixture.tenantOrigin = tenantOrigin; +const notification = createConnectorEventNotifier({ + getStack: async id => { + assert.equal(id, fixture.instanceId); + const stack = await registry.getStack(id); + if (!stack || !tenantOrigin) return undefined; + return { id: stack.id, lifecycleState: stack.lifecycleState, providerRefs: [{ provider: "railway", resourceType: "service_domain:web", resourceId: "fixture-tenant", name: "tenant.example", metadata: { upstreamOrigin: tenantOrigin } }] }; + }, + wake: async id => { assert.equal(id, fixture.instanceId); return { status: "awake" }; }, + sign: input => signer.signControlAssertion(input), +}); +const hint = async () => { const instance = await store.getInstance(fixture.instanceId); assert.ok(instance); await notification(instance); }; +const restartWaker = async () => { + await worker?.stop(); + worker = new ConnectorEventDeliveryWorker({ store, deliver: notification, onError: error => failures.push(String(error?.stack ?? error)) }); + wakerStarts++; worker.start(); +}; +// PRIVATE controlled-provider wire witnesses: never derive from App rows/ACKs. +const wireRequests = new WeakMap(); +const receivedSessionStatuses = new Map(); +const wireMethods = new Set(["auth.test", "users.info", "conversations.info", "conversations.list", "conversations.open", "conversations.history", "conversations.replies", + "reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral", "chat.postMessage", "chat.scheduleMessage", "chat.deleteScheduledMessage", "chat.delete", + "chat.startStream", "chat.appendStream", "chat.stopStream", "agents.sessions.setStatus", "assistant.threads.setStatus", "assistant.threads.setTitle", "assistant.threads.setSuggestedPrompts", "views.open", "views.update"]); +function usefulReplyWitness(message) { + if (message.channel !== fixture.channelId || message.user !== fixture.botId || !postedMessages.has(message.ts) + || ![fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(message.thread_ts)) return null; + // The pinned SDK sends markdown_text exclusively for {markdown}; raw/string + // messages use text. Do not scan unrelated JSON/metadata for an apparent run. + let text; + if (typeof message.markdown_text === "string") { + if (message.text !== undefined || message.blocks !== undefined) return null; + text = message.markdown_text; + } else if (typeof message.text === "string" && message.blocks === undefined) text = message.text; + else return null; + const match = /^Scoped Slack useful reply for run ([a-f0-9-]{36})$/.exec(text); + if (!match || match[0] !== text) return null; + return { id: message.ts, threadTs: message.thread_ts, runId: match[1] }; +} +function wireSourceFrames(error) { + return String(error?.stack ?? "").split("\n").slice(1, 9).flatMap(frame => { + const match = /slack-form-journey-host\.mjs:(\d+):(\d+)/.exec(frame); + return match ? [`controlled-provider:${match[1]}:${match[2]}`] : []; + }); +} +const platform = async (req, res, pathname) => { + const raw = await read(req), encoding = req.headers["content-type"] ?? ""; + const body = encoding.includes("application/json") ? JSON.parse(raw.toString()) : Object.fromEntries(new URLSearchParams(raw.toString())); + if (typeof body.view === "string") body.view = JSON.parse(body.view); + const method = pathname.slice("/fixture/slack/".length); + assert.ok(req.method === "POST" || req.method === "GET" && method === "conversations.list", "fixture_method_contract"); + const token = req.headers.authorization?.replace(/^Bearer /, "") ?? body.token; + assert.equal(token, fixture.botToken); + wireRequests.set(req, { method: wireMethods.has(method) ? method : "unsupported-slack-method", methodDigest: createHash("sha256").update(method).digest("hex"), + ownedProviderId: typeof body.ts === "string" && postedMessages.has(body.ts) ? body.ts + : typeof body.oldest === "string" && postedMessages.has(body.oldest) ? body.oldest : null }); + calls.push({ method, at: Date.now(), ...(method.startsWith("views.") ? { body } : {}) }); + if (method === "auth.test") { + res.setHeader("x-oauth-scopes", process.env.SLACK_FORM_FIXTURE_SCOPES ?? ""); + return json(res, 200, { ok: true, team_id: fixture.teamId, user_id: fixture.botId, user: "fixture" }); + } + if (method === "users.info") return json(res, 200, { ok: true, user: { id: fixture.userId, name: "operator", is_bot: false, profile: { display_name: "Operator" } } }); + if (method === "conversations.info") return json(res, 200, { ok: true, channel: { id: fixture.channelId, name: "fixture", is_member: true } }); + if (method === "conversations.list") return json(res, 200, { ok: true, channels: [{ id: fixture.channelId, name: "fixture", is_member: true }], response_metadata: { next_cursor: "" } }); + if (method === "agents.sessions.setStatus") { + // Exact production setSlackSessionStatus wire contract; record the real + // idempotent effect before acknowledging it. Never settle App rows here. + assert.equal(req.method, "POST"); assert.ok(encoding.includes("application/json")); + assert.deepEqual(Object.keys(body).sort(), ["channel_id", "status", "thread_ts"]); + assert.equal(body.channel_id, fixture.channelId); + assert.ok([fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(body.thread_ts)); + assert.ok(["processing", "active", "suspended", "closed"].includes(body.status)); + const previous = receivedSessionStatuses.get(body.thread_ts) ?? null; + receivedSessionStatuses.set(body.thread_ts, body.status); + console.info(JSON.stringify({ event: "controlled-slack-session-status-received", method, + methodDigest: createHash("sha256").update(method).digest("hex"), status: 200, + ownedChannel: true, ownedThreadTs: body.thread_ts, previousStatus: previous, sessionStatus: body.status, + effectRecordedBeforeAcknowledgement: receivedSessionStatuses.get(body.thread_ts) === body.status })); + return json(res, 200, { ok: true }); + } + if (["reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral"].includes(method)) return json(res, 200, { ok: true, ts: fixture.threadTs, channel: fixture.channelId }); + if (method === "chat.postMessage") { + assert.equal(body.channel, fixture.channelId); + const ts = `1788.${String(200000 + calls.length).padStart(6, "0")}`; + const message = { ...body, ts, user: fixture.botId }; + postedMessages.set(ts, message); + const witness = usefulReplyWitness(message); + console.info(JSON.stringify({ event: "controlled-slack-message-received", method: "chat.postMessage", status: 200, + ownedProviderId: ts, ownedChannel: message.channel === fixture.channelId, + ownedThread: [fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(message.thread_ts), + contentField: typeof message.markdown_text === "string" ? "markdown_text" : typeof message.text === "string" ? "text" : "other", + exactUsefulText: witness !== null, usefulRunId: witness?.runId ?? null, + contentDigest: typeof message.markdown_text === "string" || typeof message.text === "string" + ? createHash("sha256").update(message.markdown_text ?? message.text).digest("hex") : null })); + return json(res, 200, { ok: true, ts, channel: fixture.channelId, message }); + } + if (method === "conversations.replies") { + assert.equal(body.channel, fixture.channelId); + assert.ok([fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(body.ts)); + assert.equal(body.limit, "1"); + assert.equal(body.inclusive, "true"); + assert.equal(body.cursor, undefined); + assert.equal(body.latest, undefined); + assert.equal(typeof body.oldest, "string"); + const message = postedMessages.get(body.oldest); + assert.ok(message, "only a message created by this fixture's real chat.postMessage may be fetched"); + assert.equal(message.thread_ts, body.ts); + if (Object.hasOwn(body, "token")) assert.equal(body.token, fixture.botToken); + assert.deepEqual(Object.keys(body).filter(key => key !== "token").sort(), ["channel", "inclusive", "limit", "oldest", "ts"]); + return json(res, 200, { ok: true, messages: [message], has_more: false, response_metadata: { next_cursor: "" } }); + } + if (method === "views.open") { + const deadline = triggers.get(body.trigger_id); + assert.ok(deadline, "only the original issued trigger may open a view"); + triggers.delete(body.trigger_id); + if (Date.now() >= deadline) return json(res, 200, { ok: false, error: "expired_trigger_id" }); + const targets = await pool.query("SELECT delivery_id FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged'", [fixture.instanceId]); + assert.equal(targets.rows.length, 1, "the original target must be durable before trigger spend"); + assert.equal(body.view.notify_on_close, true); + assert.ok(!body.view.blocks.some(block => block.type === "input")); + assert.equal(body.view.callback_id, "paperclip_slack_form_loading_v1"); + const view = { ...body.view, id: "VJOURNEY", hash: "loading-hash", app_id: fixture.appId, state: { values: {} } }; + views.set(view.id, view); return json(res, 200, { ok: true, view }); + } + if (method === "views.update") { + const old = views.get(body.view_id); assert.ok(old); + assert.equal(body.hash, old.hash); assert.equal(body.view.external_id, old.external_id); + assert.equal(body.view.notify_on_close, true); assert.ok(body.view.blocks.some(block => block.type === "input")); + assert.ok(tenantOrigin); + const response = await fetch(`${tenantOrigin}/__fixture/committed-hold`, { headers: { "x-fixture-control": fixture.controlNonce } }); + assert.equal(response.status, 200); + const hold = await response.json(); + assert.equal(hold.status, "processing"); assert.equal(hold.phase, "update_attempted"); + assert.deepEqual(hold.view, { id: old.id, hash: old.hash, externalId: old.external_id }); + assert.equal(hold.operationId, old.external_id.slice("pc-form-".length)); + const view = { ...old, ...body.view, hash: "useful-hash" }; + views.set(view.id, view); return json(res, 200, { ok: true, view }); + } + throw new Error(`Unexpected actual SDK request ${method}`); +}; +const snapshot = async () => ({ + sessionStatuses: [...receivedSessionStatuses].map(([threadTs, status]) => ({ threadTs, status })), + usefulReplies: [...postedMessages.values()].flatMap(message => { + const witness = usefulReplyWitness(message); + return witness ? [witness] : []; + }), + calls, views: [...views.values()], failures, wakerStarts, droppedHints, lostTerminalReplies, lostAckReplies, + instance: await store.getInstance(fixture.instanceId), + bindings: (await store.listWebhookBindings(fixture.instanceId)).map(row => ({ id: row.id, profile: row.profile, metadata: row.providerMetadata })), + targets: (await pool.query("SELECT delivery_id,lease_id,lease_expires_at FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged' ORDER BY delivery_id", [fixture.instanceId])).rows, +}); +const server = createServer((req, res) => { void (async () => { + const url = new URL(req.url, fixture.origin); + if (url.pathname.startsWith("/__fixture/")) { + assert.equal(req.headers["x-fixture-control"], fixture.controlNonce); + if (req.method === "GET" && url.pathname === "/__fixture/state") return json(res, 200, await snapshot()); + if (req.method === "POST" && url.pathname === "/__fixture/max-header") { + assert.ok(tenantOrigin); + const instance = await store.getInstance(fixture.instanceId); assert.ok(instance); + const initial = await store.listWebhookBindings(fixture.instanceId); + assert.equal(initial.length, 2); + assert.ok(broker.slackEvents && broker.slackForms); + for (let index = 1; index < 500; index++) { + const event = await broker.slackEvents.enroll(instance, fixture.actorId, { + companyId: fixture.companyId, endpointId: randomUUID(), connectionId: randomUUID(), publicId: randomBytes(32).toString("base64url"), + runtimeGeneration: 1, teamId: fixture.teamId, appId: `AHEADER${index}`, signingSecret: randomBytes(16).toString("hex"), + }); + await broker.slackForms.enroll(instance, fixture.actorId, event, `xoxb-controlled-header-${index}`); + } + const rows = await store.listWebhookBindings(fixture.instanceId); assert.equal(rows.length, 1000); + const start = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:start", requestId: randomUUID() }); + const started = await fetch(`${tenantOrigin}/api/instance/task-drain`, { method: "POST", redirect: "error", + headers: { "content-type": "application/json", "x-paperclip-cloud-control": start }, body: JSON.stringify({ purpose: "idle", ttlMs: 30000 }), signal: AbortSignal.timeout(10000) }); + assert.equal(started.status, 200); const drain = await started.json(); assert.equal(typeof drain.ownerId, "string"); + let primary, result; + try { + const incomingWake = await createIdleIncomingWakeCapability({ stackId: fixture.instanceId, ownerId: drain.ownerId, + apiEnabled: false, brokerEnabled: true, store, readSlackCoverage: id => broker.readSlackCoverage(id) }); + assert.ok(incomingWake?.slackBroker); assert.equal(incomingWake.slackBroker.entries.length, 500); + const token = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:read", requestId: randomUUID(), idleCapabilities: { incomingWake } }); + const url = `${tenantOrigin}/api/instance/task-drain?idleSleepSafety=1&idleSleepDiagnostics=1&ownerId=${encodeURIComponent(drain.ownerId)}`; + const baseline = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:read", requestId: randomUUID() }); + const tampered = baseline.slice(0, -1) + (baseline.endsWith("A") ? "B" : "A"); + const denied = await fetch(url, { headers: { "x-paperclip-cloud-control": tampered }, redirect: "error", signal: AbortSignal.timeout(10000) }); + const tamperedStatus = denied.status; await denied.arrayBuffer(); assert.equal(tamperedStatus, 401); + const response = await fetch(url, { headers: { "x-paperclip-cloud-control": token }, redirect: "error", signal: AbortSignal.timeout(10000) }); + const raw = await response.text(); let report; try { report = JSON.parse(raw); } catch {} + result = { bindingCount: rows.length, coverageEntries: incomingWake.slackBroker.entries.length, + headerBytes: Buffer.byteLength(token), responseStatus: response.status, normalRouteReached: !!report?.idleSleepSafety, tamperedStatus }; + console.info(JSON.stringify({ diagnostic: "slack_form_header_wire", ...result })); + } catch (error) { primary = error; throw error; } + finally { + const token = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:stop", requestId: randomUUID() }); + try { const response = await fetch(`${tenantOrigin}/api/instance/task-drain?ownerId=${encodeURIComponent(drain.ownerId)}`, { method: "DELETE", + headers: { "x-paperclip-cloud-control": token }, redirect: "error", signal: AbortSignal.timeout(10000) }); + assert.equal(response.status, 200); await response.arrayBuffer(); + } catch (error) { throw primary ? new AggregateError([primary, error], "Header preflight and normal hold cleanup failed") : error; } + } + return json(res, 200, result); + } + if (req.method === "POST" && url.pathname === "/__fixture/idle-projection") { + assert.ok(tenantOrigin); + const request = async (method, action, suffix, body, idleCapabilities) => { + const response = await fetch(`${tenantOrigin}/api/instance/task-drain${suffix}`, { method, redirect: "error", + headers: { "content-type": "application/json", "x-paperclip-cloud-control": signer.signControlAssertion({ stackId: fixture.instanceId, + action, requestId: randomUUID(), ...(idleCapabilities ? { idleCapabilities } : {}) }) }, + ...(body ? { body: JSON.stringify(body) } : {}), signal: AbortSignal.timeout(10_000) }); + assert.equal(response.status, 200); return await response.json(); + }; + const drain = await request("POST", "task-drain:start", "", { purpose: "idle", ttlMs: 30_000 }); + assert.equal(typeof drain.ownerId, "string"); + let primary, projection; + try { + const incomingWake = await createIdleIncomingWakeCapability({ stackId: fixture.instanceId, ownerId: drain.ownerId, + apiEnabled: false, brokerEnabled: true, store, readSlackCoverage: id => broker.readSlackCoverage(id) }); + const report = await request("GET", "task-drain:read", `?idleSleepSafety=1&idleSleepDiagnostics=1&ownerId=${encodeURIComponent(drain.ownerId)}`, + undefined, incomingWake ? { incomingWake } : undefined); + const safety = report.idleSleepSafety; assert.ok(safety?.diagnostics); + assert.ok(["none", "present", "unknown"].includes(safety.backgroundWork)); + assert.ok(safety.diagnostics.checks.length <= 200); + projection = { backgroundWork: safety.backgroundWork, capabilityPresent: Boolean(incomingWake), + checks: safety.diagnostics.checks.map(value => ({ reason: value.reason, state: value.state })) }; + } catch (error) { primary = error; throw error; } + finally { + try { await request("DELETE", "task-drain:stop", `?ownerId=${encodeURIComponent(drain.ownerId)}`); } + catch (error) { throw primary ? new AggregateError([primary, error], "Idle report and hold cleanup failed") : error; } + } + return json(res, 200, { idleProjection: projection }); + } + const body = JSON.parse((await read(req)).toString() || "{}"); + if (req.method === "POST" && url.pathname === "/__fixture/tenant") { assert.equal(loopback(body.origin), tenantOrigin); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/trigger") { assert.equal(typeof body.id, "string"); assert.equal(typeof body.actionTs, "string"); assert.ok(!triggers.has(body.id)); triggers.set(body.id, Number(body.actionTs) * 1000 + 3000); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/faults") { for (const key of Object.keys(body)) { assert.ok(Object.hasOwn(faults, key)); assert.equal(body[key], true); faults[key] = true; } return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/restart-waker") { await restartWaker(); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/stop-waker") { await worker?.stop(); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/shutdown") { json(res, 200, await snapshot()); queueMicrotask(() => { void shutdown(); }); return; } + throw new Error("Unknown closed fixture operation"); + } + if (url.pathname.startsWith("/fixture/slack/")) return platform(req, res, url.pathname); + if (url.pathname.startsWith("/v1/connector/")) { + const loss = url.pathname === "/v1/connector/events/ack" && faults.loseAckReply; + if (url.pathname === "/v1/connector/events/ack") { + const original = await pool.query("SELECT delivery_id FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged'", [fixture.instanceId]); + if (original.rows.some(row => row.delivery_id.startsWith("slack_form_"))) { + assert.ok(tenantOrigin); + const proof = await fetch(`${tenantOrigin}/__fixture/proof-accepted`, { headers: { "x-fixture-control": fixture.controlNonce } }); + assert.equal(proof.status, 200); + const accepted = await proof.json(); assert.equal(accepted.deliveryId, original.rows.find(row => row.delivery_id.startsWith("slack_form_")).delivery_id); + assert.equal(accepted.proof.operationId, accepted.deliveryId.slice("slack_form_".length)); + assert.ok(accepted.proof.appReceiptId); + } + } + if (url.pathname === "/v1/connector/slack-forms" && faults.loseTerminalReply) { + const end = res.end.bind(res); + res.end = function(chunk, ...args) { + if (typeof chunk === "string" && chunk.includes('"slackForm"') && chunk.includes('"sealed"')) { + let requestKind; + try { requestKind = JSON.parse(JSON.parse(Buffer.concat(captured).toString()).operation).kind; } catch {} + if (requestKind === "terminal-ack") { faults.loseTerminalReply = false; lostTerminalReplies++; res.destroy(); return res; } + } + return end(chunk, ...args); + }; + const captured = []; let capturedBytes = 0; + const originalIterator = req[Symbol.asyncIterator].bind(req); + req[Symbol.asyncIterator] = async function* () { + for await (const chunk of originalIterator()) { + const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk); capturedBytes += buffer.byteLength; + if (capturedBytes <= SLACK_FORM_OPERATION_BYTES + 16_000) captured.push(Buffer.from(buffer)); + yield chunk; + } + }; + } + if (loss) { faults.loseAckReply = false; const end = res.end.bind(res); res.end = function(chunk, ...args) { if (res.statusCode === 200) { lostAckReplies++; res.destroy(); return res; } return end(chunk, ...args); }; } + const handled = await broker.handle(req, res, url); assert.equal(handled, true); return; + } + assert.equal(url.pathname, `/api/chat-webhooks/${fixture.publicId}/slack`); + const handled = await handleTenantSlackForm(req, res, { instanceId: fixture.instanceId, publicId: fixture.publicId, owner: broker.slackForms, ingress: broker.slackFormIngress }); + if (handled.handled) return; + if (!handled.consumedBody) { + const event = await handleTenantSlackEvent(req, res, { instanceId: fixture.instanceId, publicId: fixture.publicId, ingress: broker.slackEvents }); + if (event.handled) return; + handled.consumedBody = event.consumedBody; + } + assert.ok(tenantOrigin); + const upstream = await fetch(`${tenantOrigin}${url.pathname}`, { method: "POST", headers: { "content-type": req.headers["content-type"] ?? "", "x-slack-request-timestamp": req.headers["x-slack-request-timestamp"] ?? "", "x-slack-signature": req.headers["x-slack-signature"] ?? "" }, body: handled.consumedBody ?? await read(req), redirect: "error" }); + res.writeHead(upstream.status, { "content-type": upstream.headers.get("content-type") ?? "text/plain", connection: "close" }); + res.end(Buffer.from(await upstream.arrayBuffer())); +})().catch(error => { + const pathname = new URL(req.url ?? "/", fixture.origin ?? "http://127.0.0.1").pathname; + const knownPaths = ["auth.test", "users.info", "conversations.info", "conversations.list", "reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral", "chat.postMessage", "views.open", "views.update"].map(method => `/fixture/slack/${method}`); + const path = knownPaths.includes(pathname) ? pathname : pathname.startsWith("/v1/connector/") ? "/v1/connector/operation" : "/tenant-webhook"; + const category = error?.message === "fixture_method_contract" ? "fixture_method_contract" : error?.code === "ERR_ASSERTION" ? "fixture_assertion" : "fixture_boundary_error"; + const wire = wireRequests.get(req); + console.error(JSON.stringify({ event: "slack-form-fixture-boundary-failed", category, + method: req.method === "GET" || req.method === "POST" ? req.method : "OTHER", path, + providerMethod: wire?.method ?? "not-classified", providerMethodDigest: wire?.methodDigest ?? null, status: res.headersSent ? res.statusCode : 500, + responseAlreadyStarted: res.headersSent, ownedProviderId: wire?.ownedProviderId ?? null, + sourceFrames: wireSourceFrames(error) })); + failures.push(String(error?.stack ?? error)); if (!res.headersSent) json(res, 500, { error: "fixture_boundary_failed" }); else res.destroy(); +}); }); +async function shutdown() { + if (stopping) return; stopping = true; + const errors = []; + try { await worker?.stop(); } catch (error) { errors.push(error); } + try { if (server.listening) { server.close(); server.closeAllConnections(); await once(server, "close"); } } catch (error) { errors.push(error); } + try { await registry.close(); } catch (error) { errors.push(error); } + if (errors.length) { console.error(new AggregateError(errors, "Cloud journey cleanup failed")); process.exitCode = 1; } +} +try { + await registry.applyMigrations(path.join(cloudRoot, "migrations"), 30_000); await registry.hydrate(); + await new CloudStackProvisioner({ provider: new FakeCloudStackProvider(), registry }).provisionDevelopmentStack({ + idempotencyKey: fixture.instanceId, accountGroup: { id: `account-${fixture.instanceId}`, slug: fixture.instanceId, displayName: "Form journey" }, owner: { id: fixture.actorId, email: `${fixture.actorId}@example.test` }, + stack: { stackId: fixture.instanceId, accountGroupId: `account-${fixture.instanceId}`, stackSlug: fixture.instanceId, displayName: "Form journey", kind: "demo", primaryHost: "tenant.example", region: "us-east-1" }, + versionPlan: { appVersion: "fixture", schemaVersion: "fixture", rolloutTrack: "stable" }, sleepPolicy: { desiredState: "awake" }, primaryCompany: { paperclipCompanyId: fixture.companyId, displayName: "Form journey", defaultBudgetCents: 0 }, + }); + await store.putCloudStackInstance({ id: fixture.instanceId, kind: "cloud_stack", companyStackId: fixture.instanceId, environment: "test", ownerUserId: fixture.actorId, + signPublicKey: signing.publicKey.export({ format: "jwk" }).x, sealPublicKey: sealing.publicKey.export({ format: "jwk" }).x, + origins: ["https://tenant.example"], activeOrigin: "https://tenant.example", label: "Form journey", status: "active", createdAt: new Date(), updatedAt: new Date() }); + server.listen(0, "127.0.0.1"); await once(server, "listening"); + const address = server.address(); assert.ok(address && typeof address !== "string"); fixture.origin = `http://127.0.0.1:${address.port}`; + broker = new ConnectorBroker({ publicOrigin: fixture.origin, environment: "test", store, providers: providerRegistryFromEnvironment({}, "test"), secrets: { resolve: async () => undefined }, + formReceiptAuthority: signer, + authenticate: async () => ({ globalUserId: fixture.actorId }), slackEvents: { secrets, secretRefPrefix: "secret:///test/form-journey" }, eventWakeEnrollmentEnabled: true, eventWakeDeliveryEnabled: true, + wakeCloudStack: async () => { if (faults.dropImmediateHint) { faults.dropImmediateHint = false; droppedHints++; return; } await hint(); }, + fetchImplementation: async (input, init) => { const url = new URL(input instanceof Request ? input.url : String(input)); assert.equal(url.origin, "https://slack.com"); assert.equal(url.pathname, "/api/views.open"); return fetch(`${fixture.origin}/fixture/slack/views.open`, init); }, + }); + await restartWaker(); + await writeFile(stateFile, JSON.stringify(fixture), { mode: 0o600, flag: "wx" }); + console.log(JSON.stringify({ event: "slack-form-host-ready", pid: process.pid, node: process.version, origin: fixture.origin, poolMax: 2 })); + process.once("SIGTERM", () => { void shutdown(); }); + process.once("SIGINT", () => { void shutdown(); }); +} catch (error) { console.error(error); process.exitCode = 1; await shutdown(); } diff --git a/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs b/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs new file mode 100644 index 00000000000..6d29e48deff --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs @@ -0,0 +1,24 @@ +// Controlled ordinary run-token acceptance helper; normal run token only. +const peer = new URL(process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN ?? "invalid:"); +const base = new URL(process.env.PAPERCLIP_API_URL ?? "invalid:"); +if (peer.protocol !== "http:" || peer.hostname !== "127.0.0.1" || peer.origin !== base.origin || peer.pathname !== "/") throw Error("unexpected scoped agent API peer"); +if (process.env.SLACK_GATE_GUARD_PID !== String(process.pid)) throw Error("missing actual process guard"); +const runId=process.env.PAPERCLIP_RUN_ID, agentId=process.env.PAPERCLIP_AGENT_ID, companyId=process.env.PAPERCLIP_COMPANY_ID, token=process.env.PAPERCLIP_API_KEY; +if (!runId || !agentId || !companyId || !token) throw Error("missing normal run identity"); +const headers={authorization:`Bearer ${token}`,"x-paperclip-run-id":runId,"content-type":"application/json"}; +async function request(path, method="GET", body) { const response=await fetch(new URL(path,base),{method,headers,...(body===undefined?{}:{body:JSON.stringify(body)})});if(!response.ok)throw Error(`normal agent API refused ${method} ${response.status}`);return response.json(); } +const run=await request(`/api/heartbeat-runs/${encodeURIComponent(runId)}`); +if(run.id!==runId || run.companyId!==companyId || run.agentId!==agentId || typeof run.contextSnapshot?.issueId!=="string")throw Error("run scope changed"); +const issueId=run.contextSnapshot.issueId, route=`/api/issues/${encodeURIComponent(issueId)}`; +const issue=await request(route);if(issue.companyId!==companyId || issue.assigneeAgentId!==agentId || !["todo","in_progress"].includes(issue.status))throw Error("task authority changed"); +await request(`${route}/checkout`,"POST",{agentId,expectedStatuses:["todo","in_progress"]}); +const body=`Scoped Slack useful reply for run ${runId}`; +const comment=await request(`${route}/comments`,"POST",{body}); +if(comment.issueId!==issueId || comment.createdByRunId!==runId || comment.authorAgentId!==agentId || comment.body!==body)throw Error("useful result attribution changed"); +const completed=await request(route,"PATCH",{status:"done"}); +if(completed.id!==issueId || completed.companyId!==companyId || completed.status!=="done")throw Error("normal completion missing"); +console.log(JSON.stringify({event:"scoped-useful-turn-completed",runId,issueId,commentId:comment.id})); +// Actual Codex CLI wire format, parsed by the normal adapter. The text is +// exactly the already-authorized run comment; normal presentation can reuse it. +console.log(JSON.stringify({type:"item.completed",item:{id:`controlled-final-${runId}`,type:"agent_message",text:body}})); +console.log(JSON.stringify({type:"turn.completed",usage:{input_tokens:0,cached_input_tokens:0,output_tokens:0}})); diff --git a/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs b/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs new file mode 100644 index 00000000000..d8bd56d3363 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs @@ -0,0 +1,153 @@ +const key = Symbol.for('paperclip.slack.fixture.prewire'); +if (!process[key]) { + const net = require('node:net'); + const tls = require('node:tls'); + const http = require('node:http'); + const https = require('node:https'); + const fs = require('node:fs'); + const loopback = new Set(['127.0.0.1', 'localhost', '::1', '[::1]']); + const listeners = new Set(); + const pgTargets = new Set(); + const pgPorts = new Set([5432, 55448]); + let peer; + if (process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN) { + const value = new URL(process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN); + if (value.protocol !== 'http:' || value.hostname !== '127.0.0.1' || !value.port + || value.pathname !== '/' || value.search || value.hash || value.username || value.password + || pgPorts.has(Number(value.port))) throw new Error('Invalid exact Slack form fixture peer'); + peer = `127.0.0.1:${Number(value.port)}`; + } + for (const name of ['PAPERCLIP_SLACK_TEST_DATABASE_URL', 'SLACK_EVENT_TEST_DATABASE_URL']) { + if (!process.env[name]) continue; + let url; + try { url = new URL(process.env[name]); } catch { throw new Error('Invalid Slack fixture database target'); } + if (!['postgres:', 'postgresql:'].includes(url.protocol) || !loopback.has(url.hostname) + || !/^[a-zA-Z0-9_]+_test$/.test(decodeURIComponent(url.pathname.slice(1))) + || ['options', 'database', 'dbname'].some(k => url.searchParams.has(k))) throw new Error('Invalid Slack fixture database target'); + const port = Number(url.port || 5432); + pgPorts.add(port); + pgTargets.add(`${url.hostname}:${port}`); + } + const sockets = new Set(); + const observed = new WeakSet(); + const attempts = []; + let peakPgSockets = 0; + // Passive scalar counters only: no packet contents or authority changes. + let pgConnectAttempts = 0, pgConnects = 0, pgCloses = 0, pgWriteCalls = 0; + let outboundRequests = 0, nonPgConnectAttempts = 0; + let activeWindow = null; + const closedWaiters = new Set(); + const record = value => { + if (process.env.SLACK_GATE_PROCESS_LOG) fs.appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify(value) + '\n'); + }; + const fail = (host, port) => { + attempts.push({ host: String(host), port: Number(port) || null }); + process.exitCode = 98; + throw new Error('Slack fixture blocked an unowned network target before transport'); + }; + const check = (host = 'localhost', port, protocol) => { + host = String(host).toLowerCase(); + if (!loopback.has(host)) fail(host, port); + if (protocol && !['http:', 'https:'].includes(protocol)) fail(host, port); + if (port && pgPorts.has(Number(port)) && !pgTargets.has(`${host}:${Number(port)}`)) fail(host, port); + if (port && !pgTargets.has(`${host}:${Number(port)}`) && !listeners.has(Number(port)) && `${host}:${Number(port)}` !== peer) fail(host, port); + }; + const checkArgs = args => { + const first = args[0]; + if (first instanceof URL || typeof first === 'string' && /^https?:/.test(first)) { + const url = first instanceof URL ? first : new URL(first); + check(url.hostname, url.port || (url.protocol === 'https:' ? 443 : 80), url.protocol); + } else if (first && typeof first === 'object') { + if (first.path && !first.port && !first.host && !first.hostname) fail('unix-socket', 0); + check(first.hostname || first.host, first.port, first.protocol); + } else if (typeof first === 'number') check(typeof args[1] === 'string' ? args[1] : undefined, first); + else fail('unknown-target', 0); + }; + const listen = net.Server.prototype.listen; + net.Server.prototype.listen = function (...args) { + this.once('listening', () => { + const address = this.address(); + if (address && typeof address === 'object' && loopback.has(address.address)) { + listeners.add(address.port); + this.once('close', () => listeners.delete(address.port)); + } + }); + return Reflect.apply(listen, this, args); + }; + for (const module of [http, https]) for (const name of ['request', 'get']) { + const original = module[name]; + module[name] = function (...args) { checkArgs(args); outboundRequests += 1; return Reflect.apply(original, this, args); }; + } + const connect = net.Socket.prototype.connect; + net.Socket.prototype.connect = function (...args) { + const normalized = Array.isArray(args[0]) ? args[0] : args; + checkArgs(normalized); + const first = normalized[0]; + const port = typeof first === 'object' ? first.port : first; + if (pgPorts.has(Number(port))) pgConnectAttempts += 1; + if (pgPorts.has(Number(port)) && !observed.has(this)) { + observed.add(this); + this.once('connect', () => { + sockets.add(this); pgConnects += 1; + peakPgSockets = Math.max(peakPgSockets, sockets.size); + if (activeWindow) activeWindow.peakPgSockets = Math.max(activeWindow.peakPgSockets, sockets.size); + }); + this.once('close', () => { + if (sockets.delete(this)) pgCloses += 1; + if (!sockets.size) for (const settled of [...closedWaiters]) settled(); + }); + } + if (!pgPorts.has(Number(port))) nonPgConnectAttempts += 1; + return Reflect.apply(connect, this, args); + }; + const write = net.Socket.prototype.write; + net.Socket.prototype.write = function (...args) { + if (observed.has(this)) pgWriteCalls += 1; + return Reflect.apply(write, this, args); + }; + const tlsConnect = tls.connect; + tls.connect = function (...args) { + const first = args[0]; + const host = first && typeof first === 'object' ? first.hostname || first.host || 'localhost' : typeof args[1] === 'string' ? args[1] : 'localhost'; + const port = first && typeof first === 'object' ? first.port : first; + if (`${host}:${Number(port)}` === peer) fail(host, port); + checkArgs(args); return Reflect.apply(tlsConnect, this, args); + }; + const fetch = globalThis.fetch; + if (fetch) globalThis.fetch = function (input, init) { + checkArgs([input instanceof Request ? input.url : input]); + outboundRequests += 1; + return fetch(input, init); + }; + const wireSnapshot = () => ({ pid: process.pid, observedAt: Date.now(), pgConnectAttempts, pgConnects, pgCloses, + pgWriteCalls, outboundRequests, nonPgConnectAttempts, livePgSockets: sockets.size }); + const waitForPgSocketsClosed = timeoutMs => new Promise((resolve, reject) => { + if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) { reject(new Error('Original settlement budget exhausted')); return; } + if (!sockets.size) { resolve(); return; } + let timer; + const settled = () => { if (sockets.size) return; clearTimeout(timer); closedWaiters.delete(settled); resolve(); }; + closedWaiters.add(settled); + timer = setTimeout(() => { closedWaiters.delete(settled); reject(new Error('Used PG sockets did not close within original settlement budget')); }, timeoutMs); + }); + const beginWireWindow = () => { + if (activeWindow || sockets.size) throw new Error('Quiet wire window requires actual PG closure'); + activeWindow = { before: wireSnapshot(), peakPgSockets: sockets.size }; + return activeWindow.before; + }; + const endWireWindow = () => { + if (!activeWindow) throw new Error('Missing quiet wire window'); + const after = wireSnapshot(), before = activeWindow.before; + const result = { before, after, peakPgSockets: activeWindow.peakPgSockets, + pgWriteCalls: after.pgWriteCalls - before.pgWriteCalls, pgConnectAttempts: after.pgConnectAttempts - before.pgConnectAttempts, + pgConnects: after.pgConnects - before.pgConnects, pgCloses: after.pgCloses - before.pgCloses, + outboundRequests: after.outboundRequests - before.outboundRequests, nonPgConnectAttempts: after.nonPgConnectAttempts - before.nonPgConnectAttempts }; + activeWindow = null; + return result; + }; + const stats = () => ({ pid: process.pid, peakPgSockets, remainingPgSockets: sockets.size, unexpectedAttempts: attempts.slice(), wire: wireSnapshot() }); + process[key] = Object.freeze({ pid: process.pid, stats, wireSnapshot, waitForPgSocketsClosed, beginWireWindow, endWireWindow }); + globalThis[key] = process[key]; + process.env.SLACK_GATE_GUARD_PID = String(process.pid); + record({ phase: 'repository-preload', pid: process.pid }); + process.on('exit', () => record({ phase: 'exit', ...stats(), exitCode: process.exitCode || 0 })); +} diff --git a/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs b/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs new file mode 100644 index 00000000000..b58345a11fd --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs @@ -0,0 +1,8 @@ +import assert from "node:assert/strict"; +import { pathToFileURL } from "node:url"; +assert.equal(process.versions.node.split(".")[0], "24"); +assert.equal(process.argv.length, 3); +const { SLACK_CHAT_BOT_SCOPES } = await import(pathToFileURL(process.argv[2])); +assert.ok(Array.isArray(SLACK_CHAT_BOT_SCOPES) && SLACK_CHAT_BOT_SCOPES.length > 0); +assert.ok(SLACK_CHAT_BOT_SCOPES.every(value => typeof value === "string" && (value === "commands" || /^[a-z_]+:[a-z_]+$/.test(value)))); +console.log(JSON.stringify({ scopes: SLACK_CHAT_BOT_SCOPES, pid: process.pid, node: process.version })); diff --git a/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs new file mode 100644 index 00000000000..3fdb65b17c5 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs @@ -0,0 +1,44 @@ +import { spawn } from "node:child_process"; +import { createRequire } from "node:module"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +if (!process.version.startsWith("v24.")) throw new Error("Slack form journey requires the current Node24 executable"); +if (process.argv.length !== 2) throw new Error("Slack form journey has one closed complete case and accepts no selectors"); +const value = process.env.PAPERCLIP_SLACK_TEST_DATABASE_URL; +const cloudValue = process.env.SLACK_EVENT_TEST_DATABASE_URL; +const fixture = new URL(value ?? "invalid:"), cloud = new URL(cloudValue ?? "invalid:"), admin = new URL(process.env.PAPERCLIP_TEST_POSTGRES_ADMIN_URL ?? "invalid:"); +for (const url of [fixture, cloud, admin]) if (!["postgres:", "postgresql:"].includes(url.protocol) || url.hostname !== "127.0.0.1" || !url.port || url.search || url.hash) throw new Error("Explicit owned PG17 fixture/admin URLs required"); +if (fixture.protocol !== cloud.protocol || fixture.protocol !== admin.protocol || admin.pathname !== "/postgres" + || !/^\/sleep2_slack_form_[a-z0-9_]+_app_test$/.test(fixture.pathname) || cloud.pathname !== fixture.pathname.replace(/_app_test$/, "_cloud_test")) throw new Error("Distinct same-grant form fixture pair required"); +const stateFile = process.env.SLACK_FORM_JOURNEY_STATE_FILE; +if (!stateFile || !path.isAbsolute(stateFile)) throw new Error("Owned private host state file required"); +const state = JSON.parse(readFileSync(stateFile, "utf8")); +const peer = new URL(state.origin), tenant = new URL(state.tenantOrigin); +for (const url of [peer, tenant]) if (url.protocol !== "http:" || url.hostname !== "127.0.0.1" || !url.port || ["55448", "5432"].includes(url.port) || url.pathname !== "/" || url.search || url.hash || url.username || url.password) throw new Error("Closed loopback HTTP peer origin required"); +if (peer.origin === tenant.origin) throw new Error("Cloud and tenant must have distinct owned listeners"); +const log = process.env.SLACK_GATE_PROCESS_LOG; +if (!log || !path.isAbsolute(log) || existsSync(log)) throw new Error("Fresh absolute guard log required"); +const packet = path.dirname(fileURLToPath(import.meta.url)); +const server = path.resolve(packet, "../../../.."); +const mode = process.env.SLACK_FORM_SHORT_GATE; +if (mode !== "callback") throw new Error("One existing closed HTTP mode required"); +const entry = path.join(server, "src/__tests__/slack-enabled-quiet.postgres.test.ts"); +const require = createRequire(path.join(server,"package.json")); +const vitest = path.join(path.dirname(require.resolve("vitest/package.json")), "vitest.mjs"); +const guard = path.join(packet, "quiet-prewire.cjs"); +const home = mkdtempSync(path.join(tmpdir(), "slack-form-journey-")); +const env = { PATH: process.env.PATH, LANG: process.env.LANG, HOME: home, TMPDIR: home, PAPERCLIP_HOME: home, + NODE_ENV: "test", NODE_OPTIONS: "", SLACK_GATE_EXPECT_PRELOAD: "1", PAPERCLIP_SLACK_TEST_DATABASE_URL: value, + SLACK_EVENT_TEST_DATABASE_URL: cloudValue, PAPERCLIP_TEST_POSTGRES_ADMIN_URL: process.env.PAPERCLIP_TEST_POSTGRES_ADMIN_URL, + SLACK_GATE_PROCESS_LOG: log, SLACK_FORM_JOURNEY_STATE_FILE: stateFile, SLACK_FORM_FIXTURE_PEER_ORIGIN: peer.origin, SLACK_FORM_SHORT_GATE: process.env.SLACK_FORM_SHORT_GATE, SLACK_PUBLICATION_REPORT: process.env.SLACK_PUBLICATION_REPORT }; +if (env.SLACK_FORM_SHORT_GATE !== "callback") throw new Error("One closed short gate required"); +const config = path.join(home,"short-gate.config.ts"); +writeFileSync(config, `import base from ${JSON.stringify(path.join(server,"vitest.config.ts"))};\nexport default {...base,root:${JSON.stringify(server)},test:{...base.test,include:[${JSON.stringify(entry)}],exclude:[],maxWorkers:1,fileParallelism:false}};\n`); +const child = spawn(process.execPath, ["--require", guard, vitest, "run", "--config", config, + "--maxWorkers=1", "--no-file-parallelism", "--reporter=default", "--reporter=json", "--outputFile=" + process.env.SLACK_PUBLICATION_REPORT, `--execArgv=--require=${guard}`], { cwd: server, env, stdio: "inherit" }); +for (const signal of ["SIGINT", "SIGTERM"]) process.on(signal, () => child.kill(signal)); +child.on("error", error => { console.error(error); process.exitCode = 1; }); +child.on("close", (code, signal) => { process.exitCode = signal ? 1 : code ?? 1; if (!process.exitCode) rmSync(home, { recursive: true, force: true }); }); diff --git a/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py new file mode 100644 index 00000000000..6447f4477c9 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py @@ -0,0 +1,216 @@ +import pathlib, json, hashlib, subprocess, os, sys, time, signal, shutil, tempfile, secrets, socket, re, urllib.request, urllib.parse, traceback +O=pathlib.Path(__file__).resolve().parent +A=O.parents[4] +D=pathlib.Path(os.environ['PAPERCLIP_CLOUD_REPO_ROOT']).resolve() +assert D.is_dir() and (D/'package.json').is_file() +MODE='callback' +assert len(sys.argv)==2, 'One exact whole qualification; pass a fresh output directory' +OUT=pathlib.Path(sys.argv[1]).resolve() +assert not OUT.exists(), 'Fresh output directory required' +OUT.mkdir(parents=True) +sha=lambda p:hashlib.sha256(pathlib.Path(p).read_bytes()).hexdigest() +F={'node24':str(pathlib.Path(os.environ['PAPERCLIP_SLACK_NODE24']).resolve()),'node22':str(pathlib.Path(os.environ['PAPERCLIP_SLACK_NODE22']).resolve())} +for key in ['node24','node22']:assert pathlib.Path(F[key]).is_file();F[key+'SHA']=sha(F[key]) +# Fail closed before git metadata, Node children, or any database owner. +app_url=os.environ['PAPERCLIP_SLACK_TEST_DATABASE_URL'];cloud_url=os.environ['SLACK_EVENT_TEST_DATABASE_URL'];admin_url=os.environ['PAPERCLIP_TEST_POSTGRES_ADMIN_URL'] +ap=urllib.parse.urlparse(app_url);cp=urllib.parse.urlparse(cloud_url);ad=urllib.parse.urlparse(admin_url) +assert ap.port is not None and 1 <= ap.port <= 65535, 'Explicit valid fixture port required before child entry' +for u in [ap,cp,ad]:assert u.scheme in ('postgres','postgresql') and u.hostname=='127.0.0.1' and u.port==ap.port and not u.query and not u.fragment +assert ap.scheme==cp.scheme==ad.scheme and ad.path=='/postgres' +assert re.fullmatch('/sleep2_slack_form_[a-z0-9_]+_app_test',ap.path) and cp.path==ap.path.replace('_app_test','_cloud_test') +database_names=[ap.path[1:],cp.path[1:]] +# Snapshot only current local sources/helpers, no committed run inventory. +inputFiles=[p for p in O.iterdir()if p.is_file()] +inputFiles.append(A/'server/src/__tests__/slack-enabled-quiet.postgres.test.ts') +sourcePins={str(p):sha(p)for p in inputFiles} +sourceHeads={} +for root in [A,D]: + sourceHeads[str(root)]=subprocess.check_output(['git','rev-parse','HEAD'],cwd=root,text=True).strip() + names=subprocess.check_output(['git','ls-files','-z'],cwd=root).decode().split('\0') + for name in names: + p=root/name + if name and p.is_file():sourcePins[str(p)]=sha(p) +# Exact installed workspace/tool setup is supplied by normal producer commands; +# no foreign outputs are copied or generated by this runtime-only helper. + +def authority(): + return {'scope':'Explicit local *_test fixture URLs/selected own checkouts; no runtime control-plane privileges','sourceHeads':sourceHeads} +def pins(): + for p,h in sourcePins.items():assert sha(p)==h, 'Qualification source changed' + for key in ['node24','node22']:assert sha(F[key])==F[key+'SHA'] +children=[]; outcomes=[]; cleanup_errors=[]; pid_samples=[] +PRIVATE=None +helper_pids=set(); helper_groups=set(); closed_groups=set(); metadata_probes=[] + +def write(name,value): + with (OUT/name).open('x') as target:json.dump(value,target,indent=2);target.write('\n') +def events(p): + if not pathlib.Path(p).exists():return [] + return [json.loads(line) for line in pathlib.Path(p).read_text().splitlines() if line.strip()] +def closed_env(label,peer=None,databases=False): + home=PRIVATE/label;home.mkdir();(home/'tmp').mkdir();(home/'cache').mkdir() + env={'PATH':str(pathlib.Path(F['node24']).parent)+':/usr/bin:/bin:/usr/sbin:/sbin','LANG':'en_US.UTF-8','HOME':str(home),'TMPDIR':str(home/'tmp'),'XDG_CACHE_HOME':str(home/'cache'),'PAPERCLIP_HOME':str(home),'NODE_OPTIONS':'','NODE_ENV':'test','CI':'true','FORCE_COLOR':'0','SLACK_GATE_PROCESS_LOG':str(OUT/(label+'-guard.jsonl'))} + if peer:env['SLACK_FORM_FIXTURE_PEER_ORIGIN']=peer + if databases:env.update({'PAPERCLIP_SLACK_TEST_DATABASE_URL':app_url,'SLACK_EVENT_TEST_DATABASE_URL':cloud_url,'PAPERCLIP_TEST_POSTGRES_ADMIN_URL':admin_url}) + return env +def spawn(label,argv,env): + auth=authority() + for node in [F['node24'],F['node22']]:assert sha(node)==F['node24SHA' if node==F['node24'] else 'node22SHA'] + log=(OUT/(label+'.log')).open('xb') + process=subprocess.Popen(argv,cwd=A/'server',env=env,stdout=log,stderr=subprocess.STDOUT,start_new_session=True) + row={'label':label,'pid':process.pid,'pgid':process.pid,'argv':argv,'envNames':sorted(env),'authority':auth,'started':time.time()} + write(label+'-started.json',row);children.append((process,log,row));print(json.dumps({'started':label,'pid':process.pid,'pgid':process.pid}),flush=True) + return process + +def group_exists(pgid): + try:os.killpg(pgid,0);return True + except ProcessLookupError:return False + +def sample_owned_groups(): + groups={row['pgid'] for _,_,row in children if row['pgid'] not in closed_groups}|(helper_groups-closed_groups) + p=subprocess.Popen(['/bin/ps','-axo','pid=,ppid=,pgid='],text=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True) + probe={'pid':p.pid,'pgid':p.pid,'started':time.time()};metadata_probes.append((p,probe)) + stdout,stderr=p.communicate(timeout=5);probe.update(exit=p.returncode,closed=time.time()) + assert p.returncode==0,stderr + assert not group_exists(p.pid),'Metadata probe group remains' + closed_groups.add(p.pid) + rows=[] + for line in stdout.splitlines(): + fields=line.split() + if len(fields)==3 and all(v.isdecimal() for v in fields) and int(fields[2]) in groups:rows.append(dict(zip(['pid','ppid','pgid'],map(int,fields)))) + pid_samples.append({'at':time.time(),'ownedGroupsOnly':rows}) + +def wait(process,label,timeout,expected=0): + start=time.monotonic();seen=set() + while process.poll() is None: + if time.monotonic()-start>=timeout:raise TimeoutError(f'{label} exceeded outer process bound {timeout}s') + if MODE in ('header','callback') and label=='app': + markers=events(OUT/'app-guard.jsonl');helpers={v['pid'] for v in markers if v.get('phase') in ('scoped-useful-process-preload','scoped-useful-version-preload')} + helper_pids.update(helpers);helper_groups.update(helpers) # Normal process adapter runChild(detached:true) creates pgid=pid; sampled metadata below corroborates when live. + current={v['pid'] for v in markers if v.get('phase')=='repository-preload'} + if current-seen:sample_owned_groups();seen|=current + time.sleep(.05) + code=process.wait();outcomes.append({'label':label,'pid':process.pid,'pgid':process.pid,'exit':code,'expectedExit':expected,'seconds':time.monotonic()-start});print(json.dumps(outcomes[-1]),flush=True) + assert code==expected,f'{label}: expected exit {expected}, actual {code}' + return code + +def ready(process,file,timeout=20): + start=time.monotonic() + while not file.exists(): + assert process.poll() is None,'Owner exited before ready; preserve setup log' + if time.monotonic()-start>timeout:raise TimeoutError('Owner ready-file bound exceeded') + time.sleep(.05) + return json.loads(file.read_text()) + + +def stop(process,label,normal_signal=True): + if process.poll() is None: + if normal_signal:process.send_signal(signal.SIGTERM) + process.wait(timeout=20) + assert process.returncode==0,f'{label} normal owner shutdown failed' + row=next(row for child,_,row in children if child is process);outcomes.append({'label':label,'pid':process.pid,'pgid':process.pid,'exit':process.returncode,'expectedExit':0,'seconds':time.time()-row['started'],'normalOwnerStop':True}) + + +def http_control(origin,nonce,path): + assert path=='shutdown' + url=urllib.parse.urlparse(origin);assert url.scheme=='http' and url.hostname=='127.0.0.1' + request=urllib.request.Request(origin+'/__fixture/'+path,data=b'{}',method='POST',headers={'x-fixture-control':nonce,'content-type':'application/json'}) + class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self,*args,**kwargs):return None + opener=urllib.request.build_opener(urllib.request.ProxyHandler({}),NoRedirect()) + with opener.open(request,timeout=15) as response: + body=response.read(2_100_000);assert response.status==200 + return {'status':response.status,'bodySHA256':hashlib.sha256(body).hexdigest(),'bodyBytes':len(body)} + +def journey(): + guard=str(O/'quiet-prewire.cjs') + source=spawn('owned-scopes',[F['node24'],'--require',guard,str(O/'read-owned-scopes.mjs'),str(A/'packages/shared/dist/index.js')],closed_env('scopes')) + wait(source,'owned-scopes',20) + scopes=json.loads((OUT/'owned-scopes.log').read_text().splitlines()[-1])['scopes'];assert scopes and all(s=='commands' or re.fullmatch('[a-z_]+:[a-z_]+',s) for s in scopes) + sock=socket.socket();sock.bind(('127.0.0.1',0));port=sock.getsockname()[1];sock.close();assert port not in (5432,55448) + tenant='http://127.0.0.1:'+str(port) + env=closed_env('cloud',tenant,True);env.update({'PAPERCLIP_CLOUD_REPO_ROOT':str(D),'SLACK_FORM_FIXTURE_SCOPES':','.join(scopes),'SLACK_GATE_PROCESS_LOG':str(OUT/'cloud-guard.jsonl'),'SLACK_PEER_CONTROL_LOG':str(OUT/'cloud-transport.jsonl')}) + host=spawn('cloud',[F['node22'],'--require',str(O/'transport-observer.cjs'),'--require',guard,str(O/'enabled-quiet-host.mjs'),str(PRIVATE/'state.json')],env) + state=ready(host,PRIVATE/'state.json',90) + origin=urllib.parse.urlparse(state['origin']);assert origin.scheme=='http' and origin.hostname=='127.0.0.1' and origin.port not in (port,5432,55448) + assert state['tenantOrigin']==tenant and (PRIVATE/'state.json').stat().st_mode & 0o777==0o600 + write('host-ready.json',{'pid':host.pid,'origin':state['origin'],'tenantOrigin':tenant,'stateSHA256':sha(PRIVATE/'state.json'),'stateMode':'0600','syntheticSecretStateNotArchived':True,'scopes':scopes}) + app_env=closed_env('app',state['origin'],True);app_env.update({'SLACK_FORM_JOURNEY_STATE_FILE':str(PRIVATE/'state.json'),'SLACK_GATE_PROCESS_LOG':str(OUT/'app-guard.jsonl'),'SLACK_FORM_SHORT_GATE':MODE,'SLACK_PUBLICATION_REPORT':str(OUT/'app.report.json')}) + app=spawn('app',[F['node24'],str(O/'run-enabled-quiet-app.mjs')],app_env) + try: + wait(app,'app',600) + report=json.loads((OUT/'app.report.json').read_text());assert report['numTotalTests']==1 and report['numPassedTests']==1 and report['numFailedTests']==0 and report['numPendingTests']==0 and report['numTodoTests']==0,report + assertions=[x for suite in report['testResults']for x in suite['assertionResults']];assert len(assertions)==1 and assertions[0]['status']=='passed',assertions + write('whole-enabled-quiet-GREEN.json',{'total':1,'pass':1,'fail':0,'scope':'Two actual composed enabled-quiet windows around ordinary useful tasks; no physical suspension or whole-area claim'}) + + finally: + if app.poll() is None:raise RuntimeError('App still active; outer cleanup must stop its exact owned group before observer') + write('cloud-shutdown-response.json',http_control(state['origin'],state['controlNonce'],'shutdown')) + stop(host,'cloud',False) + assert not group_exists(app.pid) and not group_exists(host.pid),'All actual owner groups must close before final observer' + app_marks=events(OUT/'app-guard.jsonl') + helpers=[v for v in app_marks if v.get('phase')=='scoped-useful-process-preload'] + assert len(helpers)==2 and len({v['pid'] for v in helpers})==2,'Exactly two actual useful-process guard entries required' + version_helpers=[v for v in app_marks if v.get('phase')=='scoped-useful-version-preload'] + for helper in helpers+version_helpers: + pid=helper['pid'];helper_pids.add(pid);helper_groups.add(pid) + exits=[v for v in app_marks if v.get('phase')=='exit' and v.get('pid')==pid] + assert len(exits)==1 and exits[0]['exitCode']==0 and exits[0]['remainingPgSockets']==0 and exits[0]['unexpectedAttempts']==[],'Actual detached helper EXIT required' + assert not group_exists(pid),'Detached useful-process group remains after normal owner drain' + closed_groups.add(pid) + write('useful-process-closure.json',{'helpers':helpers,'versionProbes':version_helpers,'actualExits':[v for v in app_marks if v.get('phase')=='exit' and v.get('pid') in helper_pids],'expectedDetachedGroups':sorted(helper_groups),'sampledMetadata':pid_samples,'allExpectedGroupsAbsent':True,'scopeLimit':'Normal adapter detached-session pgid=pid plus actual guard entry/EXIT and group-absence checks; short lived descendants may escape OS sampling, not a full OS census.'}) + guard_receipts={} + for name in ['app','cloud']: + marks=events(OUT/(name+'-guard.jsonl'));entries=[v for v in marks if v.get('phase')=='repository-preload'];assert entries,f'{name}: actual before-import marker required' + assert all(not v.get('unexpectedAttempts') for v in marks) + exits=[v for v in marks if v.get('phase')=='exit'];assert all(v.get('remainingPgSockets')==0 for v in exits) + assert any(v['pid']==entries[0]['pid'] for v in exits),f'{name}: actual main guard exit required' + missing=sorted({v['pid'] for v in entries}-{v['pid'] for v in exits}) + if name!='app':assert not missing + guard_receipts[name]={'entries':entries,'exits':exits,'missingWorkerExits':missing} + actual_worker_stats=[] + for line in (OUT/'app.log').read_text().splitlines(): + if '"networkStats"' not in line:continue + try:value=json.loads(line[line.index('{'):]) + except (ValueError,json.JSONDecodeError):continue + if 'networkStats' in value:actual_worker_stats.append(value['networkStats']) + known={v['pid'] for v in guard_receipts['app']['entries']} + assert actual_worker_stats and all(v['pid'] in known and v['remainingPgSockets']==0 and v['unexpectedAttempts']==[] for v in actual_worker_stats) + write('actual-guard-receipts.json',{'owners':guard_receipts,'actualWorkerPostCleanupStats':actual_worker_stats}) + write('journey-result.json',{'status':'one enabled quiet/reactivation control GREEN; no managed JWKS rotation, physical suspension or whole-area claim','outcomes':outcomes,'caseBodyCapSeconds':360,'outerAppSeconds':600,'configuredAppPotential':10,'configuredCloudPool':2,'duringCaseObserver':0,'finalObserver':0,'externalFinalObserverOnlyAfterControllerClosed':True,'actualGuardLogs':['app-guard.jsonl','cloud-guard.jsonl','observer-guard.jsonl'],'missingWorkerExitLimit':'An actual worker preload marker is mandatory. A missing worker exit marker is reported, never replaced with a parent marker. Per-process guard peaks do not establish an exact concurrent aggregate SQL peak.','physicalAndRestartLimit':'Fixture controls wake/routing and restarts delivery owners, not physical platform suspension or whole service processes.'}) + +primary=None +try: + auth=authority();pins() + PRIVATE=pathlib.Path(tempfile.mkdtemp(prefix='paperclip-slack-quiet-'));os.chmod(PRIVATE,0o700) + write('controller-started.json',{'pid':os.getpid(),'pgid':os.getpgrp(),'mode':MODE,'authority':auth,'privateRoot':str(PRIVATE),'sourcePinsSHA':hashlib.sha256(json.dumps(sourcePins,sort_keys=True).encode()).hexdigest(),'started':time.time()}) + journey() +except BaseException as error: + primary={'type':type(error).__name__,'message':str(error),'traceback':traceback.format_exc()};write('primary-failure.json',primary) +finally: + for process,log,row in reversed(children): + try: + if process.poll() is None: + os.killpg(process.pid,signal.SIGTERM) + try:process.wait(timeout=20) + except subprocess.TimeoutExpired:raise RuntimeError('Owned group remains after SIGTERM; retain live cohort and forensics, no hard kill') + log.close() + if group_exists(process.pid):raise RuntimeError('Owned group remains after parent exit; retain for caller identity audit, no historical-group signal') + closed_groups.add(process.pid) + except BaseException as error:cleanup_errors.append({'label':row['label'],'type':type(error).__name__,'message':str(error)}) + for probe,row in metadata_probes: + if probe.poll() is None or group_exists(row['pgid']):cleanup_errors.append({'metadataProbe':row,'error':'Metadata probe remains; retain for caller audit'}) + for pid in sorted(helper_groups): + if group_exists(pid):cleanup_errors.append({'helperPID':pid,'pgid':pid,'error':'Detached helper group remains; retain for identity audit, no blind signal'}) + if PRIVATE: + if any(group_exists(process.pid) for process,_,row in children) or any(group_exists(pid) for pid in helper_groups) or any(probe.poll() is None or group_exists(row['pgid']) for probe,row in metadata_probes): + cleanup_errors.append({'privateRoot':str(PRIVATE),'error':'Retained owned root while an owned process group remains'}) + elif primary or cleanup_errors: + cleanup_errors.append({'privateRoot':str(PRIVATE),'forensicsRetained':True}) + else: + try:shutil.rmtree(PRIVATE) + except BaseException as error:cleanup_errors.append({'privateRoot':str(PRIVATE),'error':str(error)}) + try:pins() + except BaseException as error:cleanup_errors.append({'sourcePinError':str(error)}) + write('controller-cleanup.json',{'controller':os.getpid(),'children':[row for _,_,row in children],'outcomes':outcomes,'pidSamples':pid_samples,'metadataProbes':[row for _,row in metadata_probes],'detachedHelperPIDs':sorted(helper_pids),'detachedHelperGroups':sorted(helper_groups),'closedGroups':sorted(closed_groups),'remainingGroups':[row['pgid'] for process,_,row in children if group_exists(process.pid)],'privateRoot':str(PRIVATE),'privateRemoved':not PRIVATE or not PRIVATE.exists(),'primary':primary,'secondaryCleanup':cleanup_errors,'databaseDisposition':'No DROP, no broad cleanup. Caller retains/audits exact explicit fixtures.'}) +if primary or cleanup_errors:sys.exit(1) diff --git a/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs b/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs new file mode 100644 index 00000000000..31443499919 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs @@ -0,0 +1,17 @@ +// Controlled exact useful-process fixture: host instrumentation, current run token only. +const permitted = new Set([ + "PATH", "HOME", "TMPDIR", "XDG_CACHE_HOME", "LANG", "LC_ALL", "CI", "NODE_ENV", "NODE_OPTIONS", + "PAPERCLIP_API_URL", "PAPERCLIP_RUN_ID", "PAPERCLIP_AGENT_ID", "PAPERCLIP_COMPANY_ID", "PAPERCLIP_API_KEY", + "SLACK_FORM_FIXTURE_PEER_ORIGIN", "SLACK_GATE_PROCESS_LOG", +]); +// The approved controller passes empty NODE_OPTIONS and the exact preload as +// explicit argv. Refuse an earlier unreviewed preload instead of keeping its +// possibly already-installed DB/origin authority after environment reduction. +if (process.env.NODE_OPTIONS) throw Error("Unexpected useful-process preload before authority reduction"); +for (const key of Object.keys(process.env)) if (!permitted.has(key)) delete process.env[key]; +require("./quiet-prewire.cjs"); +if (process.env.SLACK_GATE_GUARD_PID !== String(process.pid)) throw Error("Missing actual useful-process guard"); +require("node:fs").appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify({ + phase: process.argv.length === 3 && process.argv[2] === "--version" ? "scoped-useful-version-preload" : "scoped-useful-process-preload", pid: process.pid, ppid: process.ppid, + argv: process.argv, execArgv: process.execArgv, envNames: Object.keys(process.env).sort(), +}) + "\n"); diff --git a/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs b/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs new file mode 100644 index 00000000000..35240ded21b --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs @@ -0,0 +1,27 @@ +const net = require('node:net'); +const tls = require('node:tls'); +const http = require('node:http'); +const dns = require('node:dns'); +const fs = require('node:fs'); +const key = Symbol.for('paperclip.slack.peer.control.transport'); +if (process[key]) throw new Error('Duplicate passive peer observer'); +let connects = 0, tlsCalls = 0, httpCalls = 0, dnsCalls = 0; +const sockets = new Set(); +const connect = net.Socket.prototype.connect; +net.Socket.prototype.connect = function (...args) { + connects++; + this.once('connect', () => sockets.add(this)); + this.once('close', () => sockets.delete(this)); + return Reflect.apply(connect, this, args); +}; +const secure = tls.connect; +tls.connect = function (...args) { tlsCalls++; return Reflect.apply(secure, this, args); }; +const request = http.request; +http.request = function (...args) { httpCalls++; return Reflect.apply(request, this, args); }; +const lookup = dns.lookup; +dns.lookup = function (...args) { dnsCalls++; return Reflect.apply(lookup, this, args); }; +const stats = () => ({ pid: process.pid, connects, tlsCalls, httpCalls, dnsCalls, sockets: sockets.size }); +process[key] = Object.freeze({ pid: process.pid, stats }); +const record = value => fs.appendFileSync(process.env.SLACK_PEER_CONTROL_LOG, JSON.stringify(value) + '\n'); +record({ phase: 'passive-observer-before-guard', pid: process.pid, node: process.version, argv: process.argv, execArgv: process.execArgv }); +process.on('exit', () => record({ phase: 'passive-observer-exit', ...stats(), exitCode: process.exitCode || 0 })); diff --git a/server/src/__tests__/idle-admission.test.ts b/server/src/__tests__/idle-admission.test.ts index 6ab60153332..55412da846b 100644 --- a/server/src/__tests__/idle-admission.test.ts +++ b/server/src/__tests__/idle-admission.test.ts @@ -34,6 +34,19 @@ describe("idle admission", () => { expect(idleWorkSnapshot().active).toBe(0); }); + it("exempts only the exact body status read from self-accounting and keeps drain mutations tracked", async () => { + const server = app(); + server.post("/api/instance/task-drain/status", (_req, res) => res.json({ active: idleWorkSnapshot().active })); + server.post("/api/instance/task-drain", (_req, res) => res.json({ active: idleWorkSnapshot().active })); + trackIdleRequestHandlers(server); + startTaskDrain({ purpose: "idle", ttlMs: 60_000 }); + expect((await request(server).post("/api/instance/task-drain/status").expect(200)).body.active).toBe(0); + expect((await request(server).post("/api/instance/task-drain").expect(200)).body.active).toBe(1); + await request(server).post("/api/instance/task-drain/status/mutate").expect(503); + await request(server).get("/api/instance/task-drain/status").expect(503); + expect(idleWorkSnapshot().active).toBe(0); + }); + it("counts a nested async handler after it sends a response", async () => { const server = app(); const router = express.Router(); diff --git a/server/src/__tests__/instance-settings-routes.test.ts b/server/src/__tests__/instance-settings-routes.test.ts index cf4803eda84..56b4b211ee1 100644 --- a/server/src/__tests__/instance-settings-routes.test.ts +++ b/server/src/__tests__/instance-settings-routes.test.ts @@ -1,4 +1,5 @@ -import { generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { DEFAULT_JSON_BODY_LIMIT } from "../http/body-limits.js"; +import { createHash, generateKeyPairSync, randomUUID, sign } from "node:crypto"; import express from "express"; import request from "supertest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; @@ -100,7 +101,7 @@ describe("instance settings routes", () => { function createApp(actor: any, pluginWorkers?: PluginWorkerManager, prepareBackup?: () => Promise, verifyControl = false) { const { errorHandler, instanceSettingsRoutes, cloudControlMiddleware } = routeModules.value; const app = express(); - app.use(express.json()); + app.use(express.json({ limit: DEFAULT_JSON_BODY_LIMIT, verify: (req, _res, raw) => { Reflect.set(req, "rawBody", raw); } })); app.use((req, _res, next) => { req.actor = actor; next(); @@ -1190,6 +1191,46 @@ describe("instance settings routes", () => { }); afterEach(() => vi.unstubAllEnvs()); + it("reads the same owner and deadline scope through the real body-bound middleware and production handler", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + const url = `/api/instance/task-drain/status?idleSleepSafety=1&scheduledWake=domain-deadlines-v1&ownerId=${ownerId}`; + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + expect(mockReadIdleSleepSafety).toHaveBeenCalledExactlyOnceWith(mockDb, expect.any(Function), Date.now, + ownerId, expect.any(Function), undefined, undefined, true, true, + { incomingWake: catalog, deadlineCapability: catalog, prepareLocalWork: expect.any(Function) }); + expect(Object.isFrozen(mockReadIdleSleepSafety.mock.calls[0][9].incomingWake)).toBe(true); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(401); + }); + it("rejects body tampering before installing capabilities or consuming the valid request", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + const url = `/api/instance/task-drain/status?idleSleepSafety=1&ownerId=${ownerId}`; + await request(createControlApp(adminActor)).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw + " ").expect(401); + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + }); + it("rejects unsigned and actor-only body authority and cross-route method replay", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }), url = `/api/instance/task-drain/status?idleSleepSafety=1&ownerId=${ownerId}`; + for (const source of ["session", "cloud_tenant", "local_implicit", "cloud_control"]) { + await request(createControlApp({ ...adminActor, source, idleCapabilities: catalog })).post(url) + .set("content-type", "application/json").send(raw).expect(403); + } + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + await request(createControlApp(adminActor)).get(url).set("x-paperclip-cloud-control", token).expect(400); + await request(createControlApp(adminActor)).post("/api/instance/task-drain").set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(401); + await request(createControlApp(adminActor)).delete("/api/instance/task-drain").set("x-paperclip-cloud-control", token).expect(401); + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + }); + it("passes only the immutable signed catalog through real middleware and the production route", async () => { const token = assertion(); const response = await request(createControlApp({ type: "none" })) diff --git a/server/src/__tests__/slack-cloud-forms.test.ts b/server/src/__tests__/slack-cloud-forms.test.ts new file mode 100644 index 00000000000..aafbee6190b --- /dev/null +++ b/server/src/__tests__/slack-cloud-forms.test.ts @@ -0,0 +1,330 @@ +import { isRetryableSlackFormUpdate } from "../services/chat-channels/slack/form-open-callback.js"; +import { slackFormConnectorIdentity, selectSlackFormRetainedConsumer } from "../services/chat-channels/slack/cloud-consumer.js"; +import { createHash, createHmac } from "node:crypto"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { slackFormSubmissionEvidence, withSlackFormSubmissionEvidence, assertSlackFormTerminalAck, lookupSlackFormDelivery, parseSlackFormOperation, type SlackFormBinding, type SlackFormView, type SlackFormOperation } from "../services/chat-channels/slack/cloud-forms.js"; + +import { parseSlackFormProofAcceptance, prepareSlackFormProofAcceptance, projectSlackFormRetainedOwner, type SlackFormTerminalIntent, type SlackFormRetainedRow, openSlackFormSubmissionIntent, prepareSlackFormSubmissionIntent } from "../services/chat-channels/slack/form-session.js"; +import { createLocalEncryptedCipher } from "../secrets/local-encrypted-provider.js"; + +afterEach(() => { vi.unstubAllEnvs(); }); +const hash = (s: string) => createHash("sha256").update(s).digest("hex"); +function fixture() { + const acceptedAt = "2099-01-01T00:00:00.000Z", secret = "0123456789abcdef0123456789abcdef"; + const binding: SlackFormBinding = { version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "a".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: `slack-event:${hash("event")}`, credentialFingerprint: hash("credential") }; + const raw = Buffer.from(new URLSearchParams({ payload: JSON.stringify({ type: "block_actions", trigger_id: "trigger", user: { id: "UTEST" }, team: { id: "TTEST" }, api_app_id: "ATEST" }) }).toString()); + const rawBody = raw.toString("base64"), timestamp = String(Date.parse(acceptedAt) / 1000); + const signature = `v0=${createHmac("sha256", secret).update(`v0:${timestamp}:`).update(raw).digest("hex")}`; + const id = hash("operation"); + const operation: SlackFormOperation = { version: 1, binding, grantId: hash("grant"), id, externalId: `pc-form-${id}`, deliveryId: `slack_form_${id}`, + requestDigest: hash(JSON.stringify([rawBody, timestamp, signature])), triggerDigest: hash("trigger"), userId: "UTEST", acceptedAt, + triggerDeadline: "2099-01-01T00:00:03.000Z", phase: "reserved", view: null, closed: null, acknowledgement: null, rejection: null }; + const packet = { version: 1, instanceId: "instance", environment: "test", operation, acceptedAt, rawBody, timestamp, signature }; + const event = { id: operation.deliveryId, provider: "slack", event: "form_open", installationId: `slack-form-open:${id}`, bindingIds: [`slack-form-open:${id}`], createdAt: acceptedAt, payload: { slackFormOpen: packet } }; + const current = { kind: "current" as const, binding, instanceId: "instance", environment: "test", signingSecret: secret }; + const lookup = vi.fn(async () => ({ version: 1, instanceId: "instance", environment: "test", operation })); + function close(view: SlackFormView): NonNullable { + const body = Buffer.from(new URLSearchParams({ payload: JSON.stringify({ type: "view_closed", api_app_id: "ATEST", team: { id: "TTEST" }, + user: { id: "UTEST" }, view: { id: view.id, hash: view.hash, external_id: view.externalId, app_id: "ATEST" } }) }).toString()); + const rawBody = body.toString("base64"), signature = `v0=${createHmac("sha256", secret).update(`v0:${timestamp}:`).update(body).digest("hex")}`; + return { kind: "closed", view, acceptedAt, requestDigest: hash(JSON.stringify([rawBody, timestamp, signature])), evidence: { + version: 1, kind: "close", submitReceiptId: null, instanceId: "instance", environment: "test", operationId: id, binding, acceptedAt, rawBody, timestamp, signature } }; + } + return { operation, packet, event, current, lookup, close, now: Date.parse(acceptedAt) + 3500 }; +} + +describe("Slack form original evidence and on-demand receipt", () => { + it("pending receipt makes one lookup and retains retry responsibility without SDK or ACK", async () => { + const f = fixture(); + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_receipt_pending", disposition: "retry" }); + expect(f.lookup).toHaveBeenCalledExactlyOnceWith({ binding: f.current.binding, operationId: f.operation.id, deliveryId: f.event.id }); + expect(f.packet.operation.phase).toBe("reserved"); + }); + it("delayed confirmed receipt preserves original acceptance and current generation", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view } }); + const result = await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now); + expect(result.operation.view).toEqual(view); expect(result.operation.acceptedAt).toBe(f.event.createdAt); + expect(result.originalPayload.trigger_id).toBe("trigger"); + }); + it.each(["companyId", "endpointId", "connectionId", "credentialFingerprint", "instanceId"] as const)("foreign %s cannot reach lookup", async key => { + const f = fixture(); + await expect(lookupSlackFormDelivery(f.event, { ...f.current, binding: { ...f.current.binding, [key]: "foreign" } }, f.lookup, f.now)).rejects.toMatchObject({ disposition: "attention" }); + expect(f.lookup).not.toHaveBeenCalled(); + }); + it("original signature failure cannot reset freshness or reach lookup", async () => { + const f = fixture(); f.packet.signature = `v0=${"0".repeat(64)}`; + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_original_authentication_failed" }); + expect(f.lookup).not.toHaveBeenCalled(); + }); + it("lookup successor identity cannot overwrite the original queue operation", async () => { + const f = fixture(); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, requestDigest: hash("successor") } }); + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_lookup_identity_changed", disposition: "attention" }); + expect(f.packet.operation.requestDigest).not.toBe(hash("successor")); + }); + it("confirmed but already closed is surfaced as closed custody, never an update instruction", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + const closed = f.close(view); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view, closed } }); + expect((await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).operation.closed).toEqual(closed); + }); + it("ambiguous provider outcome remains explicit, never becomes fabricated confirmation", async () => { + const f = fixture(); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "ambiguous" } }); + const result = await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now); + expect(result.operation.phase).toBe("ambiguous"); expect(result.operation.view).toBeNull(); + }); + it("active or merely closed views cannot ACK before durable owner transfer", () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "confirmed", view }, "11111111-1111-1111-1111-111111111111")).toThrow("form_view_active"); + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "confirmed", view, closed: f.close(view) }, "11111111-1111-1111-1111-111111111111")).toThrow(); + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "acknowledged", view, closed: f.close(view), + acknowledgement: { appReceiptId: "11111111-1111-1111-1111-111111111111", acceptedAt: f.operation.acceptedAt } }, "11111111-1111-1111-1111-111111111111")).not.toThrow(); + }); + it("retained custody accepts only sealed verified close settlement after local credentials removed", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view, closed: f.close(view) } }); + const result = await lookupSlackFormDelivery(f.event, { kind: "retained", binding: f.current.binding, instanceId: "instance", environment: "test" }, f.lookup, f.now); + expect(result.authority).toBe("retained"); expect(result.operation.closed?.view.id).toBe("VTEST"); + }); + it("retained unresolved operation cannot authorize a new useful form", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view } }); + await expect(lookupSlackFormDelivery(f.event, { kind: "retained", binding: f.current.binding, instanceId: "instance", environment: "test" }, f.lookup, f.now)) + .rejects.toMatchObject({ code: "form_retained_resource_unresolved", disposition: "attention" }); + }); + it("definite no-effect is identity-bound and exchanged trigger is never a no-effect proof", async () => { + const f = fixture(), observation = { source: "provider_response", status: 200, body: { ok: false, error: "expired_trigger_id" } }; + const responseDigest = hash(JSON.stringify([f.operation.id, "expired_trigger_id", observation])); + const rejection: NonNullable = { code: "expired_trigger_id", observedAt: f.operation.acceptedAt, responseDigest, + evidence: { version: 1, kind: "rejection", instanceId: "instance", environment: "test", operationId: f.operation.id, binding: f.current.binding, + observedAt: f.operation.acceptedAt, code: "expired_trigger_id", responseDigest, observation } }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "rejected", rejection } }); + expect((await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).operation.rejection?.responseDigest).toBe(responseDigest); + expect(() => parseSlackFormOperation({ ...f.operation, phase: "rejected", rejection: { ...rejection, code: "exchanged_trigger_id" } })).toThrow(); + }); + it("closed parser refuses unknown fields and conflicting view identity", () => { + const f = fixture(); + expect(() => parseSlackFormOperation({ ...f.operation, token: "forbidden" })).toThrow(); + expect(() => parseSlackFormOperation({ ...f.operation, phase: "confirmed", view: { id: "VTEST", hash: "h", externalId: "foreign" } })).toThrow(); + }); +}); + +function submissionRequest(endpointId = "endpoint") { + const f = fixture(), payload = { type: "view_submission", api_app_id: "ATEST", team: { id: "TTEST" }, user: { id: "UTEST" }, + view: { id: "VTEST", hash: "useful-hash", external_id: f.operation.externalId, app_id: "ATEST", callback_id: "opaque-submit", state: { values: { secret: "private-answer-for-terminal-cipher-proof" } } } }; + const raw = new URLSearchParams({ payload: JSON.stringify(payload) }).toString(), timestamp = "4070908800"; + const signature = `v0=${createHmac("sha256", f.current.signingSecret).update(`v0:${timestamp}:`).update(raw).digest("hex")}`; + const request = new Request("https://paperclip.invalid/slack", { method: "POST", headers: { "content-type": "application/x-www-form-urlencoded", + "x-slack-request-timestamp": timestamp, "x-slack-signature": signature }, body: raw }); + return { f, request, raw, endpointId }; +} + +describe("Slack form submission leaf evidence, without SDK/domain authorization claims", () => { + it("preserves original SDK request and scopes concurrent immutable evidence", async () => { + const a = submissionRequest("a"), b = submissionRequest("b"); + const result = await Promise.all([a, b].map(async f => withSlackFormSubmissionEvidence(f.request, f.endpointId, true, async () => { + await Promise.resolve(); const original = slackFormSubmissionEvidence()!; + expect(Object.isFrozen(original)).toBe(true); expect(original.endpointId).toBe(f.endpointId); + expect(Buffer.from(original.rawBody, "base64").toString()).toBe(f.raw); + expect(await f.request.text()).toBe(f.raw); return original.endpointId; + }))); + expect(result).toEqual(["a", "b"]); expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("absent feature does not read request; failed normal callback discards context", async () => { + const f = submissionRequest(), error = new Error("controlled normal authentication refusal"); + await withSlackFormSubmissionEvidence(f.request, "endpoint", false, async () => { expect(slackFormSubmissionEvidence()).toBeUndefined(); }); + expect(f.request.bodyUsed).toBe(false); + await expect(withSlackFormSubmissionEvidence(f.request, "endpoint", true, async () => { throw error; })).rejects.toBe(error); + expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("bounded capture cannot run the callback after excessive input", async () => { + const request = new Request("https://paperclip.invalid/slack", { method: "POST", body: "x".repeat(1048577) }), work = vi.fn(async () => {}); + await expect(withSlackFormSubmissionEvidence(request, "endpoint", true, work)).rejects.toMatchObject({ code: "form_submission_too_large" }); + expect(work).not.toHaveBeenCalled(); expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("existing local cipher protects original evidence and refuses wrong key or foreign custody", async () => { + const f = submissionRequest(); vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "ab".repeat(32)); + const cipher = createLocalEncryptedCipher("read"); + await withSlackFormSubmissionEvidence(f.request, "endpoint", true, async () => { + const original = slackFormSubmissionEvidence()!; + const intent = prepareSlackFormSubmissionIntent({ version: 1, binding: f.f.current.binding, operationId: f.f.operation.id, deliveryId: f.f.event.id, + userId: "UTEST", grantId: f.f.operation.grantId, publicationId: "publication", interactionId: "interaction", appReceiptId: "11111111-1111-4111-8111-111111111111" }, + { id: "VTEST", hash: "useful-hash", externalId: f.f.operation.externalId }, "22222222-2222-4222-8222-222222222222", original, cipher); + expect(JSON.stringify(intent)).not.toContain("private-answer-for-terminal-cipher-proof"); + expect(JSON.stringify(intent)).not.toContain(original.rawBody); + expect(openSlackFormSubmissionIntent(intent, cipher).rawBody).toBe(original.rawBody); + expect(() => openSlackFormSubmissionIntent({ ...intent, binding: { ...intent.binding, companyId: "foreign" } }, cipher)).toThrow("form_submission_cipher_binding_changed"); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "cd".repeat(32)); + const wrong = createLocalEncryptedCipher("read"); + expect(() => openSlackFormSubmissionIntent(intent, wrong)).toThrow("form_submission_cipher_unavailable"); + }); + }); +}); + +function closedProofFixture() { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }, appReceiptId = "11111111-1111-1111-1111-111111111111"; + const intent: SlackFormTerminalIntent = { version: 1, binding: f.current.binding, operationId: f.operation.id, deliveryId: f.operation.deliveryId, + userId: f.operation.userId, grantId: f.operation.grantId, publicationId: "publication", interactionId: "interaction", appReceiptId, view, + outcome: "closed", submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + const operation: SlackFormOperation = { ...f.operation, phase: "terminal_proven", view, closed: f.close(view), + acknowledgement: { appReceiptId, acceptedAt: f.operation.acceptedAt } }; + const row: SlackFormRetainedRow = { id: appReceiptId, companyId: intent.binding.companyId, endpointId: intent.binding.endpointId, kind: "slack_form_session", + providerActionId: `slack-form-session:${intent.operationId}`, status: "cancelled", payload: { binding: intent.binding, operationId: intent.operationId, + userId: intent.userId, grantId: intent.grantId, publicationId: intent.publicationId, interactionId: intent.interactionId }, result: { phase: "closed", terminalIntent: intent } }; + return { f, intent, operation, row }; +} +describe("Slack form two phase terminal acceptance, provider boundaries only", () => { + it("accepts only actual terminal proof for the same durable App receipt without finalizing Cloud ACK", () => { + const f = closedProofFixture(), before = structuredClone(f.operation); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(parseSlackFormProofAcceptance(accepted)).toEqual(accepted); expect(accepted.appReceiptId).toBe(f.row.id); + expect(f.operation).toEqual(before); expect(f.operation.phase).toBe("terminal_proven"); + }); + it("pending, foreign instance, changed receipt and changed terminal intent cannot become proof accepted", () => { + const f = closedProofFixture(); + expect(() => prepareSlackFormProofAcceptance(f.intent, { ...f.operation, phase: "confirmed", acknowledgement: null }, "instance")).toThrow(); + expect(() => prepareSlackFormProofAcceptance(f.intent, f.operation, "foreign")).toThrow(); + expect(() => prepareSlackFormProofAcceptance({ ...f.intent, appReceiptId: "22222222-2222-2222-2222-222222222222" }, f.operation, "instance")).toThrow(); + expect(() => prepareSlackFormProofAcceptance({ ...f.intent, view: { ...f.intent.view!, hash: "foreign" } }, f.operation, "instance")).toThrow(); + }); + it("phase parser refuses a proof without exact terminal evidence or acceptance", () => { + const f = closedProofFixture(); + expect(() => parseSlackFormOperation({ ...f.operation, acknowledgement: null })).toThrow(); + expect(() => parseSlackFormOperation({ ...f.operation, closed: null })).toThrow(); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(() => parseSlackFormProofAcceptance({ ...accepted, outcome: "invented" })).toThrow(); + expect(() => parseSlackFormProofAcceptance({ ...accepted, providerToken: "forbidden" })).toThrow(); + }); + it("retained accepted history supplies a stable event identity without making completed history polling work", () => { + const f = closedProofFixture(); + const pending = projectSlackFormRetainedOwner(f.row, "instance"); expect(pending?.requiresDelivery).toBe(true); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const retained = projectSlackFormRetainedOwner({ ...f.row, result: { ...f.row.result, cloudProofAcceptance: accepted } }, "instance"); + expect(retained).toMatchObject({ subject: "github-bot-events", companyId: "company", requiresDelivery: false }); + expect(retained?.binding).toEqual(f.intent.binding); + }); + it("foreign malformed history is isolated before parsing and changed current receipt is attention", () => { + const f = closedProofFixture(); + expect(projectSlackFormRetainedOwner({ ...f.row, payload: { binding: { instanceId: "foreign", malformed: true } } }, "instance")).toBeNull(); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(() => projectSlackFormRetainedOwner({ ...f.row, companyId: "foreign" }, "instance")).toThrow(); + expect(() => projectSlackFormRetainedOwner({ ...f.row, result: { ...f.row.result, cloudProofAcceptance: { ...accepted, appReceiptId: "22222222-2222-2222-2222-222222222222" } } }, "instance")).toThrow(); + }); + it("definite no effect proof requires the recorded digest and no view or closed input", () => { + const f = closedProofFixture(), observation = { ok: false, error: "expired_trigger_id" }, responseDigest = hash(JSON.stringify([f.operation.id, "expired_trigger_id", observation])); + const operation: SlackFormOperation = { ...f.operation, view: null, closed: null, rejection: { code: "expired_trigger_id", observedAt: f.operation.acceptedAt, + responseDigest, evidence: { version: 1, kind: "rejection", instanceId: "instance", environment: "test", operationId: f.operation.id, binding: f.operation.binding, + observedAt: f.operation.acceptedAt, code: "expired_trigger_id", responseDigest, observation } } }; + const intent: SlackFormTerminalIntent = { ...f.intent, outcome: "no_effect", view: null, rejectionDigest: responseDigest }; + expect(prepareSlackFormProofAcceptance(intent, operation, "instance").outcome).toBe("no_effect"); + expect(() => prepareSlackFormProofAcceptance({ ...intent, rejectionDigest: hash("foreign") }, operation, "instance")).toThrow(); + }); +}); + +it("submitted proof accepts only the encrypted same submission and is idempotent without another encryption", () => { + const f = submissionRequest(); vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "ab".repeat(32)); + const cipher = createLocalEncryptedCipher("read"), encrypt = vi.spyOn(cipher, "encrypt"), base = closedProofFixture(); + const original = { endpointId: "endpoint", acceptedAt: f.f.operation.acceptedAt, rawBody: Buffer.from(f.raw).toString("base64"), + timestamp: f.request.headers.get("x-slack-request-timestamp")!, signature: f.request.headers.get("x-slack-signature")! }; + const view = { id: "VTEST", hash: "useful-hash", externalId: f.f.operation.externalId }, submitReceiptId = "22222222-2222-4222-8222-222222222222"; + const intent = prepareSlackFormSubmissionIntent(base.intent, view, submitReceiptId, original, cipher); + const operation: SlackFormOperation = { ...base.operation, view, closed: { kind: "submitted", view, acceptedAt: original.acceptedAt, + requestDigest: hash(JSON.stringify([original.rawBody, original.timestamp, original.signature])), evidence: { version: 1, kind: "submission", submitReceiptId, + instanceId: "instance", environment: "test", operationId: base.operation.id, binding: base.operation.binding, acceptedAt: original.acceptedAt, + rawBody: original.rawBody, timestamp: original.timestamp, signature: original.signature } } }; + const accepted = prepareSlackFormProofAcceptance(intent, operation, "instance", cipher); + expect(accepted.outcome).toBe("submitted"); expect(prepareSlackFormProofAcceptance(intent, operation, "instance", cipher)).toEqual(accepted); + expect(encrypt).toHaveBeenCalledTimes(1); + expect(() => prepareSlackFormProofAcceptance(intent, { ...operation, closed: { ...operation.closed!, requestDigest: hash("foreign") } }, "instance", cipher)).toThrow(); +}); + + +describe("retained form consumer identity", () => { + it("old connector mocks and absent Cloud do not acquire form authority", () => { + expect(slackFormConnectorIdentity(null)).toBeNull(); + expect(slackFormConnectorIdentity({ leaseEvents: vi.fn() })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => ({ instanceId: "instance", environment: "foreign" }) })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => { throw new Error("unsupported"); } })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => ({ instanceId: "instance", environment: "test" }) })).toEqual({ instanceId: "instance", environment: "test" }); + }); + it("malformed owned history cannot starve a valid proof-accepted ACK consumer", () => { + const f = closedProofFixture(), accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const good = { ...f.row, payload: { ...f.row.payload, environment: "test" }, result: { ...f.row.result, cloudProofAcceptance: accepted } }; + const malformed = { ...f.row, id: "malformed", payload: { binding: { instanceId: "instance", malformed: true }, environment: "test" } }; + const result = selectSlackFormRetainedConsumer([malformed, good], { instanceId: "instance", environment: "test" }); + expect(result.attention).toEqual([{ actionId: "malformed", code: "form_retained_owner_invalid" }]); + expect(result.owner).toMatchObject({ subject: "github-bot-events", companyId: "company", requiresDelivery: false }); + expect(result.owner?.binding).toEqual(f.intent.binding); + }); + it("a foreign or wrong-environment row cannot supply terminal lease authority", () => { + const f = closedProofFixture(); + const foreign = { ...f.row, payload: { binding: { instanceId: "foreign", malformed: true } } }; + expect(selectSlackFormRetainedConsumer([foreign], { instanceId: "instance", environment: "test" })).toEqual({ owner: null, attention: [] }); + const result = selectSlackFormRetainedConsumer([{ ...f.row, payload: { ...f.row.payload, environment: "production" } }], { instanceId: "instance", environment: "test" }); + expect(result.owner).toBeNull(); expect(result.attention).toHaveLength(1); + }); + it("lost enrollment response retains public custody without inventing a credential binding", () => { + const f = closedProofFixture(), { eventBindingId, credentialFingerprint: _, ...identity } = f.intent.binding; + const row: SlackFormRetainedRow = { ...f.row, kind: "slack_form_owner", payload: { enrollment: { ...identity, bindingId: eventBindingId }, subject: "actor", environment: "test" }, result: { phase: "staged" } }; + const result = selectSlackFormRetainedConsumer([row], { instanceId: "instance", environment: "test" }); + expect(result.attention).toEqual([]); + expect(result.owner).toEqual({ subject: "github-bot-events", companyId: "company", binding: null, requiresDelivery: true }); + }); +}); + + +it("retained session selection keeps unfinished GC custody but completed exact receipt history supplies no consumer", () => { + const f = closedProofFixture(), proof = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const receipt = { version: 1 as const, kind: "slack_form_finalized" as const, binding: f.intent.binding, environment: "test" as const, + operationId: f.intent.operationId, deliveryId: f.intent.deliveryId, appReceiptId: f.row.id, grantId: f.intent.grantId, + publicationId: f.intent.publicationId, interactionId: f.intent.interactionId, userId: f.intent.userId, + viewDigest: hash(JSON.stringify([f.intent.view!.id, f.intent.view!.hash, f.intent.view!.externalId])), + terminalDigest: proof.terminalDigest, outcome: "closed" as const, finalizedAt: f.operation.acceptedAt }; + const base = { ...f.row, payload: { ...f.row.payload, environment: "test" }, result: { ...f.row.result, cloudProofAcceptance: proof, + formAck: { version: 1, phase: "transferred", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }, nextAttemptAt: null } }; + const identity = { instanceId: "instance", environment: "test" as const }; + expect(selectSlackFormRetainedConsumer([base], identity)).toEqual({ owner: null, attention: [] }); + for (const phase of ["accepted", "gc_pending"]) { + const pending = { ...base, result: { ...base.result, formAck: { ...base.result.formAck, phase }, nextAttemptAt: f.operation.acceptedAt } }; + expect(selectSlackFormRetainedConsumer([pending], identity).owner).toMatchObject({ companyId: f.row.companyId, subject: "github-bot-events", requiresDelivery: false }); + } + const malformed = { ...base, result: { ...base.result, formAck: { ...base.result.formAck, receipt: { ...receipt, binding: { ...receipt.binding, companyId: "foreign" } } } } }; + const result = selectSlackFormRetainedConsumer([malformed], identity); + expect(result.owner !== null || result.attention.length > 0).toBe(true); + const stillDue = { ...base, result: { ...base.result, nextAttemptAt: f.operation.acceptedAt } }; + expect(selectSlackFormRetainedConsumer([stillDue], identity).owner).not.toBeNull(); +}); + +describe("Slack held-view retry boundary", () => { + const limited = () => Object.assign(new Error("controlled definite Slack rejection"), { code: "slack_webapi_platform_error", data: { error: "ratelimited" } }); + const held = () => { + const f = fixture(), view = { id: "VRETRY", hash: "loading-hash", externalId: f.operation.externalId }; + return { operation: { ...f.operation, phase: "confirmed" as const, view }, currentBinding: f.current.binding, + environment: "test", sessionActionId: "held-session", updateAttempted: true }; + }; + it("retains the exact held attempt on a definite retryable update refusal", () => { + const context = held(), before = structuredClone(context); + expect(isRetryableSlackFormUpdate(context, limited())).toBe(true); + expect(context).toEqual(before); + }); + it("does not replay an ambiguous update or an accepted update whose local commit failed", () => { + const context = held(); + expect(isRetryableSlackFormUpdate(context, new Error("response lost after possible update"))).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, updatedView: { ...context.operation.view, hash: "accepted-hash" } }, limited())).toBe(false); + }); + it("does not reopen a native trigger, unprepared session, stale binding, or closed view", () => { + const context = held(), f = fixture(); + expect(isRetryableSlackFormUpdate(undefined, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, sessionActionId: undefined }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, updateAttempted: false }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, currentBinding: { ...context.currentBinding, runtimeGeneration: 2 } }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, operation: { ...context.operation, closed: f.close(context.operation.view) } }, limited())).toBe(false); + }); + it("preserves definite invalid-view and stale-hash refusals as nonretryable", () => { + for (const error of ["invalid_arguments", "hash_conflict"]) expect(isRetryableSlackFormUpdate(held(), + Object.assign(new Error("controlled definite view refusal"), { code: "slack_webapi_platform_error", data: { error } }))).toBe(false); + }); +}); diff --git a/server/src/__tests__/slack-cloud-owner-work.test.ts b/server/src/__tests__/slack-cloud-owner-work.test.ts index 587b2ea822a..7343cce4b98 100644 --- a/server/src/__tests__/slack-cloud-owner-work.test.ts +++ b/server/src/__tests__/slack-cloud-owner-work.test.ts @@ -6,7 +6,7 @@ import { and, eq, sql } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { SLACK_CHAT_BOT_SCOPES } from "@paperclipai/shared"; import { activityLog, agentRuntimeState, agentTaskSessions, agentWakeupRequests, agents, applyPendingMigrations, authUsers, - chatDeliveries, chatEndpoints, chatExternalPrincipals, companySecretBindings, chatIdentityLinks, chatSlackRegistrations, companies, companyMemberships, + chatActions, chatDeliveries, chatEndpoints, chatExternalPrincipals, companySecretBindings, chatIdentityLinks, chatSlackRegistrations, companies, companyMemberships, companySecrets, companySkills, costEvents, financeEvents, heartbeatRunEvents, heartbeatRuns, issueComments, issues, toolApplications, toolConnections, closeCheckpointDb, closeFileStorageDb, closeLivenessDb, createDb } from "@paperclipai/db"; import { databaseWorkPending, reconcileDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db/work-signals"; @@ -229,7 +229,16 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu expect(Date.parse(retained.nextAttemptAt!)).toBeGreaterThan(Date.now()); expect(cloud.calls.slice(start).filter(call => call.enrollment?.endpointId === a)).toEqual([]); const retryAt = Date.parse(retained.nextAttemptAt!); - expect(owner.nextPollAt()).toBe(retryAt); + const formRetries = await db.select({ result: chatActions.result }).from(chatActions).where(and( + eq(chatActions.companyId, companyId), + sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`, + sql`${chatActions.payload}->>'environment' = ${cloud.config.environment}`, + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${cloud.config.instanceId}`, + sql`${chatActions.result}->>'nextAttemptAt' is not null`)); + expect(formRetries.some(value => value.result?.failureCode === "form_owner_broker_unavailable")).toBe(true); + const formDeadlines = formRetries.map(value => Date.parse(String(value.result?.nextAttemptAt))); + expect(formDeadlines.every(Number.isFinite)).toBe(true); + expect(owner.nextPollAt()).toBe(Math.min(retryAt, ...formDeadlines)); releases.splice(0).forEach(release => release()); await Promise.all(heldLeases); pollTime = retryAt - 1; await owner.pollOnce(); diff --git a/server/src/__tests__/slack-enabled-quiet.postgres.test.ts b/server/src/__tests__/slack-enabled-quiet.postgres.test.ts new file mode 100644 index 00000000000..eb9c26267c6 --- /dev/null +++ b/server/src/__tests__/slack-enabled-quiet.postgres.test.ts @@ -0,0 +1,361 @@ +import { registerBudgetEnforcementWork } from "../services/budget-enforcement-worker.js"; +import { deliverBudgetResumeContinuations } from "../services/budget-resume-continuation.js"; +import { registerChatDeliveryWork } from "../services/chat-delivery-work.js"; +import { registerChatActionWork } from "../services/chat-action-work.js"; +import { isIdleTaskDrainActive, idleWorkSnapshot } from "../services/task-admission.js"; +import { actorMiddleware } from "../middleware/auth.js"; +import { issueRoutes } from "../routes/issues.js"; +import { agentRoutes } from "../routes/agents.js"; +import { createStorageService } from "../storage/service.js"; +import { createLocalDiskStorageProvider } from "../storage/local-disk-provider.js"; +import { parseSlackFormAckWork, assertSlackFormReceiptHistory } from "../services/chat-channels/slack/form-ack-work.js"; +import { verifySlackFormFinalizedReceipt } from "../services/chat-channels/slack/receipt-authority.js"; +import { createHmac, randomUUID } from "node:crypto"; +import { once } from "node:events"; +import { readFile, mkdtemp, rm } from "node:fs/promises"; +import { createServer } from "node:http"; +import os from "node:os"; +import { fileURLToPath } from "node:url"; +import path from "node:path"; +import express from "express"; +import { and, eq } from "drizzle-orm"; +import { expect, it, vi } from "vitest"; +import { activityLog, agentRuntimeState, agentTaskSessions, agentWakeupRequests, agents, applyPendingMigrations, authUsers, + chatActions, chatConversations, chatEndpoints, chatExternalPrincipals, chatPublications, chatSlackRegistrations, + closeCheckpointDb, closeFileStorageDb, closeLivenessDb, companies, companyMemberships, companySecrets, companySkills, + costEvents, createDb, financeEvents, heartbeatRunEvents, heartbeatRuns, issueComments, issues, toolApplications, toolConnections } from "@paperclipai/db"; +import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; +import { guardSlackFixtureNetwork } from "./helpers/slack-cloud/transport.js"; +import { ChatRuntime } from "../services/chat-runtime.js"; +import { CAPABILITIES } from "../services/chat-channels/provider-catalog.js"; +import { chatChannelService } from "../services/chat-channels.js"; +import { reconcilePersistedRuntimeServicesOnStartup } from "../services/workspace-runtime.js"; +import { markIdleStartupComplete } from "../services/idle-local-work.js"; +import { idleAdmissionMiddleware, trackIdleRequestHandlers } from "../middleware/idle-admission.js"; +import { heartbeatService } from "../services/heartbeat.js"; +import { githubConnectionEventService } from "../services/github-connection-events.js"; +import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js"; +import { registerGitHubConnectionEventWork } from "../services/github-connection-work.js"; +import { issueThreadInteractionService } from "../services/issue-thread-interactions.js"; +import { cloudControlMiddleware } from "../middleware/cloud-control.js"; +import { instanceSettingsRoutes } from "../routes/instance-settings.js"; +import { connectionEventWakeRoutes } from "../routes/connection-event-wake.js"; +import { currentSlackFormOwnerBinding, slackFormSetupRecord } from "../services/chat-channels/slack/cloud-forms.js"; + +type PlatformView = { id: string; hash: string; external_id: string; app_id: string; callback_id: string; private_metadata: string; + blocks: { type: string; block_id?: string; element?: { action_id: string; type: string } }[]; state: { values: Record> } }; +type HostState = { + usefulReplies?: { id: string; threadTs: string; runId: string }[]; + bindingCount?: number; coverageEntries?: number; responseStatus?: number; normalRouteReached?: boolean; tamperedStatus?: number; headerBytes?: number; + calls: { method: string; at: number; body?: unknown }[]; views: PlatformView[]; failures: string[]; + wakerStarts: number; droppedHints: number; lostTerminalReplies: number; lostAckReplies: number; + idleProjection?: { backgroundWork: "none" | "present" | "unknown"; checks: { reason: string; state: string }[]; capabilityPresent: boolean }; + instance: { eventWakeVersion?: number }; targets: { delivery_id: string; lease_id: string | null; lease_expires_at: string | null }[]; + bindings: { id: string; profile?: string; metadata?: Record }[]; +}; +type Fixture = { + companyId: string; endpointId: string; connectionId: string; applicationId: string; agentId: string; actorId: string; + instanceId: string; publicId: string; teamId: string; appId: string; botId: string; userId: string; channelId: string; threadTs: string; + botToken: string; signingSecret: string; controlNonce: string; signPrivate: string; sealPrivate: string; controlJwks: unknown; origin: string; tenantOrigin: string; +}; +const appUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_SLACK_TEST_DATABASE_URL"); +const stateFile = process.env.SLACK_FORM_JOURNEY_STATE_FILE; +const shortGate = process.env.SLACK_FORM_SHORT_GATE; +if (appUrl && stateFile && shortGate !== "callback") throw new Error("One closed enabled-quiet callback qualification required"); + +it.skipIf(!appUrl || !stateFile)("enabled Slack finishes ordinary useful tasks and becomes composed quiet before and after fresh signed work", async () => { + const network = guardSlackFixtureNetwork({ requirePreload: true }); + let db: ReturnType | undefined; + let heartbeat: ReturnType | undefined; + let service: ReturnType | undefined; + let coordinator: ReturnType | undefined; + let worker: ReturnType | undefined; + let budgetWorker: ReturnType | undefined; + let fixture: Fixture | undefined, directory: string | undefined, primary: unknown; + const errors: unknown[] = [], ownerErrors: unknown[] = [], observations: unknown[] = []; + const app = express(); + const server = createServer(app); + const sleep = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)); + try { + fixture = JSON.parse(await readFile(stateFile!, "utf8")) as Fixture; + const f = fixture; + expect(new URL(f.origin).hostname).toBe("127.0.0.1"); + expect(new URL(appUrl!).pathname).toMatch(/^\/sleep2_slack_form_[a-z0-9_]+_app_test$/); + const control = async (name: string, body?: unknown): Promise => { + const response = await fetch(`${f.origin}/__fixture/${name}`, { method: body === undefined ? "GET" : "POST", + headers: { "x-fixture-control": f.controlNonce, "content-type": "application/json" }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) }); + expect(response.status).toBe(200); + return await response.json() as HostState; + }; + const signed = (value: unknown, form = false) => { + const body = form ? new URLSearchParams({ payload: JSON.stringify(value) }).toString() : JSON.stringify(value); + const timestamp = String(Math.floor(Date.now() / 1000)); + return { body, headers: { "content-type": form ? "application/x-www-form-urlencoded" : "application/json", "x-slack-request-timestamp": timestamp, + "x-slack-signature": `v0=${createHmac("sha256", f.signingSecret).update(`v0:${timestamp}:${body}`).digest("hex")}` } }; + }; + const post = async (request: ReturnType, origin = f.origin) => { + const started = Date.now(); + const response = await fetch(`${origin}/api/chat-webhooks/${f.publicId}/slack`, { method: "POST", ...request }); + const text = await response.text(); + return { status: response.status, text, elapsed: Date.now() - started }; + }; + await applyPendingMigrations(appUrl!); + db = createDb(appUrl!, { maxConnections: 2, maxLifetimeSeconds: 0 }); + const database = db; + const reconstruction = await reconcilePersistedRuntimeServicesOnStartup(database); + expect(reconstruction).toEqual({ reconciled: 0, adopted: 0, stopped: 0, backfilled: 0, restarted: 0, restartFailed: 0, exposureReservationDrift: [] }); + directory = await mkdtemp(path.join(os.tmpdir(), "form-journey-")); + for (const [key, value] of Object.entries({ + PAPERCLIP_CLOUD_CONNECTOR_BASE_URL: f.origin, PAPERCLIP_CLOUD_CONNECTOR_ENVIRONMENT: "test", PAPERCLIP_CLOUD_CONNECTOR_INSTANCE_ID: f.instanceId, + PAPERCLIP_CLOUD_CONNECTOR_SIGN_PRIVATE_KEY: f.signPrivate, PAPERCLIP_CLOUD_CONNECTOR_SEAL_PRIVATE_KEY: f.sealPrivate, + PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify(f.controlJwks), PAPERCLIP_CLOUD_STACK_ID: f.instanceId, + PAPERCLIP_SECRETS_MASTER_KEY_FILE: path.join(directory, "master.key"), + PAPERCLIP_AGENT_JWT_SECRET: `${randomUUID()}${randomUUID()}`, PAPERCLIP_API_URL: f.tenantOrigin, + })) vi.stubEnv(key, value); + const now = new Date(); + await db.insert(companies).values({ id: f.companyId, name: "Form boundary journey", issuePrefix: `F${f.companyId.slice(0, 7)}` }); + await db.insert(authUsers).values({ id: f.actorId, name: "Form operator", email: `${f.actorId}@example.test`, emailVerified: true, createdAt: now, updatedAt: now }); + await db.insert(companyMemberships).values({ companyId: f.companyId, principalType: "user", principalId: f.actorId, status: "active", membershipRole: "operator" }); + await db.insert(agents).values({ id: f.agentId, companyId: f.companyId, name: "Scoped useful process", adapterType: "codex_local", status: "idle", + adapterConfig: { engine: "cli", command: fileURLToPath(new URL("./helpers/slack-cloud/controlled-codex-cli.cjs", import.meta.url)), cwd: directory, + env: { SLACK_FORM_FIXTURE_PEER_ORIGIN: f.tenantOrigin, CODEX_HOME: path.join(directory, "owned-controlled-cli-home"), + OPENAI_API_KEY: "controlled-command-no-real-provider-authority" }, timeoutSec: 10, graceSec: 1 }, + runtimeConfig: { heartbeat: { wakeOnDemand: true, intervalSec: 0 } } }); + await db.insert(companyMemberships).values({ companyId: f.companyId, principalType: "agent", principalId: f.agentId, status: "active", membershipRole: "member" }); + await db.insert(toolApplications).values({ id: f.applicationId, companyId: f.companyId, name: "Slack", type: "chat" }); + await db.insert(toolConnections).values({ id: f.connectionId, companyId: f.companyId, applicationId: f.applicationId, name: "Slack form", uid: f.endpointId, + transport: "chat_sdk", connectionPurpose: "channel", enabled: true, status: "active" }); + await db.insert(chatEndpoints).values({ id: f.endpointId, companyId: f.companyId, connectionId: f.connectionId, assignedAgentId: f.agentId, publicId: f.publicId, + provider: "slack", status: "draft", sponsorUserId: f.actorId, allowGroupChats: true, allowUnlinkedPeople: false, + capabilities: CAPABILITIES.slack, setup: { step: "provider_setup" } }); + await db.insert(chatSlackRegistrations).values({ companyId: f.companyId, endpointId: f.endpointId, requestId: randomUUID(), status: "configured", appId: f.appId, + manifest: {}, manifestHash: "fixture", callbackUri: `https://tenant.example/api/chat-webhooks/${f.publicId}/slack`, createdByUserId: f.actorId }); + const runtime = new ChatRuntime(), replace = runtime.replaceEndpoint.bind(runtime); + vi.spyOn(runtime, "replaceEndpoint").mockImplementation(input => { + if (input.providerConfig.provider !== "slack") throw new Error("Unexpected form fixture provider"); + return replace({ ...input, providerConfig: { ...input.providerConfig, credentials: { ...input.providerConfig.credentials, apiUrl: `${f.origin}/fixture/slack/` } } }); + }); + const providerFetch: typeof fetch = (input, init) => { + const url = new URL(input instanceof Request ? input.url : String(input)); + if (url.origin !== "https://slack.com" || !url.pathname.startsWith("/api/")) throw new Error("Unexpected provider leaf request"); + return fetch(`${f.origin}/fixture/slack/${url.pathname.slice(5)}`, init); + }; + heartbeat = heartbeatService(db); + service = chatChannelService(db, { runtime, heartbeat, fetch: providerFetch, publicBaseUrl: "https://tenant.example" }); + const channels = service; + app.use(idleAdmissionMiddleware); + app.use(actorMiddleware(database, { deploymentMode: "authenticated" })); + app.use(cloudControlMiddleware()); app.use(connectionEventWakeRoutes(db)); + app.use("/api/instance/task-drain", express.json()); + app.use("/api", instanceSettingsRoutes(database)); + app.use("/api/issues", express.json()); + app.use("/api", issueRoutes(database, createStorageService(createLocalDiskStorageProvider(path.join(directory, "storage"))))); + app.use("/api", agentRoutes(database, { deploymentMode: "authenticated" })); + app.post(`/api/chat-webhooks/${f.publicId}/slack`, express.raw({ type: () => true, limit: "1mb" }), async (req, res, next) => { + try { + const headers = new Headers(); + for (const [name, value] of Object.entries(req.headers)) if (typeof value === "string" && name !== "host") headers.set(name, value); + const response = await channels.handleWebhook(f.publicId, "slack", new Request(`https://tenant.example${req.path}`, { method: "POST", headers, body: new Uint8Array(req.body) })); + res.status(response.status); response.headers.forEach((value, name) => res.setHeader(name, value)); res.end(Buffer.from(await response.arrayBuffer())); + } catch (error) { next(error); } + }); + app.use((error: unknown, _req: express.Request, res: express.Response, _next: express.NextFunction) => { ownerErrors.push(error); res.status(500).end("fixture handler failed"); }); + expect(new URL(f.tenantOrigin).hostname).toBe("127.0.0.1"); + trackIdleRequestHandlers(app); + server.listen(Number(new URL(f.tenantOrigin).port), "127.0.0.1"); await once(server, "listening"); + const address = server.address(); if (!address || typeof address === "string") throw new Error("Missing tenant fixture address"); + const tenantOrigin = `http://127.0.0.1:${address.port}`; + expect(tenantOrigin).toBe(f.tenantOrigin); + await control("tenant", { origin: tenantOrigin }); + const startOwner = async () => { + // Same queue owners and idle-drain gate as normal app.ts composition. + // This fixture has no optional CloudWarmStandby owner (normal default). + const canDispatch = () => !isIdleTaskDrainActive(); + coordinator = createDeliveryWorkCoordinator({ owner: database, canRun: canDispatch, canReconcile: () => true, + onError: error => ownerErrors.push(error) }); + const deliveries = registerChatDeliveryWork(coordinator, channels, canDispatch); + const actions = registerChatActionWork(coordinator, channels, canDispatch); + const ownedHeartbeat = heartbeat; + if (!ownedHeartbeat) throw new Error("Missing normal heartbeat budget authority"); + budgetWorker = registerBudgetEnforcementWork(database, coordinator, { + cancelWorkForScope: ownedHeartbeat.cancelBudgetScopeWork, + deliverResumeContinuations: () => deliverBudgetResumeContinuations(database, { + prepareChatRetry: channels.prepareFailedChatRunRetry, + resumeTaskRetry: (companyId, requestId) => ownedHeartbeat.resumeTaskRetry(companyId, requestId), + }), + }); + worker = registerGitHubConnectionEventWork(database, coordinator, githubConnectionEventService(database), canDispatch); + await Promise.all([deliveries.ready, actions.ready, worker.ready, budgetWorker.ready]); + }; + const stopOwner = async () => { await worker?.stop(); await budgetWorker?.stop(); await coordinator?.stop(); worker = undefined; budgetWorker = undefined; coordinator = undefined; }; + await startOwner(); + markIdleStartupComplete({ scheduledBackups: false }); + await service.configure(f.endpointId, { action: "configure", credentials: { botToken: f.botToken, signingSecret: f.signingSecret } }, f.actorId); + expect((await post(signed({ type: "url_verification", team_id: f.teamId, api_app_id: f.appId, challenge: "form-journey-challenge" }), tenantOrigin)).status).toBe(200); + await service.configure(f.endpointId, { action: "verify" }, f.actorId); + const [principal] = await db.insert(chatExternalPrincipals).values({ companyId: f.companyId, provider: "slack", providerAccountId: f.teamId, externalId: f.userId, kind: "user", displayName: "Operator" }).returning(); + expect(principal).toBeDefined(); + const link = await service.createLinkIntent(f.endpointId, principal!.id, 1800); + const token = new URL(link.confirmationUrl).searchParams.get("token"); if (!token) throw new Error("Missing normal identity link token"); + await service.confirmIdentityLink(token, f.actorId); + expect((await service.test(f.endpointId, { optionalSlackTestForUser: f.actorId })).status).toBe("active"); + const endpoint = async () => (await database.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!; + await vi.waitFor(async () => { + expect(currentSlackFormOwnerBinding(slackFormSetupRecord((await endpoint()).setup))).not.toBeNull(); + expect((await control("state")).instance.eventWakeVersion).toBe(1); + }, { timeout: 15_000 }); + const startupProjection = (await control("idle-projection", {})).idleProjection; + expect(startupProjection).toBeDefined(); + console.info(JSON.stringify({ diagnostic: "slack_form_idle_startup", ...startupProjection })); + expect(startupProjection?.checks.filter(value => value.reason.startsWith("local.") && value.state === "unknown")).toEqual([]); + const bootstrap = signed({ type: "event_callback", event_id: `EvForm${randomUUID().replaceAll("-", "")}`, team_id: f.teamId, api_app_id: f.appId, + event: { type: "app_mention", text: `<@${f.botId}> help me choose`, user: f.userId, channel: f.channelId, ts: f.threadTs, event_ts: f.threadTs } }); + expect((await post(bootstrap)).status).toBe(200); + const conversation = async () => (await database.select().from(chatConversations).where(eq(chatConversations.endpointId, f.endpointId)))[0]; + await vi.waitFor(async () => expect(await conversation()).toBeDefined(), { timeout: 15_000 }); + const current = (await conversation())!; + const enabledIdentity = async () => { + const [endpoint] = await database.select().from(chatEndpoints).where(and(eq(chatEndpoints.companyId, f.companyId), eq(chatEndpoints.id, f.endpointId))); + const [connection] = await database.select().from(toolConnections).where(and(eq(toolConnections.companyId, f.companyId), eq(toolConnections.id, f.connectionId))); + expect(endpoint?.status).toBe("active"); expect(connection?.enabled).toBe(true); + return { endpointId: endpoint!.id, connectionId: connection!.id, capabilities: endpoint!.capabilities, setup: endpoint!.setup }; + }; + const configured = await enabledIdentity(); + const usefulRuns = async (count: number) => { + const runs = await database.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, f.companyId), eq(heartbeatRuns.agentId, f.agentId))); + expect(runs).toHaveLength(count); + const receipts = []; + for (const run of runs) { + expect(run.status).toBe("succeeded"); expect(run.exitCode).toBe(0); + const issueId = run.contextSnapshot?.issueId; + if (typeof issueId !== "string") throw new Error("Missing admitted actual task identity"); + const [task] = await database.select().from(issues).where(and(eq(issues.companyId, f.companyId), eq(issues.id, issueId))); + expect(task?.status).toBe("done"); expect(task?.checkoutRunId).toBeNull(); expect(task?.executionRunId).toBeNull(); + const replies = await database.select().from(issueComments).where(and(eq(issueComments.issueId, issueId), eq(issueComments.createdByRunId, run.id), eq(issueComments.authorAgentId, f.agentId))); + expect(replies).toHaveLength(1); expect(replies[0]?.body).toBe(`Scoped Slack useful reply for run ${run.id}`); + const publications = await database.select().from(chatPublications).where(and(eq(chatPublications.companyId, f.companyId), eq(chatPublications.endpointId, f.endpointId), eq(chatPublications.commentId, replies[0]!.id))); + expect(publications).toHaveLength(1); expect(publications[0]?.state).toBe("published"); expect(publications[0]?.providerMessageId).toBeTruthy(); + const platform = await control("state"); + expect(platform.usefulReplies?.filter(reply => reply.runId === run.id && reply.id === publications[0]!.providerMessageId)).toHaveLength(1); + receipts.push({ runId: run.id, issueId, commentId: replies[0]!.id, publicationId: publications[0]!.id, providerMessageId: publications[0]!.providerMessageId }); + } + expect(new Set(receipts.map(value => value.issueId)).size).toBe(count); + return receipts.sort((left, right) => left.runId.localeCompare(right.runId)); + }; + type WireSnapshot = { pid: number; observedAt: number; pgConnectAttempts: number; pgConnects: number; pgCloses: number; + pgWriteCalls: number; outboundRequests: number; nonPgConnectAttempts: number; livePgSockets: number }; + const wire = Reflect.get(process, Symbol.for("paperclip.slack.fixture.prewire")) as { + pid: number; wireSnapshot(): WireSnapshot; waitForPgSocketsClosed(timeoutMs: number): Promise; + beginWireWindow(): WireSnapshot; endWireWindow(): { before: WireSnapshot; after: WireSnapshot; peakPgSockets: number; + pgWriteCalls: number; pgConnectAttempts: number; pgConnects: number; pgCloses: number; outboundRequests: number; nonPgConnectAttempts: number }; + }; + expect(wire?.pid).toBe(process.pid); + const observeQuiet = async (label: string, count: number) => { + const settlementDeadline = Date.now() + 95_000; + let ordinaryQueryWitness: { before: WireSnapshot; after: WireSnapshot } | undefined; + let lastInspection: { observedAt: number; projection: HostState["idleProjection"]; coordinatorNextWakeAt: number | null; localWork: ReturnType; ownerErrorCount: number } | undefined; + let reportedSignature: string | undefined, diagnosticTransitions = 0; + try { + await vi.waitFor(async () => { + const queryBefore = wire.wireSnapshot(); + await usefulRuns(count); // Existing ordinary reads are the counter liveness control; no extra query. + const queryAfter = wire.wireSnapshot(); + expect(queryAfter.pgWriteCalls).toBeGreaterThan(queryBefore.pgWriteCalls); + ordinaryQueryWitness ??= { before: queryBefore, after: queryAfter }; + const projection = (await control("idle-projection", {})).idleProjection; + // Retain only values already returned by this required normal read. + // Passive snapshots below do not query, drain, schedule or change authority. + lastInspection = { observedAt: Date.now(), projection, coordinatorNextWakeAt: coordinator?.nextWakeAt() ?? null, + localWork: idleWorkSnapshot(), ownerErrorCount: ownerErrors.length }; + const signature = JSON.stringify(projection); + if (signature !== reportedSignature && diagnosticTransitions < 16) { + reportedSignature = signature; diagnosticTransitions += 1; + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_owner_observation", label, + sequence: diagnosticTransitions, ...lastInspection, noAdditionalRead: true })); } catch { /* Observation cannot change assertion. */ } + } + expect(projection).toBeDefined(); expect(projection?.backgroundWork).toBe("none"); + }, { timeout: Math.max(1, settlementDeadline - Date.now()), interval: 250 }); + } catch (error) { + // Log before finally deletes fixture rows, without extra SQL/HTTP. + // The original error remains primary even if evidence output fails. + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_wait_failure", label, + lastInspection, observedAt: Date.now(), noAdditionalRead: true })); } catch { /* Preserve original failure. */ } + throw error; + } + const before = { observedAt: Date.now(), receipts: await usefulRuns(count), config: await enabledIdentity(), state: await control("state"), projection: (await control("idle-projection", {})).idleProjection }; + expect(before.state.targets).toHaveLength(0); expect(before.state.failures).toEqual([]); expect(before.config).toEqual(configured); + expect(before.projection?.backgroundWork).toBe("none"); + // All existing before-snapshot reads above are complete. Wait only on + // actual close events, inside the SAME original 95s settlement budget. + console.info(JSON.stringify({ diagnostic: "slack_enabled_used_pg_close_before", label, + observedAt: Date.now(), settlementDeadline, ordinaryQueryWitness, wire: wire.wireSnapshot(), noAdditionalRead: true })); + try { await wire.waitForPgSocketsClosed(settlementDeadline - Date.now()); } + catch (error) { + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_used_pg_close_failure", label, + observedAt: Date.now(), settlementDeadline, wire: wire.wireSnapshot(), noAdditionalRead: true })); } catch { /* Preserve original failure. */ } + throw error; + } + expect(wire.wireSnapshot().livePgSockets).toBe(0); + const wireBefore = wire.beginWireWindow(); + await sleep(30_000); // Original observation duration; no SQL/control reads in this window. + const wireWindow = wire.endWireWindow(); // Immediately after sleep, BEFORE any after-snapshot query. + console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_wire_window", label, ordinaryQueryWitness, wireBefore, wireWindow })); + expect(wireWindow.pgWriteCalls).toBe(0); expect(wireWindow.pgConnectAttempts).toBe(0); + expect(wireWindow.pgConnects).toBe(0); expect(wireWindow.pgCloses).toBe(0); + expect(wireWindow.peakPgSockets).toBe(0); expect(wireWindow.after.livePgSockets).toBe(0); + expect(wireWindow.outboundRequests).toBe(0); expect(wireWindow.nonPgConnectAttempts).toBe(0); + const after = { observedAt: Date.now(), receipts: await usefulRuns(count), config: await enabledIdentity(), state: await control("state"), projection: (await control("idle-projection", {})).idleProjection }; + expect(after.projection?.backgroundWork).toBe("none"); expect(after.receipts).toEqual(before.receipts); expect(after.config).toEqual(configured); + expect(after.state.targets).toHaveLength(0); expect(after.state.failures).toEqual([]); + expect(after.state.usefulReplies).toEqual(before.state.usefulReplies); + expect(after.state.calls).toHaveLength(before.state.calls.length); + console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_window", label, startedAt: before.observedAt, endedAt: after.observedAt, + runCount: count, usefulReplies: after.state.usefulReplies?.length, targets: after.state.targets.length, + configurationRetained: true, before: before.projection, after: after.projection, noPhysicalOrSqlZeroClaim: true })); + }; + await observeQuiet("initial-useful-task", 1); + const secondThread = (Number(f.threadTs) + 1).toFixed(6); + const fresh = signed({ type: "event_callback", event_id: `EvFresh${randomUUID().replaceAll("-", "")}`, team_id: f.teamId, api_app_id: f.appId, + event: { type: "app_mention", text: `<@${f.botId}> complete a fresh useful task`, user: f.userId, channel: f.channelId, ts: secondThread, event_ts: secondThread } }); + const admitted = await post(fresh); expect(admitted.status).toBe(200); expect(admitted.elapsed).toBeLessThan(3000); + const duplicate = await post(fresh); expect(duplicate.status).toBe(200); expect(duplicate.elapsed).toBeLessThan(3000); + await observeQuiet("fresh-useful-reactivation-and-duplicate", 2); + await usefulRuns(2); expect(await enabledIdentity()).toEqual(configured); + expect(ownerErrors).toEqual([]); + } catch (error) { primary = error; } + finally { + for (const close of [async () => worker?.stop(), async () => budgetWorker?.stop(), async () => coordinator?.stop(), async () => service?.shutdown(), async () => heartbeat?.drainActiveRunExecutions(), + async () => { if (server.listening) { server.close(); server.closeAllConnections(); await once(server, "close"); } }]) { + try { await close(); } catch (error) { errors.push(error); } + } + if (db && fixture) { + const f = fixture; + try { + await db.delete(chatEndpoints).where(eq(chatEndpoints.companyId, f.companyId)); + await db.delete(financeEvents).where(eq(financeEvents.companyId, f.companyId)); + await db.delete(costEvents).where(eq(costEvents.companyId, f.companyId)); + await db.delete(activityLog).where(eq(activityLog.companyId, f.companyId)); + await db.delete(agentTaskSessions).where(eq(agentTaskSessions.companyId, f.companyId)); + await db.delete(agentRuntimeState).where(eq(agentRuntimeState.companyId, f.companyId)); + await db.delete(heartbeatRunEvents).where(eq(heartbeatRunEvents.companyId, f.companyId)); + await db.delete(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId)); + await db.delete(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId)); + await db.delete(issueComments).where(eq(issueComments.companyId, f.companyId)); + await db.delete(issues).where(eq(issues.companyId, f.companyId)); + await db.delete(agents).where(eq(agents.companyId, f.companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, f.companyId)); + await db.delete(companySecrets).where(eq(companySecrets.companyId, f.companyId)); + await db.delete(companySkills).where(eq(companySkills.companyId, f.companyId)); + await db.delete(companies).where(eq(companies.id, f.companyId)); await db.delete(authUsers).where(eq(authUsers.id, f.actorId)); + } catch (error) { errors.push(error); } + for (const close of [closeCheckpointDb, closeFileStorageDb, closeLivenessDb]) { try { await close(db, 1); } catch (error) { errors.push(error); } } + try { await db.$client.end({ timeout: 1 }); } catch (error) { errors.push(error); } + } + try { network.assertQuiet(); } catch (error) { errors.push(error); } + network.restore(); vi.restoreAllMocks(); vi.unstubAllEnvs(); + if (directory) { try { await rm(directory, { recursive: true, force: true }); } catch (error) { errors.push(error); } } + } + if (primary && errors.length) throw new AggregateError([primary, ...errors], "Slack form journey body and cleanup failed"); + if (primary) throw primary; + if (errors.length) throw new AggregateError(errors, "Slack form journey cleanup failed"); +}, 360_000); diff --git a/server/src/__tests__/slack-form-publication-readiness.test.ts b/server/src/__tests__/slack-form-publication-readiness.test.ts new file mode 100644 index 00000000000..dcf0fd3bc2e --- /dev/null +++ b/server/src/__tests__/slack-form-publication-readiness.test.ts @@ -0,0 +1,116 @@ +import { describe, expect, it, vi } from "vitest"; +import { chatActions, chatEndpoints, chatPublications, toolConnections, type Db } from "@paperclipai/db"; +import { createSlackFormPublicationStage } from "../services/chat-channels/slack/form-publication.js"; + +const fixture = vi.hoisted(() => ({ interactions: [] as unknown[] })); +vi.mock("../services/issue-thread-interactions.js", () => ({ + issueThreadInteractionService: () => ({ listForIssue: async () => fixture.interactions }), +})); +const binding = { version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", connectionId: "connection", + publicId: "p".repeat(43), runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", + eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }; +const { credentialFingerprint: _fingerprint, eventBindingId, ...identity } = binding; +const enrollment = { ...identity, bindingId: eventBindingId }; +const scheduledIntent = { version: 1, id: "intent", active: true, subject: "actor", instanceId: "instance", + companyId: "company", endpointId: "endpoint", connectionId: "connection", publicId: binding.publicId, + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", attempts: 0, nextAttemptAt: "2030-01-01T00:00:00Z" }; +const confirmedSetup = { runtimeGeneration: 1, slackCloudInstanceId: "instance", slackCloudSubject: "actor", + slackCloudEvents: { version: 1, runtimeGeneration: 1, bindingId: eventBindingId, teamId: "TTEST", appId: "ATEST" } }; +function currentOwner() { + return { id: "owner", status: "issued", payload: { enrollment, binding, subject: "actor", environment: "test" }, + result: { phase: "enrolled", nextAttemptAt: null as string | null, desiredActive: true } }; +} +function inputs(setup: Record, owners: unknown[] = []) { + const endpoint: typeof chatEndpoints.$inferSelect = { + id: "endpoint", companyId: "company", connectionId: "connection", provider: "slack", + setup: { step: "provider_setup", ...setup }, status: "active", publicId: binding.publicId, providerAccountId: "TTEST", + assignedAgentId: "agent", sponsorUserId: "actor", publicationMode: "automatic", externalExecutionPolicy: "restricted", + deploymentMode: "direct", providerAccountLabel: null, botExternalId: null, botUsername: null, botDisplayName: null, + botAvatarUrl: null, communicationInstructions: "", executionDefaults: null, allowDirectMessages: true, + requireAtMention: false, allowGroupChats: false, allowUnlinkedPeople: false, concurrencyPolicy: "queue", + capabilities: { threads: true, directMessages: true, nativeStreaming: true, messageEdits: true, messageDeletes: true, + reactions: true, files: true, cards: true, actions: true, modals: true, slashCommands: true, + ephemeralMessages: true, proactiveDirectMessages: false }, + healthMessage: null, lastEventAt: null, lastPublicationAt: null, lastError: null, activatedAt: null, archivedAt: null, + createdAt: new Date(0), updatedAt: new Date(0), + }; + const publication = { id: "publication", conversationId: "conversation", companyId: endpoint.companyId, + endpointId: endpoint.id, issueId: "issue", payload: { interactionId: "interaction" } } as typeof chatPublications.$inferSelect; + const connection = { id: "connection", companyId: "company", status: "active", enabled: true }; + fixture.interactions = [{ id: "interaction", companyId: "company", issueId: "issue", status: "pending", kind: "ask_user_questions", + payload: { version: 1, title: "Choose", questions: ["first", "second"].map(id => ({ id, prompt: "Choose", + selectionMode: "single", options: [{ id: "a", label: "A" }] })) } }]; + const select = vi.fn((columns?: unknown) => ({ from: (table: unknown) => ({ where: () => ({ limit: async () => + table === chatEndpoints ? [endpoint] : table === toolConnections ? [connection] + : table === chatActions && columns ? [{ id: "open" }] : owners, + }) }) })); + const db = { select } as unknown as Db; + return { endpoint, publication, connection, select, prepare: createSlackFormPublicationStage(db) }; +} +describe("Slack form publication enrollment readiness", () => { + it("waits for the first managed enrollment before a forms binding exists", async () => { + const f = inputs({ slackCloudIntent: scheduledIntent, runtimeGeneration: 1 }); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toEqual({ kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2000 }); + expect(f.select).toHaveBeenCalledTimes(3); + }); + it("preserves local Slack publication when no managed Cloud enrollment is configured", async () => { + const f = inputs({ step: "ready" }); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toBeNull(); + expect(f.select).not.toHaveBeenCalled(); + }); + it("does not wait for unsupported or no longer pending interactions", async () => { + const f = inputs({ slackCloudIntent: scheduledIntent, runtimeGeneration: 1 }); + fixture.interactions = []; + await expect(f.prepare(f.endpoint, f.publication)).resolves.toBeNull(); + expect(f.select).not.toHaveBeenCalled(); + }); + it("retains fail-closed parsing for a present malformed binding", async () => { + const f = inputs({ ...confirmedSetup, slackCloudForms: { version: 1 } }); + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "invalid_form_receipt" }); + }); + it("waits for an exact current scheduled forms owner, including a retained older binding", async () => { + const owner = { ...currentOwner(), payload: { enrollment, subject: "actor", environment: "test" }, + result: { phase: "staged", nextAttemptAt: "2030-01-01T00:00:00Z", desiredActive: true } }; + for (const retained of [undefined, { ...binding, runtimeGeneration: 2 }]) { + const f = inputs({ ...confirmedSetup, ...(retained ? { slackCloudForms: retained } : {}) }, [owner]); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toMatchObject({ kind: "wait" }); + } + }); + it("returns the commit hook only for the exact current enrolled owner", async () => { + const f = inputs({ ...confirmedSetup, slackCloudForms: binding }, [currentOwner()]); + expect(typeof await f.prepare(f.endpoint, f.publication)).toBe("function"); + }); + it("preserves an attention endpoint's exact usable enrolled owner without allowing attention waits", async () => { + const ready = inputs({ ...confirmedSetup, slackCloudForms: binding }, [currentOwner()]); + ready.endpoint.status = "attention"; + expect(typeof await ready.prepare(ready.endpoint, ready.publication)).toBe("function"); + const pending = inputs({ ...confirmedSetup }, [{ ...currentOwner(), payload: { enrollment, subject: "actor", environment: "test" }, + result: { phase: "staged", nextAttemptAt: "2030-01-01T00:00:00Z", desiredActive: true } }]); + pending.endpoint.status = "attention"; + await expect(pending.prepare(pending.endpoint, pending.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); + it.each(["attention", "retired", "transferred", "stopped_deadline", "missing", "ambiguous", "stale_binding", "foreign_subject"] as const)( + "does not grant budget-free polling to %s forms responsibility", async fault => { + const owner = currentOwner(); + if (fault === "attention") { owner.status = "attention"; owner.result.phase = "staged"; owner.result.nextAttemptAt = null; } + if (fault === "retired") owner.result.desiredActive = false; + if (fault === "transferred") owner.result.phase = "transferred"; + if (fault === "stopped_deadline") { owner.result.phase = "staged"; owner.result.nextAttemptAt = null; } + if (fault === "foreign_subject") owner.payload.subject = "other"; + const f = inputs({ ...confirmedSetup, slackCloudForms: fault === "stale_binding" ? { ...binding, runtimeGeneration: 2 } : binding }, + fault === "missing" ? [] : fault === "ambiguous" ? [owner, { ...owner, id: "duplicate" }] : [owner]); + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); + it.each(["attention", "inactive", "generation", "company", "retired", "connection"] as const)( + "does not grant budget-free polling to a %s events intent", async fault => { + const intent = { ...scheduledIntent, nextAttemptAt: scheduledIntent.nextAttemptAt as string | null }; + if (fault === "attention") intent.nextAttemptAt = null; + if (fault === "inactive") intent.active = false; + if (fault === "generation") intent.runtimeGeneration = 2; + if (fault === "company") intent.companyId = "foreign"; + const f = inputs({ slackCloudInstanceId: "instance", runtimeGeneration: 1, slackCloudIntent: intent, + ...(fault === "retired" ? { slackCloudRetiredGeneration: 1 } : {}) }); + if (fault === "connection") f.connection.enabled = false; + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); +}); diff --git a/server/src/__tests__/slack-form-retirement-work.test.ts b/server/src/__tests__/slack-form-retirement-work.test.ts new file mode 100644 index 00000000000..2ef3b751586 --- /dev/null +++ b/server/src/__tests__/slack-form-retirement-work.test.ts @@ -0,0 +1,510 @@ +import { readSlackIdleIncomingProof } from "../services/chat-channels/slack/idle-incoming.js"; +import { idleChatHistoryChecksForOwner } from "../services/chat-channels/idle-history-proof.js"; +import { registerConnectionEventWake, hasConnectionEventWakeReceiver } from "../services/connection-event-wake.js"; +import { slackConfiguredBindingsFingerprint } from "../services/chat-channels/slack/configured-coverage.js"; +import { verifyCloudControlAssertion, CLOUD_CONTROL_AUDIENCE, CLOUD_CONTROL_JWS_TYPE } from "../services/cloud-runtime-identity.js"; +import type { VerifiedIdleCapabilities } from "../services/idle-incoming-wake-capability.js"; +import { retainSlackFormOpeningProof } from "../services/chat-channels/slack/form-opening-proof.js"; +import { readSlackFormRetainedConsumer } from "../services/chat-channels/slack/cloud-consumer.js"; +import { parseSlackFormAckWork, transferSlackFormAckReceipt } from "../services/chat-channels/slack/form-ack-work.js"; +import { parseSlackFormFinalizedReceipt, slackFormFinalizedReceiptDigest } from "../services/chat-channels/slack/ack-receipt.js"; +import { createPaperclipCloudConnector } from "../services/paperclip-cloud-connector.js"; +import { githubConnectionEventService } from "../services/github-connection-events.js"; +import { createHash, createHmac, generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { and, eq, sql } from "drizzle-orm"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { activityLog, agents, applyPendingMigrations, authUsers, chatActions, chatEndpoints, companies, companyMemberships, + toolApplications, toolConnections, createDb, closeCheckpointDb, closeFileStorageDb, closeLivenessDb } from "@paperclipai/db"; +import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; +import { stageSlackFormEnrollment, stageSlackFormOwnerConfiguration, reconcileSlackFormEnrollment, reconcilePendingSlackFormWork } from "../services/chat-channels/slack/form-owner.js"; +import { acceptSlackFormRetirementLocally, transferSlackFormRetirement } from "../services/chat-channels/slack/form-retirement-work.js"; +import { slackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "../services/chat-channels/slack/ack-retirement-receipt.js"; +import { parseSlackFormBinding } from "../services/chat-channels/slack/cloud-forms.js"; +import { retrySlackFormAcknowledgement } from "../services/chat-channels/slack/form-retry.js"; +import { connectionManagementService } from "../services/connection-management.js"; + +const external = resolveExternalTestDatabaseUrl("PAPERCLIP_SLACK_RETIREMENT_TEST_DATABASE_URL"); +if (process.env.SLACK_GATE_EXPECT_PRELOAD === "1" && !external) throw new Error("Guarded retirement SQL requires an explicit external database"); +(external ? describe : describe.skip)("Slack retirement original-custody SQL handoff", () => { + let db: ReturnType, companyId: string, endpointId: string, connectionId: string, applicationId: string, agentId: string, actorId: string, actionId: string; + const identity = { instanceId: "retirement-sql-instance", environment: "test" as const }; + let binding: ReturnType, compactJws: string; + const row = async () => (await db.select().from(chatActions).where(and(eq(chatActions.id, actionId), eq(chatActions.companyId, companyId))))[0]!; + const trigger = "slack_retirement_controlled_update"; + function signCurrentReceipt(kid = "retirement-sql-key") { + const pair = generateKeyPairSync("ed25519"); + vi.stubEnv("PAPERCLIP_CLOUD_STACK_ID", identity.instanceId); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] })); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const receipt = slackFormRetirementReceipt(binding, identity.environment, actorId); + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + return body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + } + + beforeAll(async () => { + if (!external) throw new Error("explicit retirement database absent"); + await applyPendingMigrations(external); + db = createDb(external, { maxConnections: 2, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); + }, 60000); + beforeEach(async () => { + companyId = randomUUID(); endpointId = randomUUID(); connectionId = randomUUID(); applicationId = randomUUID(); agentId = randomUUID(); actorId = `retirement-${randomUUID()}`; + const now = new Date(); + await db.insert(companies).values({ id: companyId, name: "Retirement custody", issuePrefix: `R${companyId.slice(0, 7)}` }); + await db.insert(authUsers).values({ id: actorId, name: "Original owner", email: `${actorId}@example.test`, emailVerified: true, createdAt: now, updatedAt: now }); + await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: actorId, membershipRole: "operator", status: "active" }); + await db.insert(agents).values({ id: agentId, companyId, name: "Custody fixture", adapterType: "process", status: "idle", runtimeConfig: { heartbeat: { wakeOnDemand: false, intervalSec: 0 } } }); + await db.insert(toolApplications).values({ id: applicationId, companyId, name: "Slack custody", type: "chat" }); + await db.insert(toolConnections).values({ id: connectionId, companyId, applicationId, name: "Slack custody", uid: `slack-${connectionId}`, + connectionPurpose: "channel", transport: "chat_sdk", status: "active", enabled: true, createdByUserId: actorId, config: { provider: "slack" } }); + binding = parseSlackFormBinding({ version: 1, instanceId: identity.instanceId, companyId, endpointId, connectionId, publicId: "p".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); + const { credentialFingerprint, eventBindingId, ...base } = binding, enrollment = { ...base, bindingId: eventBindingId }; + await db.insert(chatEndpoints).values({ id: endpointId, companyId, connectionId, assignedAgentId: agentId, provider: "slack", publicId: binding.publicId, + sponsorUserId: actorId, status: "active", allowUnlinkedPeople: false, setup: { step: "provider_setup" } }); + await db.update(chatEndpoints).set({ setup: sql`${JSON.stringify({ step: "provider_setup", runtimeGeneration: 1, slackCloudInstanceId: identity.instanceId, + slackCloudSubject: actorId, slackCloudEvents: { runtimeGeneration: 1, bindingId: eventBindingId, teamId: binding.teamId, appId: binding.appId } })}::jsonb` }).where(eq(chatEndpoints.id, endpointId)); + actionId = await db.transaction(tx => stageSlackFormEnrollment(tx, { enrollment, subject: actorId, environment: identity.environment })); + await reconcileSlackFormEnrollment(db, { actionId, identity, resolveBotToken: async () => "xoxb-controlled-test-only", protocol: async input => { + expect(input.operation.kind).toBe("enroll"); expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); return { binding }; + } }); + expect((await row()).result?.phase).toBe("enrolled"); + await db.transaction(async tx => { + await tx.update(chatEndpoints).set({ status: "paused" }).where(and(eq(chatEndpoints.id, endpointId), eq(chatEndpoints.companyId, companyId))); + await stageSlackFormOwnerConfiguration(tx, endpointId, false, actorId, identity); + }); + compactJws = signCurrentReceipt(); + }); + afterEach(async () => { + const errors: unknown[] = []; + for (const cleanup of [ + () => db.execute(sql.raw(`DROP TRIGGER IF EXISTS ${trigger} ON chat_actions`)), + () => db.execute(sql.raw(`DROP FUNCTION IF EXISTS ${trigger}()`)), + () => db.delete(chatActions).where(eq(chatActions.companyId, companyId)), + () => db.delete(chatEndpoints).where(eq(chatEndpoints.companyId, companyId)), + () => db.delete(toolConnections).where(eq(toolConnections.companyId, companyId)), + () => db.delete(toolApplications).where(eq(toolApplications.companyId, companyId)), + () => db.delete(agents).where(eq(agents.companyId, companyId)), + () => db.delete(activityLog).where(eq(activityLog.companyId, companyId)), + () => db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)), + () => db.delete(companies).where(eq(companies.id, companyId)), + () => db.delete(authUsers).where(eq(authUsers.id, actorId)), + ]) try { await cleanup(); } catch (error) { errors.push(error); } + vi.unstubAllEnvs(); + if (errors.length) throw new AggregateError(errors, "retirement SQL cleanup failure"); + }); + afterAll(async () => { + const closed = await Promise.allSettled([closeCheckpointDb(db, 1), closeFileStorageDb(db, 1), closeLivenessDb(db, 1)]); + await db?.$client.end({ timeout: 1 }); + const errors = closed.flatMap(value => value.status === "rejected" ? [value.reason] : []); + if (errors.length) throw new AggregateError(errors, "retirement auxiliary closure failure"); + }); + function signedRetainedHistoryCapabilities(): VerifiedIdleCapabilities { + const key=generateKeyPairSync("ed25519"),kid="retained-history-current",now=Math.floor(Date.now()/1000),generation="e".repeat(64); + const capabilities={incomingWake:{version:1,ownerId:randomUUID(),generation,routes:["slack-broker-events-v1"], + slackBroker:{version:1,instanceId:identity.instanceId,generation,bindingsFingerprint:slackConfiguredBindingsFingerprint([]),entries:[]}}}; + const encode=(value:unknown)=>Buffer.from(JSON.stringify(value)).toString("base64url"); + const body=encode({alg:"EdDSA",typ:CLOUD_CONTROL_JWS_TYPE,kid})+"."+encode({v:1,iss:"paperclip-cloud",aud:CLOUD_CONTROL_AUDIENCE, + sub:identity.instanceId,action:"task-drain:read",requestId:randomUUID(),iat:now,exp:now+60,idleCapabilities:capabilities}); + const claims=verifyCloudControlAssertion({compactJws:body+"."+sign(null,Buffer.from(body),key.privateKey).toString("base64url"),expectedAction:"task-drain:read", + env:{PAPERCLIP_CLOUD_STACK_ID:identity.instanceId,PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS:JSON.stringify({keys:[{...key.publicKey.export({format:"jwk"}),kid,use:"sig",alg:"EdDSA"}]})}}); + if(!claims.idleCapabilities)throw new Error("Actual signed retained-history capability absent"); + return claims.idleCapabilities as VerifiedIdleCapabilities; + } + async function finishedHistory(ownerAlreadyTransferred=false) { + // The original owner reaches its real normal transferred state. Additional + // rows explicitly represent already-committed historical custody, not live + // provider calls or a fabricated provider result in this tested journey. + if(!ownerAlreadyTransferred)await acceptSlackFormRetirementLocally(db,await row(),identity,compactJws); + const receipt=slackFormRetirementReceipt(binding,identity.environment,actorId); + const protocol=async(input:{operation:Record})=>input.operation.refreshRetirementReceipt?{retirementReceiptJws:compactJws} + :input.operation.acceptedRetirementReceipt?{acceptedRetirementDigest:receipt.receiptId}:{releasedRetirementDigest:receipt.receiptId}; + if(!ownerAlreadyTransferred){await transferSlackFormRetirement(db,await row(),identity,protocol);await transferSlackFormRetirement(db,await row(),identity,protocol);} + expect(slackFormRetirementHistoryReceipt(await row())).not.toBeNull(); + const history=Array.from({length:1005},(_,index)=>({id:`b1000000-0000-4000-8000-${String(index).padStart(12,"0")}`,companyId,endpointId, + kind:"slack_form_grant",providerActionId:"slack-form-grant-history:"+index,status:"cancelled", + payload:{binding,environment:identity.environment,subject:actorId},result:{phase:"transferred",nextAttemptAt:null, + retirement:{version:1,binding,instanceId:identity.instanceId,environment:identity.environment,subject:actorId,receiptDigest:receipt.receiptId,receipt}}})); + for(const entry of history)expect(slackFormRetirementHistoryReceipt(entry)).not.toBeNull(); + await db.insert(chatActions).values(history); + return{history,receipt}; + } + async function inspectHistory(abortBeforeRead?:number) { + const capabilities=signedRetainedHistoryCapabilities(),unregister=registerConnectionEventWake(db,()=>{}); + let reads=0,selects=0; + try { + return await db.transaction(async tx=>{ + const select=vi.spyOn(tx,"select"); + const started=performance.now(),remaining=()=>{ + const left=2000-(performance.now()-started);if(left<=0)throw new Error("Incoming wake inspection budget exhausted");return Math.max(1,Math.floor(left)); + }; + const beforeRead=async()=>{ + reads++;if(reads===abortBeforeRead)throw new Error("Controlled existing inspection budget exhausted before next page"); + if(reads*2>64)throw new Error("Incoming wake query budget exhausted"); + await tx.execute(sql`SELECT set_config('statement_timeout', ${`${remaining()}ms`}, true)`);remaining(); + }; + try { + const proof=await readSlackIdleIncomingProof(tx,{capabilities,identity,receiverBound:hasConnectionEventWakeReceiver(db),registeredOptions:()=>false,beforeRead,remaining}); + const check=idleChatHistoryChecksForOwner(proof?.completedActionIds).find(value=>value.reason==="chat_actions"); + if(!check)throw new Error("Actual chat action owner query absent"); + await beforeRead();const query=typeof check.query==="string"?sql.raw(check.query):check.query; + const blockers=await tx.execute<{count:number}>(sql`SELECT count(*)::int AS count FROM (${query}) retained_history_blockers`); + return{proof,blockers:blockers[0]!.count,reads}; + }finally{selects=select.mock.calls.length;select.mockRestore();} + }); + }finally{unregister();if(abortBeforeRead!==undefined){expect(reads).toBe(abortBeforeRead);expect(selects).toBe(abortBeforeRead-1);}} + } + it("more than one page of valid finished Slack custody remains provable by the real incoming and action owners",async()=>{ + const{history}=await finishedHistory();const result=await inspectHistory(); + expect(result.proof?.completedActionIds).toEqual([actionId,...history.map(entry=>entry.id)].sort()); + expect(result.proof?.endpointIds).toEqual([]);expect(result.proof?.connectionIds).toEqual([]); + expect(result.blockers).toBe(0);expect(result.reads).toBeGreaterThanOrEqual(5); + },15000); + for(const fault of ["unfinished","malformed","foreign_company"] as const)it(`a ${fault} receipt after the first completed history page still blocks the actual action owner`,async()=>{ + const{history,receipt}=await finishedHistory(),id="ffffffff-ffff-4fff-bfff-ffffffffffff",last=history.at(-1)!; + const payload=fault==="foreign_company"?{...last.payload,binding:{...binding,companyId:randomUUID()}}:last.payload; + const result=fault==="unfinished"?{phase:"issued",nextAttemptAt:null}:fault==="malformed" + ?{...last.result,retirement:{...last.result.retirement,receipt:{...receipt,receiptId:"not-a-retirement-digest"}}}:last.result; + await db.insert(chatActions).values({...last,id,providerActionId:"slack-form-grant-history:last",status:fault==="unfinished"?"issued":last.status,payload,result}); + const inspected=await inspectHistory();expect(inspected.proof?.completedActionIds).toHaveLength(1006); + expect(inspected.proof?.completedActionIds).not.toContain(id);expect(inspected.blockers).toBe(1); + },15000); + it("a deadline before the next history page refuses proof without issuing that read",async()=>{ + await finishedHistory();await expect(inspectHistory(4)).rejects.toThrow("Controlled existing inspection budget exhausted before next page"); + },15000); + + it("a finished session on a later page certifies only its exact earlier open and submit actions",async()=>{ + const ack=await acceptedAckForRefresh("ffffffff-ffff-4fff-bfff-fffffffffffe"); + const readAck=async()=>(await db.select().from(chatActions).where(eq(chatActions.id,ack.id)))[0]!; + const work=parseSlackFormAckWork((await readAck()).result?.formAck); + if(!work.receipt)throw new Error("Normal retained ACK fixture lost its receipt"); + const digest=slackFormFinalizedReceiptDigest(work.receipt); + const protocol=async(input:{operation:Record})=>input.operation.refreshAckReceipt?{ackReceiptJws:ack.currentJws} + :input.operation.acceptedAckReceipt?{acceptedReceiptDigest:digest,needsTerminalOriginal:false}:{releasedAckDigest:digest}; + await transferSlackFormAckReceipt(db,{companyId,actionId:ack.id,instanceId:identity.instanceId,protocol}); + await transferSlackFormAckReceipt(db,{companyId,actionId:ack.id,instanceId:identity.instanceId,protocol}); + const finished=await readAck();expect(parseSlackFormAckWork(finished.result?.formAck).phase).toBe("transferred"); + const openId="00000000-0000-4000-8000-000000000001",submitId="00000000-0000-4000-8000-000000000002"; + const payload={interactionId:finished.payload.interactionId,publicationId:finished.payload.publicationId}; + await db.insert(chatActions).values([ + {id:openId,companyId,endpointId,kind:"question_form_open",status:"processed",providerActionId:"slack-history-linked-open",payload}, + {id:submitId,companyId,endpointId,kind:"question_form_submit",status:"processed",providerActionId:"slack-history-linked-submit",payload}]); + await db.update(chatActions).set({payload:{...finished.payload,openActionRowId:openId,submitActionRowId:submitId}}).where(eq(chatActions.id,ack.id)); + await finishedHistory(true); + const proof=await inspectHistory();expect(proof.proof?.completedActionIds).toHaveLength(1009);expect(proof.blockers).toBe(0); + expect(proof.proof?.completedActionIds).toEqual(expect.arrayContaining([ack.id,openId,submitId])); + await db.update(chatActions).set({payload:{...payload,publicationId:"changed-retained-publication"}}).where(eq(chatActions.id,openId)); + const changed=await inspectHistory();expect(changed.proof?.completedActionIds).not.toContain(openId);expect(changed.blockers).toBe(1); + },15000); + + // Seed retained accepted ACK custody, as in the existing GC controls; this is not a provider-delivery proof. + async function acceptedAckForRefresh(appReceiptId=randomUUID()) { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retire = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: digest } : { releasedRetirementDigest: digest }; + await transferSlackFormRetirement(db, await row(), identity, retire); + await transferSlackFormRetirement(db, await row(), identity, retire); + const id = appReceiptId, operationId = "e".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const originalBody = JSON.stringify({ type: "block_actions", user: { id: "UTEST" } }); + const timestamp = String(Math.floor(Date.now() / 1000)); + const original = { rawBody: Buffer.from(originalBody).toString("base64"), timestamp, + signature: `v0=${createHmac("sha256", "controlled-signing-secret").update(`v0:${timestamp}:${originalBody}`).digest("hex")}` }; + const requestDigest = createHash("sha256").update(JSON.stringify([original.rawBody, original.timestamp, original.signature])).digest("hex"); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: id, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const signAck = (kid: string) => { + const pair = generateKeyPairSync("ed25519"), encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] })); + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + return body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + }; + const oldJws = signAck("ack-before-refresh"); + await db.insert(chatActions).values({ id, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, requestDigest, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent: { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: id, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: id, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "accepted", leaseId: "exact-refresh-lease", deliveryId, receiptJws: oldJws, receipt }, nextAttemptAt: now } }); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "9".repeat(64)); + await db.transaction(tx => retainSlackFormOpeningProof(tx, { companyId, appReceiptId: id, original, + operation: { version: 1, binding, grantId: "grant", id: operationId, externalId: "controlled-external-id", deliveryId, requestDigest, triggerDigest: "f".repeat(64), + userId: "UTEST", acceptedAt: now, triggerDeadline: now, phase: "rejected", view: null, closed: null, acknowledgement: null, rejection: null } })); + return { id, oldJws, currentJws: signAck("ack-after-refresh"), original }; + } + for (const kind of ["retirement", "ack"] as const) for (const boundary of ["before_refresh", "after_refresh", "after_concurrent_write"] as const) { + it(`${kind} owner failure ${boundary} retains exact custody and durable retry without overwriting another writer`, async () => { + let id: string, oldJws: string, currentJws: string, original: { rawBody: string; timestamp: string; signature: string } | undefined; + if (kind === "retirement") { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + id = actionId; oldJws = compactJws; currentJws = signCurrentReceipt("retirement-after-refresh"); + } else ({ id, oldJws, currentJws, original } = await acceptedAckForRefresh()); + const read = async () => (await db.select().from(chatActions).where(and(eq(chatActions.id, id), eq(chatActions.companyId, companyId))))[0]!; + const before = await read(), reported: string[] = []; + let refreshes = 0, transfers = 0, concurrent: typeof before | undefined; + const owner = { identity, resolveBotToken: async () => { throw new Error("retained work must not read credentials"); }, + report: (code: string) => { reported.push(code); }, protocol: async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt || input.operation.refreshAckReceipt) { + refreshes++; + if (boundary === "before_refresh") throw new Error("controlled failure before refresh"); + return kind === "retirement" ? { retirementReceiptJws: currentJws } : { ackReceiptJws: currentJws }; + } + transfers++; + expect(input.operation[kind === "retirement" ? "acceptedRetirementReceipt" : "acceptedAckReceipt"]).toBe(currentJws); + if (kind === "ack") expect(input.operation.originalOpening).toEqual(original); + if (boundary === "after_concurrent_write") { + const refreshed = await read(); + await db.update(chatActions).set({ status: "attention", payload: { ...refreshed.payload, protectedConcurrentWriter: true }, + result: { ...refreshed.result, nextAttemptAt: null, protectedConcurrentWriter: true }, updatedAt: new Date() }).where(eq(chatActions.id, id)); + concurrent = await read(); + } + throw new Error("controlled transfer failure after exact signature refresh"); + } }; + const started = Date.now(), due = await reconcilePendingSlackFormWork(db, owner), after = await read(); + expect(refreshes).toBe(1); expect(transfers).toBe(boundary === "before_refresh" ? 0 : 1); + expect(reported).toEqual(["form_owner_broker_unavailable"]); + if (boundary === "after_concurrent_write") { + expect(concurrent).toBeDefined(); expect(after).toEqual(concurrent); expect(due).toBeNull(); + } else { + expect(after.payload).toEqual(before.payload); expect(after.status).toBe(before.status); + const result = kind === "retirement" ? { ...before.result, retirementReceiptJws: boundary === "before_refresh" ? oldJws : currentJws } + : { ...before.result, formAck: { ...parseSlackFormAckWork(before.result?.formAck), receiptJws: boundary === "before_refresh" ? oldJws : currentJws } }; + expect(after.result).toEqual({ ...result, attempts: 1, failureCode: "form_owner_broker_unavailable", failureReason: null, nextAttemptAt: expect.any(String) }); + expect(due).toBe(Date.parse(String(after.result?.nextAttemptAt))); expect(due).toBeGreaterThanOrEqual(started + 5000); + if (due === null) throw new Error("Normal owner lost durable backoff"); + expect(await reconcilePendingSlackFormWork(db, owner, new Date(due - 1))).toBe(due); + expect(refreshes).toBe(1); expect(await read()).toEqual(after); + } + }, 15000); + } + it("commits original-subject acceptance before Cloud compaction and retries a lost transfer after reconstruction", async () => { + let retireCalls = 0, cleanupCalls = 0; + await reconcileSlackFormEnrollment(db, { actionId, identity, resolveBotToken: async () => { throw new Error("retirement must not read credentials"); }, protocol: async input => { + expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); + if (input.operation.kind === "retire") { retireCalls++; return { retired: true }; } + cleanupCalls++; return { cleaned: true, instanceId: identity.instanceId, environment: identity.environment, binding, retirementReceiptJws: compactJws }; + } }); + const accepted = await row(); expect(accepted.result?.phase).toBe("retirement_accepted"); expect(accepted.result?.retirementReceiptJws).toBe(compactJws); + let compacted = false, transfers = 0; const lost = new Error("lost actual transfer response"); + const protocol = async (input: { subject: string; companyId: string; operation: Record }) => { + expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: compactJws }; + if (input.operation.releaseRetirementFact) return { releasedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + expect(input.operation).toEqual({ kind: "cleanup", binding, acceptedRetirementReceipt: compactJws }); + transfers++; compacted = true; if (transfers === 1) throw lost; + return { acceptedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + }; + const reported: string[] = [], owner = { identity, protocol, resolveBotToken: async () => { throw new Error("no new use"); }, report: (code: string) => { reported.push(code); } }; + const started = Date.now(), due = await reconcilePendingSlackFormWork(db, owner); + const pending = await row(); expect(compacted).toBe(true); expect(pending.payload).toEqual(accepted.payload); + expect(pending.result?.phase).toBe("retirement_accepted"); expect(pending.result?.retirementReceiptJws).toBe(compactJws); + expect(pending.result?.attempts).toBe(1); expect(reported).toEqual(["form_owner_broker_unavailable"]); + expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); expect(due).toBeGreaterThanOrEqual(started + 5000); + if (due === null) throw new Error("normal owner did not retain retry deadline"); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + expect(await reconcilePendingSlackFormWork(db, owner)).not.toBeNull(); + expect((await row()).result?.phase).toBe("retirement_gc_pending"); + expect(await reconcilePendingSlackFormWork(db, owner)).toBeNull(); + const complete = await row(); expect(slackFormRetirementHistoryReceipt(complete)?.subject).toBe(actorId); + expect(complete.result?.retirementReceiptJws).toBeUndefined(); expect(retireCalls).toBe(1); expect(cleanupCalls).toBe(1); expect(transfers).toBe(2); + }, 15000); + it("a lost cleanup reply before App acceptance keeps original debt and the normal owner retries it", async () => { + const original = await row(), lost = new Error("cleanup reply lost before local acceptance"); + let cleanups = 0; + const owner = { identity, resolveBotToken: async () => { throw new Error("no credential reads during retirement"); }, report: () => {}, protocol: async (input: { operation: Record }) => { + if (input.operation.kind === "retire") return { retired: true }; + cleanups++; if (cleanups === 1) throw lost; + return { cleaned: true, instanceId: identity.instanceId, environment: identity.environment, binding, retirementReceiptJws: compactJws }; + } }; + const due = await reconcilePendingSlackFormWork(db, owner), pending = await row(); + expect(pending.payload).toEqual(original.payload); expect(pending.result?.phase).toBe("retired_cleanup"); expect(pending.result?.retirementReceiptJws).toBeUndefined(); + expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); if (due === null) throw new Error("cleanup lost its retry deadline"); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + await reconcilePendingSlackFormWork(db, owner); expect((await row()).result?.phase).toBe("retirement_accepted"); expect(cleanups).toBe(2); + }, 15000); + it("acceptance SQL rollback preserves old proof debt and a stale snapshot cannot clear a successor", async () => { + const before = await row(); expect(actionId).toMatch(/^[a-f0-9-]+$/); + await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF NEW.id='${actionId}' AND NEW.result->>'phase'='retirement_accepted' THEN RAISE EXCEPTION 'controlled retirement acceptance rollback'; END IF; RETURN NEW; END $$`)); + await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON chat_actions FOR EACH ROW EXECUTE FUNCTION ${trigger}()`)); + await expect(acceptSlackFormRetirementLocally(db, before, identity, compactJws)).rejects.toThrow(); expect(await row()).toEqual(before); + await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON chat_actions`)); await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`)); + await db.update(chatActions).set({ result: { ...before.result, preservedSuccessor: "exact" } }).where(eq(chatActions.id, actionId)); + const successor = await row(); await expect(acceptSlackFormRetirementLocally(db, before, identity, compactJws)).rejects.toMatchObject({ code: "form_retirement_acceptance_changed" }); + expect(await row()).toEqual(successor); + }); + it("foreign company and a changed accepted digest cannot clear local responsibility", async () => { + const before = await row(); await expect(acceptSlackFormRetirementLocally(db, { ...before, companyId: randomUUID() }, identity, compactJws)).rejects.toMatchObject({ code: "form_retirement_authority_changed" }); + expect(await row()).toEqual(before); await acceptSlackFormRetirementLocally(db, before, identity, compactJws); const accepted = await row(); + await expect(transferSlackFormRetirement(db, accepted, identity, async input => input.operation.refreshRetirementReceipt ? { retirementReceiptJws: compactJws } : { acceptedRetirementDigest: "0".repeat(64) })).rejects.toMatchObject({ code: "form_retirement_transfer_unconfirmed" }); + expect(await row()).toEqual(accepted); + }); + it("current company-authorized Retry rearms retained debt and revoked authority cannot", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + await db.update(chatActions).set({ status: "attention", result: { ...(await row()).result, failureCode: "form_retirement_verifier_unavailable", nextAttemptAt: null } }).where(eq(chatActions.id, actionId)); + const management = await connectionManagementService(db).authorize({ type: "board", userId: actorId, source: "session", ignoreInstanceAdmin: true }, "connections:configure", + { type: "connection_management", companyId, connectionId, purpose: "chat", operation: "configure", readOnly: false }); + const before = await row(); expect((await retrySlackFormAcknowledgement(db, { companyId, endpointId, actionId, identity: () => identity, management })).state).toBe("scheduled"); + const pending = await row(); expect(pending.payload).toEqual(before.payload); expect(pending.result?.retirementReceiptJws).toBe(compactJws); expect(pending.result?.nextAttemptAt).not.toBeNull(); + await db.delete(companyMemberships).where(and(eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, actorId))); + await expect(retrySlackFormAcknowledgement(db, { companyId, endpointId, actionId, identity: () => identity, management })).rejects.toThrow(); + expect(await row()).toEqual(pending); + }); + it("current signature refresh changes only the JWS and a successor CAS cannot be overwritten", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const accepted = await row(), currentJws = signCurrentReceipt("B"); + const protocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: currentJws } : { acceptedRetirementDigest: "0".repeat(64) }; + await expect(transferSlackFormRetirement(db, accepted, identity, protocol)).rejects.toMatchObject({ code: "form_retirement_transfer_unconfirmed" }); + const refreshed = await row(); expect(refreshed.payload).toEqual(accepted.payload); expect(refreshed.status).toBe(accepted.status); + expect(refreshed.result).toEqual({ ...accepted.result, retirementReceiptJws: currentJws }); + const nextJws = signCurrentReceipt("C"); let transfers = 0; + await db.update(chatActions).set({ result: { ...refreshed.result, protectedSuccessor: true } }).where(eq(chatActions.id, actionId)); + const successor = await row(); + await expect(transferSlackFormRetirement(db, refreshed, identity, async input => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: nextJws }; + transfers++; return { acceptedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + })).rejects.toMatchObject({ code: "form_signature_refresh_changed" }); + expect(transfers).toBe(0); expect(await row()).toEqual(successor); + }); + it("durable transfer tracks GC and retries a lost release reply without the old signing key", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); const currentJws = signCurrentReceipt("B"); + const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + let releases = 0; const lost = new Error("lost canonical fact release response"); + const protocol = async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: currentJws }; + if (input.operation.acceptedRetirementReceipt) return { acceptedRetirementDigest: digest }; + expect(input.operation).toEqual({ kind: "cleanup", binding, releaseRetirementFact: { receiptId: digest } }); + releases++; if (releases === 1) throw lost; return { releasedRetirementDigest: digest }; + }; + await transferSlackFormRetirement(db, await row(), identity, protocol); + const transferred = await row(); expect(transferred.result?.phase).toBe("retirement_gc_pending"); expect(transferred.result?.nextAttemptAt).not.toBeNull(); expect(releases).toBe(0); + const owner = { identity, protocol, resolveBotToken: async () => { throw new Error("GC has no new credential use"); }, report: () => {} }; + const due = await reconcilePendingSlackFormWork(db, owner); const pending = await row(); expect(pending.result?.phase).toBe("retirement_gc_pending"); + expect(pending.result?.attempts).toBe(1); expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); if (due === null) throw new Error("GC debt lost its retry deadline"); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", ""); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + expect(await reconcilePendingSlackFormWork(db, owner)).toBeNull(); expect((await row()).result?.phase).toBe("transferred"); expect(releases).toBe(2); + }, 15000); + it("transfer rollback cannot issue GC before the actual App SQL commit", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); const currentJws = signCurrentReceipt("B"); + expect(actionId).toMatch(/^[a-f0-9-]+$/); + await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF NEW.id='${actionId}' AND NEW.result->>'phase'='retirement_gc_pending' THEN RAISE EXCEPTION 'controlled transfer commit rollback'; END IF; RETURN NEW; END $$`)); + await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON chat_actions FOR EACH ROW EXECUTE FUNCTION ${trigger}()`)); + let transfers = 0, releases = 0; const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const protocol = async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: currentJws }; + if (input.operation.acceptedRetirementReceipt) { transfers++; return { acceptedRetirementDigest: digest }; } + releases++; return { releasedRetirementDigest: digest }; + }; + await expect(transferSlackFormRetirement(db, await row(), identity, protocol)).rejects.toThrow(); + expect(transfers).toBe(1); expect(releases).toBe(0); expect((await row()).result?.phase).toBe("retirement_accepted"); + await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON chat_actions`)); await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`)); + await transferSlackFormRetirement(db, await row(), identity, protocol); expect((await row()).result?.phase).toBe("retirement_gc_pending"); expect(releases).toBe(0); + await transferSlackFormRetirement(db, await row(), identity, protocol); expect((await row()).result?.phase).toBe("transferred"); expect(releases).toBe(1); + }); + + it("a real paused endpoint polls reconstructed GC debt but completed exact session history stops recurring empty leases", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const retirementDigest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retirementProtocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: retirementDigest } : { releasedRetirementDigest: retirementDigest }; + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + expect((await row()).result?.phase).toBe("transferred"); + expect((await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)))[0]?.status).toBe("paused"); + const sessionId = randomUUID(), operationId = "c".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: sessionId, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const pair = generateKeyPairSync("ed25519"), kid = "session-canonical-control"; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), pair.privateKey).toString("base64url"); + const terminalIntent = { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: sessionId, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }; + await db.insert(chatActions).values({ id: sessionId, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: sessionId, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "gc_pending", leaseId: "exact-session-lease", deliveryId, receiptJws, receipt }, nextAttemptAt: now } }); + const signing = generateKeyPairSync("ed25519"), sealing = generateKeyPairSync("x25519"); + const signPrivateKey = signing.privateKey.export({ format: "jwk" }).d, sealPrivateKey = sealing.privateKey.export({ format: "jwk" }).d; + if (!signPrivateKey || !sealPrivateKey) throw new Error("normal controlled connector private keys absent"); + const connector = createPaperclipCloudConnector({ config: { baseUrl: "https://broker.example.test", instanceId: identity.instanceId, + environment: identity.environment, signPrivateKey, sealPrivateKey }, request: async () => { throw new Error("Unexpected external connector request"); } }); + const lease = vi.spyOn(connector, "leaseEvents").mockResolvedValue(null); + expect((await readSlackFormRetainedConsumer(db, connector)).owner).toMatchObject({ companyId, requiresDelivery: false }); + const pendingWorker = githubConnectionEventService(db, { connector, env: { PAPERCLIP_GITHUB_EVENT_POLLING: "1" } }); + await pendingWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(1); expect(pendingWorker.nextPollAt()).not.toBeNull(); + let releases = 0; + await transferSlackFormAckReceipt(db, { companyId, actionId: sessionId, instanceId: identity.instanceId, protocol: async values => { + expect(values.subject).toBe("github-bot-events"); expect(values.companyId).toBe(companyId); + expect(values.operation).toEqual({ kind: "receipt", binding, operationId, deliveryId, + releaseAckFact: { leaseId: "exact-session-lease", receiptDigest: slackFormFinalizedReceiptDigest(receipt), appReceiptId: sessionId } }); + releases++; return { releasedAckDigest: slackFormFinalizedReceiptDigest(receipt) }; + } }); + const [completed] = await db.select().from(chatActions).where(and(eq(chatActions.id, sessionId), eq(chatActions.companyId, companyId))); + expect(completed?.result?.formAck).toMatchObject({ phase: "transferred", leaseId: "exact-session-lease", receipt }); + expect(completed?.result?.nextAttemptAt).toBeNull(); expect(releases).toBe(1); + expect(await readSlackFormRetainedConsumer(db, connector)).toEqual({ owner: null, attention: [] }); + const completedWorker = githubConnectionEventService(db, { connector, env: { PAPERCLIP_GITHUB_EVENT_POLLING: "1" } }); + await completedWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(2); expect(completedWorker.nextPollAt()).toBeNull(); + await completedWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(2); + }); + it("bounded retained-consumer pages pass more than32 completed rows to discover an unfinished tail without dropping attention", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const retirementDigest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retirementProtocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: retirementDigest } : { releasedRetirementDigest: retirementDigest }; + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + expect((await row()).result?.phase).toBe("transferred"); + expect((await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)))[0]?.status).toBe("paused"); + const sessionId = randomUUID(), operationId = "c".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: sessionId, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const pair = generateKeyPairSync("ed25519"), kid = "session-canonical-control"; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), pair.privateKey).toString("base64url"); + const terminalIntent = { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: sessionId, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }; + await db.insert(chatActions).values({ id: sessionId, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: sessionId, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "gc_pending", leaseId: "exact-session-lease", deliveryId, receiptJws, receipt }, nextAttemptAt: now } }); + await transferSlackFormAckReceipt(db, { companyId, actionId: sessionId, instanceId: identity.instanceId, + protocol: async () => ({ releasedAckDigest: slackFormFinalizedReceiptDigest(receipt) }) }); + const [completed] = await db.select().from(chatActions).where(and(eq(chatActions.id, sessionId), eq(chatActions.companyId, companyId))); + if (!completed) throw new Error("Actual GC transition lost its session"); + for (let n = 1; n <= 33; n++) { + const id = `00000000-0000-4000-8000-${n.toString(16).padStart(12, "0")}`, op = n.toString(16).padStart(64, "0"), delivery = `slack_form_${op}`; + const historyReceipt = { ...receipt, operationId: op, deliveryId: delivery, appReceiptId: id }; + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt: historyReceipt }); + const historyJws = body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + await db.insert(chatActions).values({ ...completed, id, providerActionId: `slack-form-session:${op}`, + payload: { ...completed.payload, operationId: op }, result: { phase: "not_opened", + terminalIntent: { ...terminalIntent, operationId: op, deliveryId: delivery, appReceiptId: id }, + cloudProofAcceptance: { version: 1, binding, operationId: op, appReceiptId: id, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "transferred", leaseId: "exact-session-lease", deliveryId: delivery, receiptJws: historyJws, receipt: historyReceipt }, nextAttemptAt: null } }); + } + const tailId = "ffffffff-ffff-4fff-8fff-ffffffffffff", tailOp = "f".repeat(64); + await db.insert(chatActions).values({ id: tailId, companyId, endpointId, kind: "slack_form_session", status: "processing", providerActionId: `slack-form-session:${tailOp}`, + payload: { binding, environment: identity.environment, operationId: tailOp, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "update_attempted", view: { id: "VTAIL", hash: "same-loading-hash", externalId: `pc-form-${tailOp}` } } }); + const connector = { instanceIdentity: () => identity }; + expect((await readSlackFormRetainedConsumer(db, connector)).owner).toEqual({ subject: "github-bot-events", companyId, binding, requiresDelivery: true }); + await db.insert(chatActions).values({ id: "00000000-0000-4000-8000-000000000000", companyId, endpointId, kind: "slack_form_session", status: "attention", providerActionId: "malformed-retained-page", + payload: { binding: { instanceId: identity.instanceId }, environment: identity.environment }, result: {} }); + const withMalformed = await readSlackFormRetainedConsumer(db, connector); + expect(withMalformed.owner).toMatchObject({ companyId, requiresDelivery: true }); + expect(withMalformed.attention).toContainEqual({ actionId: "00000000-0000-4000-8000-000000000000", code: "form_retained_owner_invalid" }); + expect(withMalformed.attention.length).toBeLessThanOrEqual(32); + }); + +}); diff --git a/server/src/http/body-limits.ts b/server/src/http/body-limits.ts index fde64e5daaa..44a7f6d2dbb 100644 --- a/server/src/http/body-limits.ts +++ b/server/src/http/body-limits.ts @@ -1,4 +1,5 @@ export const DEFAULT_JSON_BODY_LIMIT = "10mb"; +export const DEFAULT_JSON_BODY_LIMIT_BYTES = 10 * 1024 * 1024; export const CHAT_WEBHOOK_BODY_LIMIT = "1mb"; export const CHAT_WEBHOOK_BODY_LIMIT_BYTES = 1024 * 1024; export const PORTABLE_JSON_BODY_LIMIT = "64mb"; diff --git a/server/src/middleware/cloud-control.ts b/server/src/middleware/cloud-control.ts index 90a6ba00936..69d3edb290b 100644 --- a/server/src/middleware/cloud-control.ts +++ b/server/src/middleware/cloud-control.ts @@ -23,6 +23,7 @@ export function verifiedIdleCapabilities(req: Request): VerifiedIdleCapabilities const ACTIONS_BY_ENDPOINT: Record> = { "/api/instance/ephemeral-enrollment/prepare": { POST: "ephemeral-enrollment:prepare" }, "/api/instance/connector-events/wake": { POST: "connector-events:wake" }, + "/api/instance/task-drain/status": { POST: "task-drain:read-body" }, "/api/instance/task-drain": { GET: "task-drain:read", POST: "task-drain:start", @@ -64,7 +65,9 @@ export function cloudControlMiddleware(): RequestHandler { return; } try { - const claims = verifyCloudControlAssertion({ compactJws: assertion, expectedAction }); + const rawBody: unknown = Reflect.get(req, "rawBody"); + const claims = verifyCloudControlAssertion({ compactJws: assertion, expectedAction, + ...(Buffer.isBuffer(rawBody) ? { rawBody } : {}) }); controlClaims.set(req, claims); if (claims.idleCapabilities) idleCapabilities.set(req, claims.idleCapabilities as VerifiedIdleCapabilities); } catch (error) { diff --git a/server/src/middleware/cloud-tenant-actor.test.ts b/server/src/middleware/cloud-tenant-actor.test.ts index c0c13cc6c42..eff8f83b671 100644 --- a/server/src/middleware/cloud-tenant-actor.test.ts +++ b/server/src/middleware/cloud-tenant-actor.test.ts @@ -219,7 +219,7 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => { expect(insertedTables).toContain(companyMemberships); }); - it.each(["/api/health", "/api/instance/task-drain"])("counts unfinished authentication on exempt %s requests", async (path) => { + it.each(["/api/health", "/api/instance/task-drain", "/api/instance/task-drain/status"])("counts unfinished authentication on exempt %s requests", async (path) => { let enterWrite!: () => void, finishWrite!: () => void; const entered = new Promise(resolve => { enterWrite = resolve; }); const pending = new Promise(resolve => { finishWrite = resolve; }); @@ -235,8 +235,9 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => { res.json(await readIdleSleepSafety(emptyDb, () => ({ ...hold, draining: true, activeRuns: 0, pendingWakes: 0 }), Date.now, hold.ownerId, async () => "none")); }); + server.post("/api/instance/task-drain/status", (_req, res) => res.sendStatus(204)); trackIdleRequestHandlers(server); - const accepted = request(server).get(path).set(VALID_HEADERS).then(response => response); + const accepted = (path.endsWith("/status") ? request(server).post(path) : request(server).get(path)).set(VALID_HEADERS).then(response => response); try { await entered; expect(idleWorkSnapshot().active).toBeGreaterThan(0); diff --git a/server/src/middleware/idle-admission.ts b/server/src/middleware/idle-admission.ts index cbcf5cb7275..64fc2beb9a0 100644 --- a/server/src/middleware/idle-admission.ts +++ b/server/src/middleware/idle-admission.ts @@ -55,6 +55,7 @@ export function completeIdleReadOnlyResponse(req: Request, application: Applicat function isControlRequest(req: Request) { const path = req.path.replace(/\/$/, ""); return (path === "/api/instance/task-drain" && ["GET", "POST", "DELETE"].includes(req.method)) || + (path === "/api/instance/task-drain/status" && req.method === "POST") || (path === "/api/health" && ["GET", "HEAD"].includes(req.method)); } @@ -71,7 +72,8 @@ export const idleAdmissionMiddleware: RequestHandler = (req, res, next) => { res.set("Retry-After", "1").status(503).json({ error: "instance_preparing_to_sleep" }); return; } - if (controlRequest && req.method === "GET" && req.path.replace(/\/$/, "") === "/api/instance/task-drain") { + if (controlRequest && ((req.method === "GET" && req.path.replace(/\/$/, "") === "/api/instance/task-drain") + || (req.method === "POST" && req.path.replace(/\/$/, "") === "/api/instance/task-drain/status"))) { next(); return; } diff --git a/server/src/routes/chat-channels.ts b/server/src/routes/chat-channels.ts index caf44d8e435..d75e0f27acb 100644 --- a/server/src/routes/chat-channels.ts +++ b/server/src/routes/chat-channels.ts @@ -473,6 +473,11 @@ export function chatChannelRoutes(db: Db, options: ChatChannelRouteOptions) { ); }); + router.post("/chat-endpoints/:endpointId/provider-work/:actionId/retry", managementGuard("connections:configure", "POST /chat-endpoints/:endpointId/provider-work/:actionId/retry"), validate(z.object({}).strict()), async (req, res) => { + if (!(await assertEndpointManagementAccess(req, res))) return; + res.json(await service.retrySlackFormReceipt(endpointId(req), String(req.params.actionId), connectionRequestContext(req))); + }); + router.post("/chat-endpoints/:endpointId/test", managementGuard("connections:configure", "POST /chat-endpoints/:endpointId/test"), async (req, res) => { if (!(await assertEndpointManagementAccess(req, res))) return; res.json(await service.test(endpointId(req), undefined, connectionRequestContext(req))); diff --git a/server/src/routes/instance-settings.ts b/server/src/routes/instance-settings.ts index 3372b295792..260d507215f 100644 --- a/server/src/routes/instance-settings.ts +++ b/server/src/routes/instance-settings.ts @@ -1,7 +1,7 @@ import type { IdleDeadlineOwners } from "../services/idle-deadline-types.js"; -import { verifiedIdleCapabilities } from "../middleware/cloud-control.js"; +import { verifiedCloudControlClaims, verifiedIdleCapabilities } from "../middleware/cloud-control.js"; import { trackIdleWork } from "../services/task-admission.js"; -import { Router, type Request } from "express"; +import { Router, type Request, type RequestHandler } from "express"; import { companies, type Db } from "@paperclipai/db"; import { patchInstanceSettingsSchema, @@ -302,7 +302,7 @@ export function instanceSettingsRoutes( }, ); - router.get("/instance/task-drain", async (req, res) => { + const readTaskDrainStatus: RequestHandler = async (req, res) => { assertBoardOrgAccess(req); if (req.query.upgradeReadiness === "1") { // This covers all companies and must not expose instance-wide work to a member. @@ -348,6 +348,13 @@ export function instanceSettingsRoutes( return; } res.json(heartbeat.getTaskDrainStatus()); + }; + router.get("/instance/task-drain", readTaskDrainStatus); + router.post("/instance/task-drain/status", async (req, res, next) => { + if (req.actor.source !== "cloud_control" || verifiedCloudControlClaims(req)?.action !== "task-drain:read-body") { + throw forbidden("Cloud body-bound read authority is required"); + } + await readTaskDrainStatus(req, res, next); }); router.post( diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.ts index 773d9acb745..ca8657c54eb 100644 --- a/server/src/routes/openapi.ts +++ b/server/src/routes/openapi.ts @@ -2553,6 +2553,19 @@ for (const operation of githubBotOperations) { }); } +registry.registerPath({ + "x-paperclip-transport": "rest", + "x-paperclip-connection-management": "connections:configure", + method: "post", + path: "/api/chat-endpoints/{endpointId}/provider-work/{actionId}/retry", + tags: ["chat-channels"], + summary: "Retry retained provider receipt work", + description: "Requires current company connection-management access. Rearms only the same authorized retained Slack receipt or cleanup responsibility. Does not create a new form or spend a trigger. Revoked, foreign, or changed ownership is refused.", + request: { params: z.object({ endpointId: z.string().uuid(), actionId: z.string().uuid() }), body: jsonBody(z.object({}).strict()) }, + responses: { 200: r.ok(z.object({ actionId: z.string().uuid(), state: z.enum(["scheduled", "already_scheduled", "complete"]) })), + 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict }, +}); + registry.registerPath({ "x-paperclip-transport": "rest", "x-paperclip-connection-management": "connections:configure", @@ -7312,6 +7325,17 @@ registry.registerPath({ responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden }, }); +registry.registerPath({ + method: "post", + path: "/api/instance/task-drain/status", + tags: ["instance"], + summary: "Read task-drain status with body-bound signed Cloud idle capabilities", + description: "Requires the task-drain:read-body Cloud control assertion bound to the exact raw JSON SHA256. The body alone and browser administrator access do not grant this authority. Uses the same status/idle-safety reader and owner/scheduled-wake semantics as GET; does not start or stop a drain.", + request: { body: jsonBody(z.object({ idleCapabilities: z.unknown().describe("Complete closed incoming-wake and deadline catalogs, validated by the existing capability parser after signature and exact raw-body binding") }).strict()), + query: z.object({ upgradeReadiness: z.literal("1").optional(), ownerId: z.string().uuid().optional(), idleSleepSafety: z.literal("1").optional(), scheduledWake: z.enum(["1", "2", "domain-deadlines-v1"]).optional() }) }, + responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden }, +}); + registry.registerPath({ method: "post", path: "/api/instance/task-drain", diff --git a/server/src/services/chat-channels.ts b/server/src/services/chat-channels.ts index bbc5a8b18bf..55fb04afc3f 100644 --- a/server/src/services/chat-channels.ts +++ b/server/src/services/chat-channels.ts @@ -1,3 +1,9 @@ +import { retrySlackFormAcknowledgement } from "./chat-channels/slack/form-retry.js"; +import { createSlackFormConnectorAccess, registerSlackFormWorkOwner, reconcilePendingSlackFormWork, stageSlackFormOwnerConfiguration } from "./chat-channels/slack/form-owner.js"; +import { createSlackFormDeliveryOwner } from "./chat-channels/slack/form-delivery.js"; +import { createSlackFormPublicationStage } from "./chat-channels/slack/form-publication.js"; +import { withSlackCloudAcceptedAt, SlackCloudReceiptError } from "./chat-channels/slack/cloud-event.js"; +import { currentSlackFormOwnerBinding, withSlackFormSubmissionEvidence } from "./chat-channels/slack/cloud-forms.js"; import { githubReviewCheckService } from "./chat-github-checks.js"; import { createSlackCloudEventIngress } from "./chat-channels/slack/cloud-ingress.js"; import { createSlackCloudEnrollment, stageSlackCloudEnrollmentIntent, registerSlackCloudEnrollmentOwner } from "./chat-channels/slack/cloud-enrollment.js"; @@ -844,7 +850,30 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { } const withCredentialMutationLease = chatCredentialMutationLease(db, options); + const slackForms = createSlackFormConnectorAccess(); + const handleSlackFormReceipt = createSlackFormDeliveryOwner(db, { + connector: slackForms.connector, protocol: slackForms.protocol, withCredentialMutationLease, resolveCredentials, + webhookUrl: publicId => `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${publicId}/slack`, + handleWebhook: (publicId, request, generation, acceptedAt) => withSlackCloudAcceptedAt(request, acceptedAt, + () => handleWebhook(publicId, "slack", request, { slackCloudGeneration: generation })), + }); + const unregisterSlackFormWork = registerSlackFormWorkOwner(db, now => reconcilePendingSlackFormWork(db, { + identity: slackForms.identity(), protocol: slackForms.protocol, connector: slackForms.connector, + resolveBotToken: async endpointId => { + const record = await endpointRecord(endpointId); + if (!record || record.endpoint.provider !== "slack") throw new SlackCloudReceiptError("form_endpoint_custody_missing", "attention"); + return withCredentialMutationLease(record.endpoint, async lease => { + const credentials = await resolveCredentials(record.endpoint, db, ["botToken"]); + await lease.assertOwned(); + if (!credentials.botToken) throw new SlackCloudReceiptError("form_bot_credential_unavailable", "attention"); + return credentials.botToken; + }); + }, + report: (reason, actionId) => logger.warn({ reason, actionId }, "Slack form custody requires attention"), + }, now)); const unregisterSlackCloudIngress = createSlackCloudEventIngress(db, { + connectorIdentity: () => slackForms.identity() ?? undefined, + handleFormReceipt: handleSlackFormReceipt, withCredentialMutationLease, resolveCredentials, runtimeGeneration: endpoint => runtimeGeneration(endpoint.setup), handleWebhook: (publicId, provider, request, generation) => handleWebhook(publicId, provider, request, { slackCloudGeneration: generation }), }); @@ -3022,12 +3051,11 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { const response = await handleGitHubWebhookEvent(endpoint, request, endpointRuntime, runtimeContext); if (response) return response; } - const response = await endpointRuntime.handleWebhook( - request, - undefined, - githubResponseDeadlineAt ?? undefined, - provider === "telegram" ? serviceReceivedAtMs : undefined, - ); + const invokeWebhook = () => endpointRuntime.handleWebhook(request, undefined, githubResponseDeadlineAt ?? undefined, + provider === "telegram" ? serviceReceivedAtMs : undefined); + const response = provider === "slack" + ? await withSlackFormSubmissionEvidence(request, endpoint.id, currentSlackFormOwnerBinding({ ...endpoint.setup }) !== null, invokeWebhook) + : await invokeWebhook(); let lifecyclePayload: unknown = null; if (response.ok) { try { @@ -3418,14 +3446,21 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { : null; } - const slackCloudEnrollment = createSlackCloudEnrollment(db, { resolveCredentials, runtimeGeneration }); + const slackCloudEnrollment = createSlackCloudEnrollment(db, { resolveCredentials, runtimeGeneration, + onCommitted: async (tx, values) => { await stageSlackFormOwnerConfiguration(tx, values.endpointId, values.active, values.subject, + { instanceId: values.instanceId, environment: values.environment }); }, + }); const unregisterSlackCloudEnrollment = registerSlackCloudEnrollmentOwner(db, async endpointId => { const record = await endpointRecord(endpointId); if (!record) throw new Error("Slack enrollment endpoint is absent"); await withCredentialMutationLease(record.endpoint, lease => slackCloudEnrollment(endpointId, undefined, lease)); }); const { configure, configureWithCredentialLease, runtimeConfig } = createChatConfiguration(db, { - stageConfigurationEffects: stageSlackCloudEnrollmentIntent, + stageConfigurationEffects: async (tx, endpointId, actorUserId) => { + await stageSlackCloudEnrollmentIntent(tx, endpointId, actorUserId); + const identity = slackForms.identity(); + if (identity) await stageSlackFormOwnerConfiguration(tx, endpointId, false, actorUserId ?? "", identity); + }, endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime, nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials, @@ -3493,6 +3528,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { get publicationTransportPreparationBarrier() { return options.publicationTransportPreparationBarrier; }, }); const publicationWorker = createChatPublicationWorker(db, { + prepareSlackFormPublication: createSlackFormPublicationStage(db), CAPABILITIES, authorizeRetainedChatSourcePublication, claimPublicationTransportAuthorization, closeProgressToReplace, commitTaskControlCompletion, currentTaskControlPayload, endpointRecord, ensureDurablePublicationTransport, finalizePublicationFailure, hasCommittedTaskControlCompletion, @@ -3510,6 +3546,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, get heartbeat() { return options.heartbeat; }, }); const { handleAction, handleModalSubmit, reconcileTerminalConfirmationActions, registerTerminalConfirmationWork } = createChatInteractionCallbacks(db, { + withCredentialMutationLease, heartbeat: options.heartbeat, CAPABILITIES, MAX_ERROR_TEXT, actionThreadMatchesConversation, nonDirectDestinationAllowed, sameTeamsThreadIdentity, ensurePrincipal, endpointRecord, runtimeCallbackRecord, runtimeCallbackEndpoint, @@ -3563,6 +3600,11 @@ return { configure, inspectPhoton, test, + retrySlackFormReceipt: async (endpointId: string, actionId: string, management?: ConnectionManagementContext) => { + if (!management) throw forbidden("Connection management authority is required"); + return retrySlackFormAcknowledgement(db, { companyId: management.resource.companyId, endpointId, actionId, + identity: slackForms.identity, management }); + }, finishSlackSetup: (endpointId: string, userId: string, managementContext?: ConnectionManagementContext) => test(endpointId, { optionalSlackTestForUser: userId }, managementContext), setupTestStatus, handleWebhook, @@ -3634,6 +3676,7 @@ return { voice.unregisterAgentTools(); unregisterGitHubCloudIngress(); unregisterSlackCloudIngress(); + unregisterSlackFormWork(); unregisterSlackCloudEnrollment(); unregisterCommittedResponseAuthority(); await Promise.allSettled([...publicationEndpointTasks.values()]); diff --git a/server/src/services/chat-channels/idle-history-proof.ts b/server/src/services/chat-channels/idle-history-proof.ts index 992c71d8645..31e23ecd1db 100644 --- a/server/src/services/chat-channels/idle-history-proof.ts +++ b/server/src/services/chat-channels/idle-history-proof.ts @@ -1,3 +1,4 @@ +import { sql, type SQL } from "drizzle-orm"; import { finishedIdleRun, finishedIdleWake } from "../idle-work-proof.js"; import { completedGithubAction } from "./github/idle-history-proof.js"; import { completedSlackAction } from "./slack/idle-history-proof.js"; @@ -8,15 +9,19 @@ import { completedVoiceAction } from "../voice/idle-history-proof.js"; import { completedTeamsAction } from "./microsoft-teams/idle-history-proof.js"; import { completedTelegramAction } from "./telegram/idle-history-proof.js"; +function actionGraph(a: string): string { + return `jsonb_typeof(${a}.payload) = 'object' + AND EXISTS (SELECT 1 FROM chat_endpoints e WHERE e.id = ${a}.endpoint_id AND e.company_id = ${a}.company_id) + AND (${a}.delivery_id IS NULL OR EXISTS (SELECT 1 FROM chat_deliveries d + WHERE d.id = ${a}.delivery_id AND d.company_id = ${a}.company_id AND d.endpoint_id = ${a}.endpoint_id))`; +} + /** A processed inbound receipt can still own an unconsumed wake. */ function completedAction(a: string): string { const wake = `EXISTS (SELECT 1 FROM agent_wakeup_requests w WHERE w.company_id = ${a}.company_id AND w.id = ${a}.id AND w.agent_id::text = ${a}.payload->>'agentId' AND w.payload->>'issueId' = ${a}.payload->>'issueId' AND ${finishedIdleWake('w')})`; - return `jsonb_typeof(${a}.payload) = 'object' - AND EXISTS (SELECT 1 FROM chat_endpoints e WHERE e.id = ${a}.endpoint_id AND e.company_id = ${a}.company_id) - AND (${a}.delivery_id IS NULL OR EXISTS (SELECT 1 FROM chat_deliveries d - WHERE d.id = ${a}.delivery_id AND d.company_id = ${a}.company_id AND d.endpoint_id = ${a}.endpoint_id)) + return `${actionGraph(a)} AND CASE WHEN ${a}.kind = 'inbound_wakeup' THEN (${a}.status IN ('processed', 'failed') AND ${wake}) OR (${a}.status = 'failed' AND ${a}.result->>'code' = 'inbound_wakeup_delivery_rejected' @@ -127,3 +132,18 @@ export const idleChatHistoryChecks = [ AND (r.native_issue_id = h.conversation_id OR r.context_snapshot->>'issueId' = h.conversation_id::text) AND ${finishedIdleRun('r')}))) IS NOT TRUE` }, ] as const; + +export function idleChatHistoryChecksForOwner(completedActionIds: readonly string[] = []): readonly { reason: string; query: string | SQL }[] { + if (!completedActionIds.length) return idleChatHistoryChecks; + const ids = [...new Set(completedActionIds)]; + if (ids.some(id => !/^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$/i.test(id))) throw new Error("Invalid completed chat action identity"); + const normal = completedAction("a"); + // One bound UUID array; no retained-history-sized SQL literal/parameter list. + const completed = sql`(${sql.raw(normal)}) OR (${sql.raw(actionGraph("a"))} AND a.id IN (SELECT completed.id FROM unnest(${sql.param(ids)}::uuid[]) AS completed(id)))`; + return idleChatHistoryChecks.map(check => { + if (check.reason !== "chat_actions" && check.reason !== "chat_deliveries") return check; + const pieces = check.query.split(normal); + if (pieces.length !== 2) throw new Error("Chat history completion predicate changed"); + return { ...check, query: sql`${sql.raw(pieces[0]!)}${completed}${sql.raw(pieces[1]!)}` }; + }); +} diff --git a/server/src/services/chat-channels/interactions/callbacks.ts b/server/src/services/chat-channels/interactions/callbacks.ts index 24dfb13fede..1db05e4f726 100644 --- a/server/src/services/chat-channels/interactions/callbacks.ts +++ b/server/src/services/chat-channels/interactions/callbacks.ts @@ -1,3 +1,9 @@ +import { stageSlackFormSubmission } from "../slack/form-session.js"; +import { slackFormSubmissionEvidence } from "../slack/cloud-forms.js"; +import { slackFormOpenContext } from "../slack/cloud-forms.js"; +import { stageSlackFormOpenCallback, confirmSlackFormOpenCallback, isRetryableSlackFormUpdate } from "../slack/form-open-callback.js"; +import { SlackCloudReceiptError } from "../slack/cloud-event.js"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; import { notifyChatEndpointConfiguration } from "../configuration-work-notifications.js"; import { notifyChatConnectionHealth } from "../connection-health-notifications.js"; import { slackQuestionFormValidationError } from "../slack/question-forms.js"; @@ -115,6 +121,7 @@ import { createChatConfirmationActions } from "./confirmation-actions.js"; type ResourceRow = typeof chatEndpointResources.$inferSelect; type PrincipalResolution = { principal: typeof chatExternalPrincipals.$inferSelect; userId: string | null; linkedDenied: boolean }; type InteractionCallbacksOptions = { + withCredentialMutationLease?: ReturnType; heartbeat: import("../../issue-assignment-wakeup.js").IssueAssignmentWakeupDeps; CAPABILITIES: Record; MAX_ERROR_TEXT: number; @@ -986,6 +993,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb await options.questionFormOpenAuthorizationBarrier?.(); let providerAttempted = false; let attemptActionId: string | null = null; + const cloudForm = event.provider === "slack" ? slackFormOpenContext() : undefined; try { const attemptProviderActionId = event.event.triggerId ? `question_form_open_attempt:${createHash("sha256") @@ -998,7 +1006,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb ) .digest("hex")}` : null; - const preflight = await db.transaction(async (tx) => { + const runPreflight = (lease?: CredentialMutationLeaseGuard) => db.transaction(async (tx) => { await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: record.endpoint.id, @@ -1037,6 +1045,9 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb }) .onConflictDoNothing() .returning({ id: chatActions.id }); + if (cloudForm) return stageSlackFormOpenCallback(tx, { context: cloudForm, lease, resolved, + companyId: record.endpoint.companyId, endpointId: record.endpoint.id, conversationId: conversation.id, + principalId: principal.principal.id, attemptProviderActionId, attemptActionId: attempt?.id ?? null }); return attempt ? { kind: "ready" as const, @@ -1045,6 +1056,10 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb } : { kind: "duplicate" as const }; }); + if (cloudForm && !options.withCredentialMutationLease) throw new SlackCloudReceiptError("form_credential_lease_unavailable", "retry"); + const preflight = cloudForm + ? await options.withCredentialMutationLease!(record.endpoint, lease => runPreflight(lease)) + : await runPreflight(); if (preflight.kind === "denied") return deny(safelyKnown); if (preflight.kind === "duplicate") { if (event.provider === "discord") { @@ -1097,6 +1112,8 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb if (event.provider === "discord") return deny(safelyKnown); return; } + if (cloudForm) await db.transaction(tx => confirmSlackFormOpenCallback(tx, { context: cloudForm, + companyId: record.endpoint.companyId, endpointId: record.endpoint.id, principalId: principal.principal.id })); if (attemptActionId) { const [settledAttempt] = await db .update(chatActions) @@ -1123,6 +1140,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb return deny(safelyKnown); } if (!providerAttempted) throw error; + if (isRetryableSlackFormUpdate(cloudForm, error)) throw error; await recordModalOpenFailure( record.endpoint, event, @@ -1713,6 +1731,12 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb } return { action: "clear" }; }; + const persistSlackFormTerminal = async (tx: DbTransaction, resolved: IssueThreadInteraction) => { + if (event.provider !== "slack" || !slackFormSubmissionEvidence()) return; + await stageSlackFormSubmission(tx, { companyId: record.endpoint.companyId, endpointId: record.endpoint.id, + conversationId: conversation.id, principalId: principal.principal.id, authorizedUserId: principal.userId!, + runtimeGeneration: runtimeContext.generation, submitReceiptId: loaded.actionRowId, publicationId: originalPublication.id, resolved }); + }; if (loaded.status === "processed") { if (!isExactProcessedReplay(loaded)) { return deny( @@ -1724,6 +1748,13 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb loaded.payload, ); } + if (event.provider === "slack" && slackFormSubmissionEvidence()) { + await db.transaction(async tx => { + await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: event.endpointId, + expectedUserId: principal.userId!, principalId: principal.principal.id, runtimeContext }); + await persistSlackFormTerminal(tx, interaction!); + }); + } if (event.event.relatedThread) { await recordCurrentMicrosoftTeamsRoute( record.endpoint, @@ -1874,6 +1905,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb interactionId: resolved.id, }); if (!completed) throw denialError(); + await persistSlackFormTerminal(tx, resolved); await tx .update(chatActions) .set({ @@ -1959,6 +1991,12 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb if (!currentInteraction || currentInteraction.status === "pending") { return false; } + if (event.provider === "slack" && slackFormSubmissionEvidence() && currentInteraction.status === "answered" + && currentInteraction.resolvedByUserId === principal.userId) { + await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: event.endpointId, + expectedUserId: principal.userId!, principalId: principal.principal.id, runtimeContext }); + await persistSlackFormTerminal(tx, currentInteraction); + } await enqueueTerminalIssueInteractionChatPublications( tx, currentInteraction, diff --git a/server/src/services/chat-channels/publications/failures.ts b/server/src/services/chat-channels/publications/failures.ts index c7e22f1c680..777bac1fe34 100644 --- a/server/src/services/chat-channels/publications/failures.ts +++ b/server/src/services/chat-channels/publications/failures.ts @@ -1,3 +1,4 @@ +import type { ChatPublicationPreparationWait } from "./preparation.js"; import { notifyChatResponseAvailability } from "../../chat-response-availability.js"; import { notifyChatEndpointConfiguration } from "../configuration-work-notifications.js"; import { notifyChatConnectionHealth } from "../connection-health-notifications.js"; @@ -58,14 +59,19 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur providerAccepted = false, runtimeContext?: LifecycleRuntimeFence | null, credentialLease?: CredentialMutationLeaseGuard, + providerInvocation: "possible" | "not_started" = "possible", + preparationWait?: ChatPublicationPreparationWait, ): Promise { const attempts = publication.attempts + 1; + const waitingBeforeSend = !providerAccepted && providerInvocation === "not_started" && preparationWait?.kind === "wait"; const disposition = providerAccepted ? { kind: "delivery_unknown" as const, reason: "Provider accepted the publication, but Paperclip could not confirm its durable result", } + : waitingBeforeSend && preparationWait + ? { kind: "retry" as const, retryAfterMs: preparationWait.retryAfterMs, reason: preparationWait.reason } : error instanceof NativeChatReviewPresentationContentionError ? { kind: "retry" as const, retryAfterMs: 250, reason: error.message } : error instanceof FailedChatRetryPublicationReadError @@ -88,7 +94,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur ), reason: error.message, } - : classifyChatPublicationError(error, attempts); + : classifyChatPublicationError(error, attempts, providerInvocation); const failure = redactSensitiveText(disposition.reason).slice( 0, MAX_ERROR_TEXT, @@ -100,6 +106,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur const terminalRetry = disposition.kind === "retry" && disposition.providerRateLimit !== true && + !waitingBeforeSend && attempts >= 5; const attachmentFailure = attachmentFailureKind(publication.payload); const attachmentFailureRuntimeContext = @@ -127,6 +134,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur .update(chatPublications) .set({ state, + ...(waitingBeforeSend ? { attempts: publication.attempts } : {}), nextAttemptAt: disposition.kind === "retry" && !terminalRetry ? new Date(Date.now() + disposition.retryAfterMs) @@ -161,7 +169,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur : state === "cancelled" ? "task_control_destination_unavailable" : "task_control_transport_failed", - attempts, + attempts: waitingBeforeSend ? publication.attempts : attempts, retryable: state === "retry", }, updatedAt: new Date(), diff --git a/server/src/services/chat-channels/publications/preparation.ts b/server/src/services/chat-channels/publications/preparation.ts index 54df34ef5bd..6bd11b3134b 100644 --- a/server/src/services/chat-channels/publications/preparation.ts +++ b/server/src/services/chat-channels/publications/preparation.ts @@ -24,6 +24,13 @@ import { splitNativePublicationText, } from "../../chat-publication-text-parts.js"; +/** Returned only before provider invocation; waiting does not consume a delivery attempt. */ +export type ChatPublicationPreparationWait = Readonly<{ + kind: "wait"; + reason: string; + retryAfterMs: number; +}>; + type PublicationPreparationOptions = { runPublicationToReplace: ReturnType["runPublicationToReplace"]; publicationTransportPreparationBarrier?: (input: { publicationId: string }) => Promise; diff --git a/server/src/services/chat-channels/publications/worker.ts b/server/src/services/chat-channels/publications/worker.ts index 831a041fd39..93085a937bd 100644 --- a/server/src/services/chat-channels/publications/worker.ts +++ b/server/src/services/chat-channels/publications/worker.ts @@ -1,3 +1,4 @@ +import type { ChatPublicationPreparationWait } from "./preparation.js"; import { notifyChatResponseAvailability } from "../../chat-response-availability.js"; import { notifyVoicePublication } from "../../voice/voice-reply-notifications.js"; import { createGitHubPublicationPolicy } from "../github/publication-policy.js"; @@ -90,6 +91,7 @@ import type { createChatPublicationPreparation } from "./preparation.js"; import type { createChatPublicationTransport } from "./transport.js"; type PublicationWorkerOptions = Pick, "runOwnershipMilestoneSupersessionReason" | "runPublicationToReplace" | "receiptReactionCompletionRunId" | "inboundWakePublicationToReplace" | "interactionResolutionPublicationToReplace" | "interactionPromptPublicationToReplace" | "progressSupersededByFastResponse" | "runProgressSupersededByPublishedInteraction" | "taskStatusPublicationToReplace" | "closeProgressToReplace"> & { CAPABILITIES: Record; + prepareSlackFormPublication?: ReturnType; authorizeRetainedChatSourcePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise; claimPublicationTransportAuthorization: ReturnType["claimPublicationTransportAuthorization"]; commitTaskControlCompletion: ReturnType["commitTaskControlCompletion"]; @@ -424,6 +426,8 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp const settlePublicationFailure = async ( error: unknown, credentialLease?: CredentialMutationLeaseGuard, + providerInvocation: "possible" | "not_started" = "possible", + preparationWait?: ChatPublicationPreparationWait, ) => { try { await finalizePublicationFailure( @@ -432,6 +436,8 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp providerAccepted, publicationRuntimeContext, credentialLease ?? publicationLease, + providerInvocation, + preparationWait, ); } catch (finalizationError) { // If the provider accepted the message, a failed receipt write @@ -782,6 +788,19 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp runtimeContext: currentPublicationRuntimeContext, }) : null; + let stageSlackForm: Exclude>>, ChatPublicationPreparationWait> | undefined; + try { + const preparation = await options.prepareSlackFormPublication?.(authorizationClaim.endpoint, publication); + if (preparation == null || typeof preparation === "function") { + stageSlackForm = preparation; + } else { + await settlePublicationFailure(new Error(preparation.reason), credentialLease, "not_started", preparation); + return; + } + } catch (error) { + await settlePublicationFailure(error, credentialLease, "not_started"); + return; + } const sent = await postSafePublication({ endpoint: authorizationClaim.endpoint, conversation: authorizationClaim.conversation, @@ -886,6 +905,7 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp "Chat publication ownership changed before commit", ); } + await stageSlackForm?.(tx); if (telegramDraft) { const completedDraft = await tx .update(chatActions) diff --git a/server/src/services/chat-channels/runtime-types.ts b/server/src/services/chat-channels/runtime-types.ts index 4b9e9e5f5ab..bae9a9221f4 100644 --- a/server/src/services/chat-channels/runtime-types.ts +++ b/server/src/services/chat-channels/runtime-types.ts @@ -69,6 +69,7 @@ export type InternalSetupState = ChatEndpointSetupState & { nextAttemptAt: string | null; failureCode?: string; }; + slackCloudForms?: import("./slack/cloud-forms.js").SlackFormBinding; slackCloudEvents?: { version: 1; bindingId: string; diff --git a/server/src/services/chat-channels/slack/ack-receipt.ts b/server/src/services/chat-channels/slack/ack-receipt.ts new file mode 100644 index 00000000000..15ac8eda507 --- /dev/null +++ b/server/src/services/chat-channels/slack/ack-receipt.ts @@ -0,0 +1,67 @@ +import { createHash } from "node:crypto"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormBinding } from "./cloud-forms.js"; + +export type SlackFormFinalizedReceipt = Readonly<{ + version: 1; + kind: "slack_form_finalized"; + binding: SlackFormBinding; + environment: "development" | "test" | "staging" | "production"; + operationId: string; + deliveryId: string; + appReceiptId: string; + grantId: string; + publicationId: string; + interactionId: string; + userId: string; + viewDigest: string | null; + terminalDigest: string; + outcome: "submitted" | "closed" | "no_effect"; + finalizedAt: string; +}>; + +function invalid(): never { throw new Error("slack_form_ack_receipt_invalid"); } +export function parseSlackFormFinalizedReceipt(value: unknown): SlackFormFinalizedReceipt { + if (!value || typeof value !== "object" || Array.isArray(value)) return invalid(); + if (Buffer.byteLength(JSON.stringify(value), "utf8") > 8192) return invalid(); + const field = (key: string): unknown => Reflect.get(value, key); + const keys = ["version", "kind", "binding", "environment", "operationId", "deliveryId", "appReceiptId", "terminalDigest", "outcome", "finalizedAt", "grantId", "publicationId", "interactionId", "userId", "viewDigest"]; + if (Object.keys(value).sort().join() !== keys.sort().join()) return invalid(); + const binding = parseSlackFormBinding(field("binding")); + const environment = field("environment"); + const operationId = field("operationId"), deliveryId = field("deliveryId"); + const appReceiptId = field("appReceiptId"), terminalDigest = field("terminalDigest"); + const grantId = field("grantId"), publicationId = field("publicationId"), interactionId = field("interactionId"), userId = field("userId"), viewDigest = field("viewDigest"); + const outcome = field("outcome"), finalizedAt = field("finalizedAt"); + if (field("version") !== 1 || field("kind") !== "slack_form_finalized" + || environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production" + || typeof operationId !== "string" || !/^[a-f0-9]{64}$/.test(operationId) + || typeof deliveryId !== "string" || deliveryId !== `slack_form_${operationId}` || typeof appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(appReceiptId) + || typeof grantId !== "string" || !grantId.length || grantId.length > 200 + || typeof publicationId !== "string" || !publicationId.length || publicationId.length > 200 + || typeof interactionId !== "string" || !interactionId.length || interactionId.length > 200 + || typeof userId !== "string" || !userId.length || userId.length > 200 + || viewDigest !== null && (typeof viewDigest !== "string" || !/^[a-f0-9]{64}$/.test(viewDigest)) + || typeof terminalDigest !== "string" || !/^[a-f0-9]{64}$/.test(terminalDigest) + || outcome !== "submitted" && outcome !== "closed" && outcome !== "no_effect" + || typeof finalizedAt !== "string" || !Number.isFinite(Date.parse(finalizedAt))) return invalid(); + return Object.freeze({ version: 1, kind: "slack_form_finalized", binding: Object.freeze(binding), environment, + operationId, deliveryId, appReceiptId, grantId, publicationId, interactionId, userId, viewDigest, terminalDigest, outcome, finalizedAt }); +} + +export function assertSameSlackFormFinalization(actual: SlackFormFinalizedReceipt, expected: SlackFormFinalizedReceipt): void { + if (!sameSlackFormBinding(actual.binding, expected.binding) + || actual.environment !== expected.environment || actual.operationId !== expected.operationId + || actual.deliveryId !== expected.deliveryId || actual.appReceiptId !== expected.appReceiptId + || actual.grantId !== expected.grantId || actual.publicationId !== expected.publicationId + || actual.interactionId !== expected.interactionId || actual.userId !== expected.userId || actual.viewDigest !== expected.viewDigest + || actual.terminalDigest !== expected.terminalDigest || actual.outcome !== expected.outcome + || actual.finalizedAt !== expected.finalizedAt) return invalid(); +} + +export function slackFormFinalizedReceiptDigest(receipt: SlackFormFinalizedReceipt): string { + const b = receipt.binding; + return createHash("sha256").update(JSON.stringify(["slack-form-finalized-receipt/v1", b.version, b.instanceId, b.companyId, b.endpointId, + b.publicId, b.connectionId, b.runtimeGeneration, b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint, + receipt.environment, receipt.operationId, receipt.deliveryId, receipt.appReceiptId, receipt.grantId, receipt.publicationId, + receipt.interactionId, receipt.userId, receipt.viewDigest, receipt.terminalDigest, receipt.outcome, receipt.finalizedAt]), "utf8").digest("hex"); +} diff --git a/server/src/services/chat-channels/slack/ack-retirement-receipt.ts b/server/src/services/chat-channels/slack/ack-retirement-receipt.ts new file mode 100644 index 00000000000..0627f1aa9ac --- /dev/null +++ b/server/src/services/chat-channels/slack/ack-retirement-receipt.ts @@ -0,0 +1,60 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { isDeepStrictEqual } from "node:util"; +import { createHash } from "node:crypto"; +import { parseSlackFormBinding as parseFormBinding, parseSlackFormPublicEnrollment, type SlackFormBinding as FormBinding } from "./cloud-forms.js"; +function closedRecord(value: unknown, keys: readonly string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== [...keys].sort().join()) throw new Error("Invalid retirement receipt shape"); + return Object.fromEntries(Object.entries(value)); +} +const formOwnerId = (b: FormBinding) => `slack-form-owner:${createHash("sha256").update(JSON.stringify([b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, b.runtimeGeneration, b.credentialFingerprint])).digest("hex")}`; + +export type SlackFormRetirementReceipt = Readonly<{ + version: 1; kind: "slack_form_retired"; binding: FormBinding; + environment: "development" | "test" | "staging" | "production"; + subject: string; ownerId: string; receiptId: string; +}>; +function identity(binding: FormBinding, environment: string, subject: string) { + const b = binding; + return ["slack-form-retirement/v1", b.version, b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, + b.runtimeGeneration, b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint, environment, subject, formOwnerId(binding)]; +} +export function slackFormRetirementReceipt(binding: FormBinding, environment: string, subject: string): SlackFormRetirementReceipt { + const parsed = parseFormBinding(binding); + return parseSlackFormRetirementReceipt({ version: 1, kind: "slack_form_retired", binding: parsed, environment, subject, + ownerId: formOwnerId(parsed), receiptId: createHash("sha256").update(JSON.stringify(identity(parsed, environment, subject))).digest("hex") }); +} +export function parseSlackFormRetirementReceipt(value: unknown): SlackFormRetirementReceipt { + const v = closedRecord(value, ["version", "kind", "binding", "environment", "subject", "ownerId", "receiptId"]); + if (Buffer.byteLength(JSON.stringify(value), "utf8") > 8192 || v.version !== 1 || v.kind !== "slack_form_retired" + || typeof v.subject !== "string" || !v.subject.length || v.subject.length > 200 + || !["development", "test", "staging", "production"].includes(String(v.environment))) throw new Error("Invalid Slack retirement receipt"); + const binding = parseFormBinding(v.binding), environment = v.environment; + if (environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production") throw new Error("Invalid Slack retirement environment"); + const receiptId = createHash("sha256").update(JSON.stringify(identity(binding, environment, v.subject))).digest("hex"); + if (v.ownerId !== formOwnerId(binding) || v.receiptId !== receiptId) throw new Error("Slack retirement identity changed"); + return Object.freeze({ version: 1, kind: "slack_form_retired", binding: Object.freeze(binding), environment, subject: v.subject, + ownerId: formOwnerId(binding), receiptId }); +} + +export type SlackRetirementHistoryRow = { kind: string; companyId: string; endpointId: string; providerActionId?: string; status: string; + payload: Record; result: Record | null }; +export function slackFormRetirementHistoryReceipt(row: SlackRetirementHistoryRow): SlackFormRetirementReceipt | null { + if (!["slack_form_owner", "slack_form_grant"].includes(row.kind) || row.result?.phase !== "transferred" || row.result?.nextAttemptAt !== null) return null; + const retired = closedRecord(row.result.retirement, ["version", "binding", "instanceId", "environment", "subject", "receiptDigest", "receipt"]); + const receipt = parseSlackFormRetirementReceipt(retired.receipt), binding = receipt.binding; + if (row.companyId !== binding.companyId || row.endpointId !== binding.endpointId || row.payload.subject !== receipt.subject + || row.payload.environment !== receipt.environment || retired.version !== 1 || retired.instanceId !== binding.instanceId + || retired.environment !== receipt.environment || retired.subject !== receipt.subject || retired.receiptDigest !== receipt.receiptId + || !isDeepStrictEqual(parseFormBinding(retired.binding), binding) + || row.kind === "slack_form_owner" && row.status !== "processed" || row.kind === "slack_form_grant" && row.status !== "cancelled") throw new Error("Slack retirement history authority changed"); + if (row.payload.binding !== undefined) { + if (!isDeepStrictEqual(parseFormBinding(row.payload.binding), binding)) throw new Error("Slack retirement history binding changed"); + } else { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(enrollment, key) !== Reflect.get(binding, key))) throw new Error("Slack retirement enrollment history changed"); + } + if (row.kind === "slack_form_owner" && !isSlackFormEnrollmentActionId(row.providerActionId, parseSlackFormPublicEnrollment(row.payload.enrollment))) + throw new Error("Slack retirement history owner changed"); + return receipt; +} diff --git a/server/src/services/chat-channels/slack/adapter.ts b/server/src/services/chat-channels/slack/adapter.ts index 9e76796544b..1238473468c 100644 --- a/server/src/services/chat-channels/slack/adapter.ts +++ b/server/src/services/chat-channels/slack/adapter.ts @@ -1,8 +1,29 @@ +import { createHash } from "node:crypto"; +import type { ModalElement } from "chat"; +import { slackDeferredCallbackActive } from "./cloud-callback.js"; +import { slackFormOpenContext } from "./cloud-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; import { SlackAdapter } from "@chat-adapter/slack"; import type { AdapterPostableMessage, LinkPreview, RawMessage } from "chat"; import { slackSelectionMessage } from "./cards.js"; export class PaperclipSlackAdapter extends SlackAdapter { + override async openModal(triggerId: string, modal: ModalElement, contextId?: string): Promise<{ viewId: string }> { + const current = slackFormOpenContext(); + if (!current) { + if (slackDeferredCallbackActive()) throw new SlackCloudReceiptError("callback_trigger_owner_required", "attention"); + return super.openModal(triggerId, modal, contextId); + } + const view = current.operation.view; + if (!view || current.operation.closed || current.updateAttempted || createHash("sha256").update(triggerId).digest("hex") !== current.operation.triggerDigest) + throw new SlackCloudReceiptError("form_provider_attempt_changed", "attention"); + current.updateAttempted = true; + const result = await super.updateModal(view.id, { ...modal, notifyOnClose: true }, { hash: view.hash, externalId: view.externalId, contextId }); + if (result.viewId !== view.id || !result.viewHash) throw new SlackCloudReceiptError("form_update_receipt_invalid", "attention"); + current.updatedView = { ...view, hash: result.viewHash }; + return { viewId: result.viewId }; + } + override async postMessage(threadId: string, message: AdapterPostableMessage): Promise { const payload = slackSelectionMessage(message); if (!payload) return super.postMessage(threadId, message); diff --git a/server/src/services/chat-channels/slack/cloud-callback.test.ts b/server/src/services/chat-channels/slack/cloud-callback.test.ts new file mode 100644 index 00000000000..b6f3088f93c --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-callback.test.ts @@ -0,0 +1,105 @@ +import { guardSlackFixtureNetwork, stubSlackFixtureTransport } from "../../../__tests__/helpers/slack-cloud/transport.js"; +import { verifySlackSignature } from "@chat-adapter/slack/webhook"; +import type { ChatSdkRuntimeCallbacks } from "../../chat-sdk-runtime.js"; +import { createChatSdkEndpointRuntime } from "../../chat-sdk-runtime.js"; +import { createHash, createHmac } from "node:crypto"; +import { describe, expect, it, vi } from "vitest"; +import { receiveSlackCloudCallback, slackDeferredCallbackActive } from "./cloud-callback.js"; +import { slackCloudAcceptedAt, type SlackCloudEvent } from "./cloud-event.js"; +const acceptedAt = Date.parse("2099-01-01T00:00:00Z"); +function fixture(lane: "command" | "action" = "command") { + const current = { instanceId: "instance", environment: "test", companyId: "company", endpointId: "endpoint", connectionId: "connection", publicId: "a".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", bindingId: `slack-event:${"b".repeat(64)}`, signingSecret: "0123456789abcdef0123456789abcdef" }; + const body = lane === "command" ? new URLSearchParams({ command: "/paperclip", text: "useful original task", trigger_id: "original-one-use-trigger", team_id: "TTEST", api_app_id: "ATEST", user_id: "UTEST", channel_id: "CTEST", response_url: "https://hooks.slack.com/commands/controlled" }).toString() + : new URLSearchParams({ payload: JSON.stringify({ type: "block_actions", api_app_id: "ATEST", team: { id: "TTEST" }, user: { id: "UTEST" }, channel: { id: "CTEST" }, actions: [{ action_id: "opaque-confirmation", action_ts: "4070908800.000001" }], trigger_id: "original-one-use-trigger" }) }).toString(); + const rawBody = Buffer.from(body).toString("base64"), timestamp = String(Math.floor(acceptedAt / 1000)); + const signature = `v0=${createHmac("sha256", current.signingSecret).update(`v0:${timestamp}:`).update(body).digest("hex")}`; + const { signingSecret: _secret, ...identity } = current; + const event: SlackCloudEvent = { id: `slack_callback_${createHash("sha256").update(JSON.stringify([current.bindingId, timestamp, rawBody])).digest("hex")}`, provider: "slack", event: lane, + installationId: current.bindingId, bindingIds: [current.bindingId], createdAt: new Date(acceptedAt).toISOString(), payload: { slackCallback: { version: 1, ...identity, lane, userId: "UTEST", channelId: "CTEST", acceptedAt: new Date(acceptedAt).toISOString(), rawBody, timestamp, signature } } }; + return { current, event, body }; +} +describe("deferred Slack command and action custody", () => { + it.each(["command", "action"] as const)("preserves original %s bytes and authentication while awaiting actual callback completion", async lane => { + const f = fixture(lane); let completed = false; + await receiveSlackCloudCallback(f.event, f.current, async request => { + expect(slackDeferredCallbackActive()).toBe(true); expect(slackCloudAcceptedAt(request)).toBe(acceptedAt); + expect(await request.text()).toBe(f.body); expect(request.headers.get("content-type")).toBe("application/x-www-form-urlencoded"); + await Promise.resolve(); completed = true; return new Response(null, { status: 200 }); + }, acceptedAt + 3500); + expect(completed).toBe(true); expect(slackDeferredCallbackActive()).toBe(false); + }); + it("does not acknowledge unaccepted or failed callback work", async () => { + const f = fixture(); + await expect(receiveSlackCloudCallback(f.event, f.current, async () => new Response(null, { status: 503 }), acceptedAt)).rejects.toMatchObject({ disposition: "retry" }); + const original = new Error("actual admission failed"); + await expect(receiveSlackCloudCallback(f.event, f.current, async () => { throw original; }, acceptedAt)).rejects.toBe(original); + expect(slackDeferredCallbackActive()).toBe(false); + }); + it("rejects forged original authentication and foreign instance without invoking the SDK", async () => { + const f = fixture(), handler = vi.fn(async () => new Response(null, { status: 200 })); + const packet = f.event.payload.slackCallback as Record; + packet.signature = "v0=" + "0".repeat(64); + await expect(receiveSlackCloudCallback(f.event, f.current, handler, acceptedAt)).rejects.toMatchObject({ disposition: "attention" }); + const foreign = fixture(); + await receiveSlackCloudCallback(foreign.event, { ...foreign.current, instanceId: "foreign" }, handler, acceptedAt); + expect(handler).not.toHaveBeenCalled(); + }); + it("does not accept a callback profile as a view submission or options response owner", async () => { + const f = fixture(), handler = vi.fn(async () => new Response(null, { status: 200 })); + f.event.event = "view_submission"; + await expect(receiveSlackCloudCallback(f.event, f.current, handler, acceptedAt)).rejects.toMatchObject({ disposition: "invalid" }); + expect(handler).not.toHaveBeenCalled(); + }); + it("crosses the actual registered Slash SDK owner and refuses ordinary expired external evidence", async () => { + const network = guardSlackFixtureNetwork(), f = fixture(); + let runtime: ReturnType | undefined; + let primary: unknown; + try { + const slash: NonNullable = async event => { + await event.event.channel.postEphemeral(event.event.user, { markdown: "Useful normal bot notice" }, { fallbackToDM: false }); + }; + const onSlashCommand = vi.fn(slash); + runtime = createChatSdkEndpointRuntime({ companyId: f.current.companyId, endpointId: f.current.endpointId, logger: "silent", + callbacks: { onMessage: async () => {}, onSlashCommand }, persistence: { read: async () => null, compareAndSet: async () => true, deleteIfVersion: async () => true }, + providerConfig: { provider: "slack", userName: "paperclip-agent", credentials: { apiUrl: "http://127.0.0.1:1/api/", botToken: "xoxb-synthetic", botUserId: "UBOT", signingSecret: f.current.signingSecret } } }); + await runtime.initialize(); + const transport = stubSlackFixtureTransport(runtime.getProviderAdapter()); + const delegate = transport.getMockImplementation(); + if (!delegate) throw new Error("Slack fixture transport is missing"); + let notices = 0; + transport.mockImplementation(async config => { + const url = new URL(config.url, config.baseURL); + if (url.hostname === "127.0.0.1" && url.pathname.endsWith("/chat.postEphemeral")) { + notices++; + const body = new URLSearchParams(String(Reflect.get(config, "data"))); + expect(body.get("channel")).toBe("CTEST"); expect(body.get("user")).toBe("UTEST"); + expect(body.has("response_url")).toBe(false); + return { data: { ok: true, message_ts: "1.2" }, status: 200, statusText: "OK", headers: {}, config, request: { path: url.pathname } }; + } + return delegate(config); + }); + const owner = runtime; + await receiveSlackCloudCallback(f.event, f.current, request => owner.handleWebhook(request), acceptedAt + 31 * 60_000); + expect(notices).toBe(1); + expect(onSlashCommand).toHaveBeenCalledOnce(); + expect(onSlashCommand).toHaveBeenCalledWith(expect.objectContaining({ endpointId: f.current.endpointId, provider: "slack", + event: expect.objectContaining({ command: "/paperclip", text: "useful original task", triggerId: "original-one-use-trigger" }) })); + await transport.assertNoErrors(); + const packet = f.event.payload.slackCallback as Record; + await expect(verifySlackSignature(f.body, new Headers({ "x-slack-request-timestamp": String(packet.timestamp), "x-slack-signature": String(packet.signature) }), + { signingSecret: f.current.signingSecret, now: () => acceptedAt + 31 * 60_000 })).rejects.toThrow("too old"); + const response = await runtime.handleWebhook(new Request("https://external.invalid", { method: "POST", body: f.body, + headers: { "content-type": "application/x-www-form-urlencoded", "x-slack-request-timestamp": String(packet.timestamp), "x-slack-signature": String(packet.signature) } })); + expect(response.status).toBe(401); expect(onSlashCommand).toHaveBeenCalledOnce(); + } catch (error) { primary = error; throw error; } + finally { + const failures: unknown[] = []; + try { await runtime?.shutdown(); } catch (error) { failures.push(error); } + try { network.assertQuiet(); } catch (error) { failures.push(error); } + try { network.restore(); } catch (error) { failures.push(error); } + if (failures.length) throw new AggregateError(primary === undefined ? failures : [primary, ...failures], "Slack registered callback cleanup failed"); + } + }); + +}); diff --git a/server/src/services/chat-channels/slack/cloud-callback.ts b/server/src/services/chat-channels/slack/cloud-callback.ts new file mode 100644 index 00000000000..cda9398f440 --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-callback.ts @@ -0,0 +1,56 @@ +import { AsyncLocalStorage } from "node:async_hooks"; +import { createHash, createHmac, timingSafeEqual } from "node:crypto"; +import { SlackCloudReceiptError, withSlackCloudAcceptedAt, type CurrentSlackCloudBinding, type SlackCloudEvent } from "./cloud-event.js"; + +const deferred = new AsyncLocalStorage(); +export const slackDeferredCallbackActive = () => deferred.getStore() === true; +export async function receiveSlackCloudCallback(event: SlackCloudEvent, current: (CurrentSlackCloudBinding & { instanceId: string; environment: string }) | null, + handleWebhook: (request: Request) => Promise, now = Date.now()): Promise { + if (!current) return; + const raw = event.payload.slackCallback; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const packet = raw as Record; + const expectedKeys = ["version", "lane", "teamId", "appId", "userId", "channelId", "instanceId", "environment", "bindingId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "acceptedAt", "timestamp", "signature", "rawBody"]; + if (Object.keys(packet).sort().join() !== expectedKeys.sort().join() || packet.version !== 1 || event.provider !== "slack" + || !["command", "action"].includes(String(packet.lane)) || event.event !== packet.lane) + throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + if (["companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "bindingId", "instanceId", "environment"].some(key => packet[key] !== Reflect.get(current, key))) return; + if (event.installationId !== current.bindingId || event.bindingIds.length !== 1 || event.bindingIds[0] !== current.bindingId + || typeof packet.timestamp !== "string" || !/^[0-9]{1,16}$/.test(packet.timestamp) + || typeof packet.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(packet.signature) + || typeof packet.rawBody !== "string" || packet.rawBody.length > 1_398_104 || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(packet.rawBody)) + throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const acceptedAt = Date.parse(event.createdAt); + if (packet.acceptedAt !== event.createdAt || !Number.isFinite(acceptedAt) || acceptedAt > now + 300_000 || now - acceptedAt >= 30 * 24 * 60 * 60_000 + || Math.abs(Math.floor(acceptedAt / 1000) - Number(packet.timestamp)) > 300) + throw new SlackCloudReceiptError("callback_acceptance_invalid", "attention"); + const body = Buffer.from(packet.rawBody, "base64"); + if (body.byteLength > 1_048_576) throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const digest = createHash("sha256").update(JSON.stringify([current.bindingId, packet.timestamp, packet.rawBody])).digest("hex"); + if (event.id !== `slack_callback_${digest}` || !timingSafeEqual(createHmac("sha256", current.signingSecret).update(`v0:${packet.timestamp}:`).update(body).digest(), Buffer.from(packet.signature.slice(3), "hex"))) + throw new SlackCloudReceiptError("callback_original_authentication_failed", "attention"); + const params = new URLSearchParams(body.toString("utf8")); + for (const key of params.keys()) if (params.getAll(key).length !== 1) throw new SlackCloudReceiptError("callback_duplicate_field", "invalid"); + if (packet.lane === "command") { + if (params.has("payload") || !/^\/[A-Za-z0-9_-]{1,32}$/.test(params.get("command") ?? "") || !params.get("trigger_id") || !params.has("text") + || params.get("team_id") !== current.teamId || params.get("api_app_id") !== current.appId || params.get("user_id") !== packet.userId || params.get("channel_id") !== packet.channelId) + throw new SlackCloudReceiptError("callback_original_identity_changed", "invalid"); + } else { + let payload: unknown; + try { payload = JSON.parse(params.get("payload") ?? "null"); } catch { throw new SlackCloudReceiptError("callback_original_invalid", "invalid"); } + if (!payload || typeof payload !== "object" || Array.isArray(payload) || [...params.keys()].join() !== "payload") throw new SlackCloudReceiptError("callback_original_invalid", "invalid"); + const actions = Reflect.get(payload, "actions"); + const field = (key: string) => { const value = Reflect.get(payload, key); return value && typeof value === "object" ? Reflect.get(value, "id") : undefined; }; + if (Reflect.get(payload, "type") !== "block_actions" || Reflect.get(payload, "api_app_id") !== current.appId || field("team") !== current.teamId || field("user") !== packet.userId || field("channel") !== packet.channelId + || !Array.isArray(actions) || actions.length !== 1 || !actions[0] || typeof actions[0] !== "object" + || typeof Reflect.get(actions[0], "action_id") !== "string" || !Reflect.get(actions[0], "action_id") || String(Reflect.get(actions[0], "action_id")).length > 255 || String(Reflect.get(actions[0], "action_id")).startsWith("pcf:")) + throw new SlackCloudReceiptError("callback_trigger_owner_required", "attention"); + } + const request = new Request("https://paperclip.invalid/slack-cloud-ingress", { method: "POST", headers: { "content-type": "application/x-www-form-urlencoded", + "x-slack-request-timestamp": packet.timestamp, "x-slack-signature": packet.signature }, body: new Uint8Array(body) }); + await deferred.run(true, () => withSlackCloudAcceptedAt(request, acceptedAt, async () => { + const response = await handleWebhook(request); + await response.body?.cancel(); + if (!response.ok) throw new SlackCloudReceiptError("callback_unaccepted", "retry"); + })); +} diff --git a/server/src/services/chat-channels/slack/cloud-consumer.ts b/server/src/services/chat-channels/slack/cloud-consumer.ts new file mode 100644 index 00000000000..71559e9e4b1 --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-consumer.ts @@ -0,0 +1,58 @@ +import { completedSlackFormSessionReceipt } from "./form-idle-history.js"; +import { and, gt, inArray, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { projectSlackFormRetainedOwner, type SlackFormRetainedRow } from "./form-session.js"; + +export type SlackFormConnectorIdentity = Readonly<{ instanceId: string; environment: "development" | "test" | "staging" | "production" }>; +export function slackFormConnectorIdentity(connector: unknown): SlackFormConnectorIdentity | null { + if (!connector || typeof connector !== "object") return null; + let value: unknown; + try { + const getter = Reflect.get(connector, "instanceIdentity"); + if (typeof getter !== "function") return null; + value = Reflect.apply(getter, connector, []); + } catch { return null; } + if (!value || typeof value !== "object" || Array.isArray(value) + || Object.keys(value).sort().join() !== "environment,instanceId") return null; + const instanceId = Reflect.get(value, "instanceId"), environment = Reflect.get(value, "environment"); + if (typeof instanceId !== "string" || !instanceId.length || instanceId.length > 200 + || environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production") return null; + return Object.freeze({ instanceId, environment }); +} + +export function selectSlackFormRetainedConsumer(rows: readonly SlackFormRetainedRow[], identity: SlackFormConnectorIdentity) { + let owner: ReturnType = null; + const attention: { actionId: string; code: string }[] = []; + for (const row of rows) { + try { + const value = projectSlackFormRetainedOwner(row, identity.instanceId, identity.environment); + if (value && row.kind === "slack_form_session" && row.result?.formAck && typeof row.result.formAck === "object" + && Reflect.get(row.result.formAck, "phase") === "transferred" && completedSlackFormSessionReceipt(row)) continue; + if (value && !owner) owner = value; + } catch { + attention.push({ actionId: row.id, code: "form_retained_owner_invalid" }); + } + } + return { owner, attention }; +} + +export async function readSlackFormRetainedConsumer(db: Pick, connector: unknown) { + const identity = slackFormConnectorIdentity(connector); + if (!identity) return { owner: null, attention: [] }; + let after: string | undefined; + const attention: { actionId: string; code: string }[] = []; + for (;;) { + const rows = await db.select().from(chatActions).where(and( + inArray(chatActions.kind, ["slack_form_session", "slack_form_owner", "slack_form_grant"]), + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${identity.instanceId}`, + after === undefined ? undefined : gt(chatActions.id, after))) + .orderBy(chatActions.id).limit(32); + const selected = selectSlackFormRetainedConsumer(rows, identity); + attention.push(...selected.attention.slice(0, Math.max(0, 32 - attention.length))); + if (selected.owner) return { owner: selected.owner, attention }; + if (rows.length < 32) return { owner: null, attention }; + const next = rows[rows.length - 1]?.id; + if (!next || next === after) throw new Error("Slack retained consumer cursor did not advance"); + after = next; + } +} diff --git a/server/src/services/chat-channels/slack/cloud-enrollment.ts b/server/src/services/chat-channels/slack/cloud-enrollment.ts index be8e33394e3..89881195fc5 100644 --- a/server/src/services/chat-channels/slack/cloud-enrollment.ts +++ b/server/src/services/chat-channels/slack/cloud-enrollment.ts @@ -145,6 +145,7 @@ export function createSlackCloudEnrollment(db: Db, options: { resolveCredentials: ReturnType["resolveCredentials"]; runtimeGeneration(setup: InternalSetupState): number; connector?: Pick; + onCommitted?(tx: Parameters[0]>[0], values: { endpointId: string; active: boolean; subject: string; instanceId: string; environment: import("../../paperclip-cloud-connector.js").PaperclipCloudConnectorEnvironment }): Promise; }) { return async (endpointId: string, _actorUserId: string | null | undefined, lease: CredentialMutationLeaseGuard) => { const [current] = await db.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints) @@ -189,6 +190,7 @@ export function createSlackCloudEnrollment(db: Db, options: { sql`${chatEndpoints.setup}->'slackCloudIntent'->>'id'=${intent.id}`, sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer,0)=${intent.runtimeGeneration}`)).returning({ id: chatEndpoints.id }); if (updated.length !== 1) throw new Error("Slack configuration changed before event enrollment committed"); + if (change.receipt) await options.onCommitted?.(tx, { endpointId, active: intent.active, subject: intent.subject, instanceId: intent.instanceId, environment: config.environment }); await lease.assertOwned(tx); }); try { diff --git a/server/src/services/chat-channels/slack/cloud-event.ts b/server/src/services/chat-channels/slack/cloud-event.ts index b589de638bc..878c2dcb096 100644 --- a/server/src/services/chat-channels/slack/cloud-event.ts +++ b/server/src/services/chat-channels/slack/cloud-event.ts @@ -3,6 +3,14 @@ import { createHash, createHmac, timingSafeEqual } from "node:crypto"; const acceptedTimes = new WeakMap(); export const slackCloudAcceptedAt = (request: Request): number | undefined => acceptedTimes.get(request); +export async function withSlackCloudAcceptedAt(request: Request, acceptedAt: number, work: () => Promise): Promise { + if (!Number.isFinite(acceptedAt)) throw new SlackCloudReceiptError("form_accepted_time_invalid", "invalid"); + const before = acceptedTimes.get(request); + acceptedTimes.set(request, acceptedAt); + try { return await work(); } + finally { if (before === undefined) acceptedTimes.delete(request); else acceptedTimes.set(request, before); } +} + export class SlackCloudReceiptError extends Error { constructor(readonly code: string, readonly disposition: "invalid" | "attention" | "retry") { super(code); } } diff --git a/server/src/services/chat-channels/slack/cloud-forms.ts b/server/src/services/chat-channels/slack/cloud-forms.ts new file mode 100644 index 00000000000..5e8e73055df --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-forms.ts @@ -0,0 +1,268 @@ +import { AsyncLocalStorage } from "node:async_hooks"; +import { createHash, createHmac, timingSafeEqual } from "node:crypto"; +import { and, eq, inArray } from "drizzle-orm"; +import { chatActions, chatConversations, chatPublications, type Db } from "@paperclipai/db"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import { slackThreadChannelId } from "./messages.js"; +import { SlackCloudReceiptError, type SlackCloudEvent } from "./cloud-event.js"; + +export type SlackFormBinding = { + version: 1; instanceId: string; companyId: string; endpointId: string; publicId: string; + connectionId: string; runtimeGeneration: number; teamId: string; appId: string; + eventBindingId: string; credentialFingerprint: string; +}; +export type SlackFormPublicEnrollment = Omit & { bindingId: string }; +export function parseSlackFormPublicEnrollment(value: unknown): SlackFormPublicEnrollment { + const v = record(value, ["version", "instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "bindingId"]); + if (v.version !== 1 || !Number.isSafeInteger(v.runtimeGeneration) || Number(v.runtimeGeneration) < 1 + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "teamId", "appId", "bindingId"].some(k => typeof v[k] !== "string" || !String(v[k]).length || String(v[k]).length > 200) + || !/^[A-Za-z0-9_-]{43}$/.test(String(v.publicId)) || !/^T[A-Z0-9]+$/.test(String(v.teamId)) || !/^A[A-Z0-9]+$/.test(String(v.appId)) + || !/^slack-event:[a-f0-9]{64}$/.test(String(v.bindingId))) throw new SlackCloudReceiptError("invalid_form_enrollment", "invalid"); + return v as unknown as SlackFormPublicEnrollment; +} +export type SlackIssuedFormGrant = { + version: 1; binding: SlackFormBinding; grantId: string; + conversationId: string; conversationGeneration: number; publicationId: string; interactionId: string; + openActionRowId: string; openActionDigest: string; submitActionRowId: string; submitActionDigest: string; + expiresAt: string; channelId: string; threadTs: string | null; providerMessageId: string; +}; +export type SlackFormView = { id: string; hash: string; externalId: string }; +export type SlackFormOperation = { + version: 1; binding: SlackFormBinding; grantId: string; id: string; externalId: string; + deliveryId: string; requestDigest: string; triggerDigest: string; userId: string; + acceptedAt: string; triggerDeadline: string; + phase: "reserved" | "attempted" | "confirmed" | "rejected" | "ambiguous" | "terminal_proven" | "acknowledged"; + view: SlackFormView | null; + closed: { kind: "closed" | "submitted"; view: SlackFormView; acceptedAt: string; requestDigest: string; evidence: { + version: 1; kind: "close" | "submission"; submitReceiptId: string | null; instanceId: string; environment: string; operationId: string; binding: SlackFormBinding; + acceptedAt: string; rawBody: string; timestamp: string; signature: string; + } } | null; + acknowledgement: { appReceiptId: string; acceptedAt: string } | null; + rejection: { code: "expired_before_spend" | "expired_trigger_id" | "invalid_auth" | "token_revoked" | "not_authed" | "invalid_arguments"; + observedAt: string; responseDigest: string; evidence: Record } | null; +}; +const hash = (s: string) => createHash("sha256").update(s).digest("hex"); +function record(value: unknown, keys: string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== keys.sort().join()) + throw new SlackCloudReceiptError("invalid_form_receipt", "invalid"); + return value as Record; +} +export function parseSlackFormBinding(value: unknown): SlackFormBinding { + const v = record(value, ["version", "instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "eventBindingId", "credentialFingerprint"]); + if (v.version !== 1 || !Number.isSafeInteger(v.runtimeGeneration) || Number(v.runtimeGeneration) < 1 + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "teamId", "appId", "eventBindingId", "credentialFingerprint"].some(k => typeof v[k] !== "string" || !(v[k] as string).length || (v[k] as string).length > 200) + || !/^[A-Za-z0-9_-]{43}$/.test(String(v.publicId)) || !/^T[A-Z0-9]+$/.test(String(v.teamId)) || !/^A[A-Z0-9]+$/.test(String(v.appId)) + || !/^slack-event:[a-f0-9]{64}$/.test(String(v.eventBindingId)) || !/^[a-f0-9]{64}$/.test(String(v.credentialFingerprint))) + throw new SlackCloudReceiptError("invalid_form_binding", "invalid"); + return v as unknown as SlackFormBinding; +} +export function slackFormSetupRecord(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("form_setup_invalid", "attention"); + const setup: Record = { ...value }; + return setup; +} +export function currentSlackFormOwnerBinding(setup: Record): SlackFormBinding | null { + if (setup.slackCloudForms === undefined) return null; + return parseSlackFormBinding(setup.slackCloudForms); +} +export const sameSlackFormBinding = (a: SlackFormBinding, b: SlackFormBinding) => Object.keys(a).every(k => Reflect.get(a, k) === Reflect.get(b, k)); +function view(value: unknown): SlackFormView { + const v = record(value, ["id", "hash", "externalId"]); + if (!/^V[A-Z0-9]+$/.test(String(v.id)) || typeof v.hash !== "string" || !v.hash.length || v.hash.length > 200 + || typeof v.externalId !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(v.externalId)) throw new SlackCloudReceiptError("invalid_form_view", "invalid"); + return v as unknown as SlackFormView; +} +export function parseSlackFormOperation(value: unknown): SlackFormOperation { + const v = record(value, ["version", "binding", "grantId", "id", "externalId", "deliveryId", "requestDigest", "triggerDigest", "userId", "acceptedAt", "triggerDeadline", "phase", "view", "closed", "acknowledgement", "rejection"]); + parseSlackFormBinding(v.binding); + if (v.version !== 1 || ![v.id, v.grantId, v.requestDigest, v.triggerDigest].every(x => typeof x === "string" && /^[a-f0-9]{64}$/.test(x)) + || v.externalId !== `pc-form-${v.id}` || v.deliveryId !== `slack_form_${v.id}` + || !/^[UW][A-Z0-9]+$/.test(String(v.userId)) || typeof v.acceptedAt !== "string" || !Number.isFinite(Date.parse(v.acceptedAt)) + || typeof v.triggerDeadline !== "string" || !Number.isFinite(Date.parse(v.triggerDeadline)) + || !["reserved", "attempted", "confirmed", "rejected", "ambiguous", "terminal_proven", "acknowledged"].includes(String(v.phase))) + throw new SlackCloudReceiptError("invalid_form_operation", "invalid"); + if (v.view !== null) view(v.view); + if (v.closed !== null) { + const c = record(v.closed, ["kind", "view", "acceptedAt", "requestDigest", "evidence"]); view(c.view); + const e = record(c.evidence, ["version", "kind", "submitReceiptId", "instanceId", "environment", "operationId", "binding", "acceptedAt", "rawBody", "timestamp", "signature"]); + parseSlackFormBinding(e.binding); + if (e.version !== 1 || !["close", "submission"].includes(String(e.kind)) || (e.kind === "close" ? e.submitReceiptId !== null : typeof e.submitReceiptId !== "string") || typeof e.instanceId !== "string" || typeof e.environment !== "string" || typeof e.operationId !== "string" + || typeof e.acceptedAt !== "string" || !Number.isFinite(Date.parse(e.acceptedAt)) || typeof e.rawBody !== "string" || e.rawBody.length > 1398104 + || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(e.rawBody) + || typeof e.timestamp !== "string" || !/^[0-9]{1,16}$/.test(e.timestamp) || typeof e.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(e.signature)) + throw new SlackCloudReceiptError("invalid_form_close_evidence", "invalid"); + if (!["closed", "submitted"].includes(String(c.kind)) || typeof c.acceptedAt !== "string" || !Number.isFinite(Date.parse(c.acceptedAt)) || typeof c.requestDigest !== "string" + || !/^[a-f0-9]{64}$/.test(c.requestDigest)) throw new SlackCloudReceiptError("invalid_form_close", "invalid"); + } + if (v.rejection !== null) { + const n = record(v.rejection, ["code", "observedAt", "responseDigest", "evidence"]); + if (!["expired_before_spend", "expired_trigger_id", "invalid_auth", "token_revoked", "not_authed", "invalid_arguments"].includes(String(n.code)) + || typeof n.observedAt !== "string" || !Number.isFinite(Date.parse(n.observedAt)) || typeof n.responseDigest !== "string" || !/^[a-f0-9]{64}$/.test(n.responseDigest) + || !n.evidence || typeof n.evidence !== "object" || Array.isArray(n.evidence)) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + const e = record(n.evidence, ["version", "kind", "instanceId", "environment", "operationId", "binding", "observedAt", "code", "responseDigest", "observation"]); + parseSlackFormBinding(e.binding); + if (e.version !== 1 || e.kind !== "rejection" || typeof e.instanceId !== "string" || typeof e.environment !== "string" || typeof e.operationId !== "string" + || !e.observation || typeof e.observation !== "object" || Array.isArray(e.observation)) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + } + if ((v.phase === "rejected" && !v.rejection) || (v.rejection && (v.closed || v.view || !["rejected", "terminal_proven", "acknowledged"].includes(String(v.phase))))) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + if (v.acknowledgement !== null) { + const a = record(v.acknowledgement, ["appReceiptId", "acceptedAt"]); + if (typeof a.appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(a.appReceiptId) || typeof a.acceptedAt !== "string" || !Number.isFinite(Date.parse(a.acceptedAt))) + throw new SlackCloudReceiptError("invalid_form_acknowledgement", "invalid"); + } + const op = v as unknown as SlackFormOperation; + if (["terminal_proven", "acknowledged"].includes(op.phase) !== (op.acknowledgement !== null) || (["terminal_proven", "acknowledged"].includes(op.phase) && !op.closed && !op.rejection)) + throw new SlackCloudReceiptError("invalid_form_acknowledgement", "invalid"); + if (((op.phase === "confirmed" || ["terminal_proven", "acknowledged"].includes(op.phase) && op.closed) && !op.view) || (op.view && op.view.externalId !== op.externalId) + || (op.closed && (op.closed.view.externalId !== op.externalId || (op.view && op.view.id !== op.closed.view.id)))) + throw new SlackCloudReceiptError("form_view_changed", "attention"); + return op; +} +export function slackIssuedFormGrantId(binding: SlackFormBinding, openActionRowId: string, submitActionRowId: string, publicationId: string, interactionId: string): string { + return hash(JSON.stringify([[binding.instanceId, binding.companyId, binding.endpointId, binding.publicId, binding.connectionId, binding.runtimeGeneration, + binding.teamId, binding.appId, binding.eventBindingId, binding.credentialFingerprint], openActionRowId, submitActionRowId, publicationId, interactionId])); +} +export async function projectSlackIssuedFormGrant(db: Pick, binding: SlackFormBinding, + resolved: ResolvedChatQuestionFormOpen): Promise { + const [publication] = await db.select().from(chatPublications).where(and(eq(chatPublications.id, resolved.publicationId), + eq(chatPublications.companyId, binding.companyId), eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.state, "published"))).limit(1); + if (!publication?.providerMessageId || !publication.publishedAt) return null; + const [conversation] = await db.select().from(chatConversations).where(and(eq(chatConversations.id, publication.conversationId), + eq(chatConversations.companyId, binding.companyId), eq(chatConversations.endpointId, binding.endpointId), eq(chatConversations.state, "active"))).limit(1); + if (!conversation) return null; + const channelId = slackThreadChannelId(conversation.externalThreadId); + const thread = /^slack:[^:]+:([0-9]+[.][0-9]+)?$/.exec(conversation.externalThreadId); + if (!channelId || !thread || conversation.externalConversationId !== `slack:${channelId}`) return null; + const rows = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), eq(chatActions.conversationId, conversation.id), inArray(chatActions.id, [resolved.openActionRowId, resolved.submitActionRowId]))); + const open = rows.find(a => a.id === resolved.openActionRowId && a.kind === "question_form_open" && a.status === "issued" && a.providerActionId === resolved.openActionId); + const submit = rows.find(a => a.id === resolved.submitActionRowId && a.kind === "question_form_submit" && a.status === "issued" && a.providerActionId === resolved.submitActionId); + if (!open || !submit || open.payload.version !== 1 || submit.payload.version !== 1 + || open.payload.interactionId !== resolved.interactionId || submit.payload.interactionId !== resolved.interactionId + || open.payload.publicationId !== resolved.publicationId || submit.payload.publicationId !== resolved.publicationId + || open.payload.formActionId !== submit.providerActionId || submit.payload.formActionId !== submit.providerActionId + || typeof open.payload.expiresAt !== "string" || open.payload.expiresAt !== submit.payload.expiresAt) return null; + const grantId = slackIssuedFormGrantId(binding, resolved.openActionRowId, resolved.submitActionRowId, resolved.publicationId, resolved.interactionId); + return { version: 1, binding, grantId, conversationId: conversation.id, conversationGeneration: conversation.sessionGeneration, + publicationId: publication.id, interactionId: resolved.interactionId, openActionRowId: open.id, openActionDigest: hash(open.providerActionId), + submitActionRowId: submit.id, submitActionDigest: hash(submit.providerActionId), expiresAt: open.payload.expiresAt, + channelId, threadTs: thread[1] || null, providerMessageId: publication.providerMessageId }; +} +export type SlackFormLookup = (values: { binding: SlackFormBinding; operationId: string; deliveryId: string }) => Promise; +export type SlackFormDeliveryAuthority = { binding: SlackFormBinding; instanceId: string; environment: string } & + ({ kind: "current"; signingSecret: string } | { kind: "retained" }); +export async function lookupSlackFormDelivery(event: SlackCloudEvent, current: SlackFormDeliveryAuthority, + lookup: SlackFormLookup, now = Date.now()): Promise<{ operation: SlackFormOperation; originalPayload: Record; authority: "current" | "retained" }> { + if (event.provider !== "slack" || event.event !== "form_open") throw new SlackCloudReceiptError("unsupported_form_receipt", "attention"); + const packet = record(event.payload.slackFormOpen, ["version", "instanceId", "environment", "operation", "acceptedAt", "rawBody", "timestamp", "signature"]); + const op = parseSlackFormOperation(packet.operation); + if (packet.version !== 1 || current.binding.instanceId !== current.instanceId || !sameSlackFormBinding(op.binding, current.binding) || packet.instanceId !== current.instanceId || packet.environment !== current.environment + || event.id !== op.deliveryId || event.createdAt !== packet.acceptedAt || event.installationId !== `slack-form-open:${op.id}` + || !event.bindingIds.includes(`slack-form-open:${op.id}`)) throw new SlackCloudReceiptError("form_binding_changed", "attention"); + const acceptedAt = typeof packet.acceptedAt === "string" ? Date.parse(packet.acceptedAt) : Number.NaN; + if (typeof packet.rawBody !== "string" || packet.rawBody.length > 1398104 || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(packet.rawBody) + || typeof packet.timestamp !== "string" || !/^[0-9]{1,16}$/.test(packet.timestamp) || typeof packet.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(packet.signature) + || !Number.isFinite(acceptedAt) || acceptedAt > now + 300000 || now - acceptedAt >= 30 * 24 * 60 * 60000 || Math.abs(Math.floor(acceptedAt / 1000) - Number(packet.timestamp)) > 300) + throw new SlackCloudReceiptError("form_original_evidence_invalid", "attention"); + const raw = Buffer.from(packet.rawBody, "base64"); + if (raw.byteLength > 1048576) throw new SlackCloudReceiptError("form_original_too_large", "invalid"); + if (packet.acceptedAt !== op.acceptedAt || hash(JSON.stringify([packet.rawBody, packet.timestamp, packet.signature])) !== op.requestDigest) + throw new SlackCloudReceiptError("form_original_authentication_failed", "attention"); + if (current.kind === "current") { + const signature = createHmac("sha256", current.signingSecret).update(`v0:${packet.timestamp}:`).update(raw).digest(); + if (!timingSafeEqual(signature, Buffer.from(packet.signature.slice(3), "hex"))) throw new SlackCloudReceiptError("form_original_authentication_failed", "attention"); + } + let original: unknown; + try { original = JSON.parse(new URLSearchParams(raw.toString("utf8")).get("payload") ?? "null"); } + catch { throw new SlackCloudReceiptError("form_original_payload_invalid", "invalid"); } + if (!original || typeof original !== "object" || Array.isArray(original)) throw new SlackCloudReceiptError("form_original_payload_invalid", "invalid"); + const body = original as Record; + if (body.type !== "block_actions" || typeof body.trigger_id !== "string" || hash(body.trigger_id) !== op.triggerDigest + || !body.user || typeof body.user !== "object" || Reflect.get(body.user, "id") !== op.userId + || !body.team || typeof body.team !== "object" || Reflect.get(body.team, "id") !== current.binding.teamId + || body.api_app_id !== current.binding.appId) throw new SlackCloudReceiptError("form_original_identity_changed", "attention"); + const receipt = record(await lookup({ binding: current.binding, operationId: op.id, deliveryId: op.deliveryId }), ["version", "instanceId", "environment", "operation"]); + let latest = parseSlackFormOperation(receipt.operation); + if (receipt.version !== 1 || receipt.instanceId !== current.instanceId || receipt.environment !== current.environment + || !sameSlackFormBinding(latest.binding, op.binding) + || ["id", "grantId", "externalId", "deliveryId", "requestDigest", "triggerDigest", "userId", "acceptedAt", "triggerDeadline"].some(k => Reflect.get(latest, k) !== Reflect.get(op, k))) + throw new SlackCloudReceiptError("form_lookup_identity_changed", "attention"); + if (latest.rejection) { + const n = latest.rejection, evidence = n.evidence; + if (evidence.instanceId !== current.instanceId || evidence.environment !== current.environment || evidence.operationId !== op.id + || !sameSlackFormBinding(parseSlackFormBinding(evidence.binding), op.binding) || evidence.observedAt !== n.observedAt || evidence.code !== n.code + || evidence.responseDigest !== n.responseDigest || hash(JSON.stringify([op.id, n.code, evidence.observation])) !== n.responseDigest) + throw new SlackCloudReceiptError("form_no_effect_evidence_changed", "attention"); + } + if (latest.closed) { + const c = latest.closed, evidence = c.evidence; + if (evidence.instanceId !== current.instanceId || evidence.environment !== current.environment || evidence.operationId !== op.id + || !sameSlackFormBinding(evidence.binding, op.binding) || evidence.acceptedAt !== c.acceptedAt + || Math.abs(Math.floor(Date.parse(evidence.acceptedAt) / 1000) - Number(evidence.timestamp)) > 300 + || hash(JSON.stringify([evidence.rawBody, evidence.timestamp, evidence.signature])) !== c.requestDigest) + throw new SlackCloudReceiptError("form_close_evidence_changed", "attention"); + const closeRaw = Buffer.from(evidence.rawBody, "base64"); + if (closeRaw.length > 1048576) throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); + if (current.kind === "current") { + const signature = createHmac("sha256", current.signingSecret).update(`v0:${evidence.timestamp}:`).update(closeRaw).digest(); + if (!timingSafeEqual(signature, Buffer.from(evidence.signature.slice(3), "hex"))) throw new SlackCloudReceiptError("form_close_authentication_failed", "attention"); + } + let closeBody: unknown; + try { closeBody = JSON.parse(new URLSearchParams(closeRaw.toString("utf8")).get("payload") ?? "null"); } + catch { throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); } + if (!closeBody || typeof closeBody !== "object" || Array.isArray(closeBody)) throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); + const v = Reflect.get(closeBody, "view"), user = Reflect.get(closeBody, "user"), team = Reflect.get(closeBody, "team"); + if (Reflect.get(closeBody, "type") !== (c.kind === "submitted" ? "view_submission" : "view_closed") + || evidence.kind !== (c.kind === "submitted" ? "submission" : "close") || Reflect.get(closeBody, "api_app_id") !== op.binding.appId + || !v || typeof v !== "object" || !user || typeof user !== "object" || !team || typeof team !== "object" + || Reflect.get(user, "id") !== op.userId || Reflect.get(team, "id") !== op.binding.teamId || Reflect.get(v, "id") !== c.view.id + || Reflect.get(v, "external_id") !== op.externalId || Reflect.get(v, "hash") !== c.view.hash || Reflect.get(v, "app_id") !== op.binding.appId) + throw new SlackCloudReceiptError("form_close_identity_changed", "attention"); + } + if (current.kind === "retained" && !latest.closed && !latest.rejection) throw new SlackCloudReceiptError("form_retained_resource_unresolved", "attention"); + if (["reserved", "attempted"].includes(latest.phase) && !latest.closed) throw new SlackCloudReceiptError("form_receipt_pending", "retry"); + return { operation: latest, originalPayload: body, authority: current.kind }; +} + +export function assertSlackFormTerminalAck(operation: SlackFormOperation, appReceiptId: string): void { + if (["terminal_proven", "acknowledged"].includes(operation.phase) && operation.acknowledgement?.appReceiptId === appReceiptId && (operation.closed || operation.rejection)) return; + throw new SlackCloudReceiptError(operation.phase === "confirmed" ? "form_view_active" : "form_resource_unresolved", + operation.phase === "confirmed" || operation.closed ? "retry" : "attention"); +} + +export type SlackFormSubmissionEvidence = Readonly<{ endpointId: string; acceptedAt: string; rawBody: string; timestamp: string; signature: string }>; +const submissions = new AsyncLocalStorage(); +export const slackFormSubmissionEvidence = (): SlackFormSubmissionEvidence | undefined => submissions.getStore(); +export async function withSlackFormSubmissionEvidence(request: Request, endpointId: string, enabled: boolean, work: () => Promise): Promise { + if (!enabled) return work(); + const acceptedAt = new Date().toISOString(), copy = request.clone(); + if (!copy.body) return work(); + const reader = copy.body.getReader(), chunks: Uint8Array[] = []; let size = 0; + try { + for (;;) { + const part = await reader.read(); if (part.done) break; + size += part.value.byteLength; + if (size > 1048576) throw new SlackCloudReceiptError("form_submission_too_large", "invalid"); + chunks.push(part.value); + } + } finally { void reader.cancel().catch(() => {}); reader.releaseLock(); } + const bytes = Buffer.concat(chunks), encoded = new URLSearchParams(bytes.toString("utf8")); + if (encoded.getAll("payload").length !== 1) return work(); + let body: unknown; + try { body = JSON.parse(encoded.get("payload")!); } catch { return work(); } + if (!body || typeof body !== "object" || Reflect.get(body, "type") !== "view_submission") return work(); + const view = Reflect.get(body, "view"); + if (!view || typeof view !== "object" || typeof Reflect.get(view, "external_id") !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(Reflect.get(view, "external_id"))) return work(); + const timestamp = request.headers.get("x-slack-request-timestamp"), signature = request.headers.get("x-slack-signature"); + if (!timestamp || !/^[0-9]{1,16}$/.test(timestamp) || !signature || !/^v0=[a-f0-9]{64}$/.test(signature)) return work(); + return submissions.run(Object.freeze({ endpointId, acceptedAt, rawBody: bytes.toString("base64"), timestamp, signature }), work); +} + +export type SlackFormOpenContext = { operation: SlackFormOperation; currentBinding: SlackFormBinding; environment?: string; + sessionActionId?: string; updateAttempted?: boolean; updatedView?: SlackFormView }; +const formOpens = new AsyncLocalStorage(); +export const slackFormOpenContext = (): SlackFormOpenContext | undefined => formOpens.getStore(); +export function withSlackFormOpenContext(operation: SlackFormOperation, currentBinding: SlackFormBinding, work: () => Promise, environment?: string): Promise { + if (operation.phase !== "confirmed" || !operation.view || operation.closed || operation.rejection || !sameSlackFormBinding(operation.binding, currentBinding)) + throw new SlackCloudReceiptError("form_new_use_not_current", "attention"); + return formOpens.run({ operation, currentBinding, environment }, work); +} diff --git a/server/src/services/chat-channels/slack/cloud-ingress.ts b/server/src/services/chat-channels/slack/cloud-ingress.ts index a4368552abc..6d701665111 100644 --- a/server/src/services/chat-channels/slack/cloud-ingress.ts +++ b/server/src/services/chat-channels/slack/cloud-ingress.ts @@ -1,3 +1,4 @@ +import { receiveSlackCloudCallback } from "./cloud-callback.js"; import type { Db } from "@paperclipai/db"; import { chatEndpoints, toolConnections } from "@paperclipai/db"; import { and, eq } from "drizzle-orm"; @@ -26,10 +27,19 @@ export function createSlackCloudEventIngress(db: Db, options: { withCredentialMutationLease: ReturnType; resolveCredentials: ReturnType["resolveCredentials"]; runtimeGeneration(endpoint: EndpointRow): number; + handleFormReceipt?(event: SlackCloudEvent): Promise; + connectorIdentity?(): { instanceId: string; environment: string } | undefined; handleWebhook(publicId: string, provider: "slack", request: Request, expectedGeneration: number): Promise; }) { return registerSlackCloudEventIngress(db, async event => { - const value = event.payload.slackEvent; + if (event.payload.slackFormOpen !== undefined) { + if (!options.handleFormReceipt) throw new SlackCloudReceiptError("form_owner_not_ready", "retry"); + return options.handleFormReceipt(event); + } + const callback = event.payload.slackCallback !== undefined; + const identity = callback ? options.connectorIdentity?.() : undefined; + if (callback && !identity) throw new SlackCloudReceiptError("callback_identity_unavailable", "retry"); + const value = callback ? event.payload.slackCallback : event.payload.slackEvent; if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("invalid_receipt", "invalid"); const packet = value as Record; if (typeof packet.endpointId !== "string" || !/^[a-f0-9-]{36}$/i.test(packet.endpointId)) @@ -45,7 +55,9 @@ export function createSlackCloudEventIngress(db: Db, options: { const endpoint = current.endpoint; if (packet.companyId !== endpoint.companyId || packet.publicId !== endpoint.publicId || packet.connectionId !== endpoint.connectionId || packet.runtimeGeneration !== options.runtimeGeneration(endpoint)) return null; - const enrolled = (endpoint.setup as InternalSetupState).slackCloudEvents; + const setup = endpoint.setup as InternalSetupState; + if (callback && setup.slackCloudInstanceId !== identity?.instanceId) throw new SlackCloudReceiptError("callback_instance_changed", "attention"); + const enrolled = setup.slackCloudEvents; if (!enrolled) throw new Error("Slack Cloud enrollment is not locally committed"); if (enrolled.version !== 1 || enrolled.runtimeGeneration !== options.runtimeGeneration(endpoint) @@ -57,6 +69,11 @@ export function createSlackCloudEventIngress(db: Db, options: { bindingId: enrolled.bindingId, teamId: enrolled.teamId, appId: enrolled.appId, signingSecret }; }); if (!binding) return; + if (callback) { + if (!binding || !identity) return; + return receiveSlackCloudCallback(event, { ...binding, ...identity }, + request => options.handleWebhook(binding.publicId, "slack", request, binding.runtimeGeneration)); + } await receiveSlackCloudEvent(event, binding, request => options.handleWebhook(binding.publicId, "slack", request, binding.runtimeGeneration)); }); diff --git a/server/src/services/chat-channels/slack/configured-coverage.test.ts b/server/src/services/chat-channels/slack/configured-coverage.test.ts new file mode 100644 index 00000000000..2a7482eeb8a --- /dev/null +++ b/server/src/services/chat-channels/slack/configured-coverage.test.ts @@ -0,0 +1,23 @@ +import { expect, it } from "vitest"; +import { CAPABILITIES } from "../provider-catalog.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { requiredSlackIngressLanes, parseSlackConfiguredCoverage, slackConfiguredBindingsFingerprint, matchSlackConfiguredCoverage } from "./configured-coverage.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const requirement = { binding, capabilities: CAPABILITIES.slack, registeredOptions: false }; +it("requires every default configured Slack lane rather than treating a form slice as complete", () => { + expect(requiredSlackIngressLanes(requirement)).toEqual(["actions", "commands", "events", "forms", "modal_close", "modal_submit"]); +}); +it("requires options whenever a real options handler is registered", () => { + expect(requiredSlackIngressLanes({ ...requirement, registeredOptions: true })).toContain("options"); +}); +it("the closed coverage parser refuses foreign instance and altered exact binding fingerprint", () => { + const proof = { version: 1, instanceId: "instance", generation: "c".repeat(64), bindingsFingerprint: slackConfiguredBindingsFingerprint([binding]), + entries: [{ binding, lanes: requiredSlackIngressLanes(requirement) }] }; + expect(parseSlackConfiguredCoverage(proof).entries[0]?.binding.connectionId).toBe("connection"); + expect(() => parseSlackConfiguredCoverage({ ...proof, instanceId: "foreign" })).toThrow(); + expect(() => parseSlackConfiguredCoverage({ ...proof, bindingsFingerprint: "d".repeat(64) })).toThrow(); +}); +it("absence of a genuinely verified capability never discharges endpoint or connection IDs", () => { + expect(matchSlackConfiguredCoverage([requirement], undefined, { instanceId: "instance", generation: "c".repeat(64) })).toBeUndefined(); +}); diff --git a/server/src/services/chat-channels/slack/configured-coverage.ts b/server/src/services/chat-channels/slack/configured-coverage.ts new file mode 100644 index 00000000000..e6573de1e4a --- /dev/null +++ b/server/src/services/chat-channels/slack/configured-coverage.ts @@ -0,0 +1,66 @@ +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { createHash } from "node:crypto"; +import type { ChatAdapterCapabilities } from "@paperclipai/shared"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormBinding } from "./cloud-forms.js"; +export const SLACK_INGRESS_LANES = ["actions", "commands", "events", "forms", "modal_close", "modal_submit", "options"] as const; +export type SlackIngressLane = typeof SLACK_INGRESS_LANES[number]; +export type SlackConfiguredRequirement = Readonly<{ binding: SlackFormBinding; capabilities: Pick; registeredOptions: boolean }>; +export type SlackConfiguredCoverage = Readonly<{ version: 1; instanceId: string; generation: string; bindingsFingerprint: string; + entries: readonly Readonly<{ binding: SlackFormBinding; lanes: readonly SlackIngressLane[] }>[] }>; +const hash = (value: unknown) => createHash("sha256").update(JSON.stringify(value)).digest("hex"); +function exact(value: unknown, keys: string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== [...keys].sort().join()) throw new Error("Invalid Slack configured coverage"); + return Object.fromEntries(Object.entries(value)); +} +export function requiredSlackIngressLanes(input: SlackConfiguredRequirement): readonly SlackIngressLane[] { + const lanes: SlackIngressLane[] = ["events"]; + if (input.capabilities.actions) lanes.push("actions"); + if (input.capabilities.slashCommands) lanes.push("commands"); + if (input.capabilities.modals) lanes.push("forms", "modal_close", "modal_submit"); + if (input.registeredOptions) lanes.push("options"); + return lanes.sort(); +} +export function slackConfiguredBindingsFingerprint(bindings: readonly SlackFormBinding[]): string { + return hash(bindings.map(b => [b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, b.runtimeGeneration, + b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint]).sort((a, b) => JSON.stringify(a) < JSON.stringify(b) ? -1 : JSON.stringify(a) > JSON.stringify(b) ? 1 : 0)); +} +export function parseSlackConfiguredCoverage(value: unknown): SlackConfiguredCoverage { + const v = exact(value, ["version", "instanceId", "generation", "bindingsFingerprint", "entries"]); + if (v.version !== 1 || typeof v.instanceId !== "string" || !v.instanceId || typeof v.generation !== "string" || !/^[a-f0-9]{64}$/.test(v.generation) + || typeof v.bindingsFingerprint !== "string" || !/^[a-f0-9]{64}$/.test(v.bindingsFingerprint) || !Array.isArray(v.entries) || v.entries.length > 1000) throw new Error("Invalid Slack configured coverage"); + const seen = new Set(); + const entries = v.entries.map(value => { + const entry = exact(value, ["binding", "lanes"]), binding = parseSlackFormBinding(entry.binding); + const key = JSON.stringify([binding.companyId, binding.endpointId, binding.connectionId]); + if (binding.instanceId !== v.instanceId || seen.has(key) || !Array.isArray(entry.lanes) || !entry.lanes.length || new Set(entry.lanes).size !== entry.lanes.length + || entry.lanes.join() !== [...entry.lanes].sort().join()) throw new Error("Invalid Slack configured coverage"); + seen.add(key); + const lanes = entry.lanes.map(lane => { + const known = SLACK_INGRESS_LANES.find(value => value === lane); + if (!known) throw new Error("Unknown Slack configured lane"); + return known; + }); + return Object.freeze({ binding: Object.freeze(binding), lanes: Object.freeze(lanes) }); + }); + if (slackConfiguredBindingsFingerprint(entries.map(entry => entry.binding)) !== v.bindingsFingerprint) throw new Error("Slack configured fingerprint changed"); + return Object.freeze({ version: 1, instanceId: v.instanceId, generation: v.generation, bindingsFingerprint: v.bindingsFingerprint, entries: Object.freeze(entries) }); +} +export function matchSlackConfiguredCoverage(requirements: readonly SlackConfiguredRequirement[], verifiedCapabilities: VerifiedIdleCapabilities | undefined, + expected: { instanceId: string; generation: string }): { endpointIds: readonly string[]; connectionIds: readonly string[] } | undefined { + const incoming = verifiedCapabilities?.incomingWake; + if (!incoming || !incoming.routes.some(route => String(route) === "slack-broker-events-v1")) return; + const raw = Reflect.get(incoming, "slackBroker"); + if (raw === undefined) return; + const verifiedProof = parseSlackConfiguredCoverage(raw); + if (incoming.generation !== expected.generation || verifiedProof.instanceId !== expected.instanceId || verifiedProof.generation !== expected.generation || requirements.length > 1000) return; + const seen = new Set(); + for (const requirement of requirements) { + const binding = requirement.binding; + if (binding.instanceId !== expected.instanceId || seen.has(binding.endpointId)) return; + seen.add(binding.endpointId); + const entry = verifiedProof.entries.find(value => sameSlackFormBinding(value.binding, binding)); + if (!entry || requiredSlackIngressLanes(requirement).some(lane => !entry.lanes.includes(lane))) return; + } + if (verifiedProof.bindingsFingerprint !== slackConfiguredBindingsFingerprint(requirements.map(value => value.binding))) return; + return { endpointIds: [...seen].sort(), connectionIds: [...new Set(requirements.map(value => value.binding.connectionId))].sort() }; +} diff --git a/server/src/services/chat-channels/slack/form-ack-work.test.ts b/server/src/services/chat-channels/slack/form-ack-work.test.ts new file mode 100644 index 00000000000..cbef64c5b3a --- /dev/null +++ b/server/src/services/chat-channels/slack/form-ack-work.test.ts @@ -0,0 +1,30 @@ +import { generateKeyPairSync, sign } from "node:crypto"; +import { afterEach, expect, it, vi } from "vitest"; +import { acceptSlackFormAckReceipt } from "./form-ack-work.js"; +import { parseSlackFormFinalizedReceipt } from "./ack-receipt.js"; + +const original = { stack: process.env.PAPERCLIP_CLOUD_STACK_ID, jwks: process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS }; +afterEach(() => { + if (original.stack === undefined) delete process.env.PAPERCLIP_CLOUD_STACK_ID; else process.env.PAPERCLIP_CLOUD_STACK_ID = original.stack; + if (original.jwks === undefined) delete process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS; else process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS = original.jwks; +}); +it("rejects a genuinely signed foreign-company receipt and its action ID before any SQL", async () => { + const keys = generateKeyPairSync("ed25519"), kid = "ack-company-test"; + process.env.PAPERCLIP_CLOUD_STACK_ID = "instance"; + process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS = JSON.stringify({ keys: [{ ...keys.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] }); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: "test", + binding: { version: 1, instanceId: "instance", companyId: "foreign", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "T1", appId: "A1", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }, + operationId: "c".repeat(64), deliveryId: "slack_form_" + "c".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "U1", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-06T00:00:00.000Z" }); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: "instance", receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), keys.privateKey).toString("base64url"); + const tx = { select: vi.fn(), update: vi.fn() }; + await expect(acceptSlackFormAckReceipt(tx, { companyId: "current", instanceId: "instance", leaseId: "lease", receiptJws })) + .rejects.toMatchObject({ code: "form_ack_company_changed" }); + expect(tx.select).not.toHaveBeenCalled(); + expect(tx.update).not.toHaveBeenCalled(); +}); diff --git a/server/src/services/chat-channels/slack/form-ack-work.ts b/server/src/services/chat-channels/slack/form-ack-work.ts new file mode 100644 index 00000000000..c038d19659e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-ack-work.ts @@ -0,0 +1,201 @@ +import { replaceSlackFormReceiptSignature } from "./form-signature-refresh.js"; +import { readSlackFormOriginalProofs } from "./form-opening-proof.js"; +import { createHash } from "node:crypto"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { verifySlackFormFinalizedReceipt } from "./receipt-authority.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormProofAcceptance } from "./form-session.js"; +import { parseSlackFormFinalizedReceipt, slackFormFinalizedReceiptDigest, type SlackFormFinalizedReceipt } from "./ack-receipt.js"; + +type Tx = Pick; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function record(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return fail("form_ack_record_invalid"); + return Object.fromEntries(Object.entries(value)); +} +export type SlackFormAckWork = Readonly<{ + version: 1; phase: "pending" | "accepted" | "gc_pending" | "transferred"; leaseId: string; deliveryId: string; + receiptJws: string | null; receipt: SlackFormFinalizedReceipt | null; +}>; +export function parseSlackFormAckWork(value: unknown): SlackFormAckWork { + const v = record(value); + if (Object.keys(v).sort().join() !== "deliveryId,leaseId,phase,receipt,receiptJws,version" || v.version !== 1 + || typeof v.leaseId !== "string" || !v.leaseId.length || v.leaseId.length > 200 + || typeof v.deliveryId !== "string" || !/^slack_form_[a-f0-9]{64}$/.test(v.deliveryId) + || v.phase !== "pending" && v.phase !== "accepted" && v.phase !== "gc_pending" && v.phase !== "transferred" + || v.receiptJws !== null && (typeof v.receiptJws !== "string" || Buffer.byteLength(v.receiptJws, "ascii") > 12000 || !/^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$/.test(v.receiptJws))) return fail("form_ack_work_invalid"); + const receipt = v.receipt === null ? null : parseSlackFormFinalizedReceipt(v.receipt); + if (v.phase === "pending" ? receipt !== null || v.receiptJws !== null : !receipt || typeof v.receiptJws !== "string") return fail("form_ack_work_invalid"); + if (receipt && receipt.deliveryId !== v.deliveryId) return fail("form_ack_delivery_changed"); + return { version: 1, phase: v.phase, leaseId: v.leaseId, deliveryId: v.deliveryId, receiptJws: typeof v.receiptJws === "string" ? v.receiptJws : null, receipt }; +} +export function assertSlackFormReceiptHistory(row: Pick, receipt: SlackFormFinalizedReceipt): void { + const intent = record(row.result?.terminalIntent), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance); + const binding = parseSlackFormBinding(row.payload.binding), view = intent.view; + let viewDigest: string | null = null; + if (view !== null) { + const parsed = record(view); + if (Object.keys(parsed).sort().join() !== "externalId,hash,id" || typeof parsed.id !== "string" || typeof parsed.hash !== "string" || typeof parsed.externalId !== "string") + return fail("form_ack_view_changed"); + viewDigest = createHash("sha256").update(JSON.stringify([parsed.id, parsed.hash, parsed.externalId]), "utf8").digest("hex"); + } + if (row.kind !== "slack_form_session" || row.companyId !== receipt.binding.companyId || row.endpointId !== receipt.binding.endpointId + || !sameSlackFormBinding(binding, receipt.binding) || !sameSlackFormBinding(proof.binding, binding) + || row.id !== receipt.appReceiptId || proof.appReceiptId !== row.id || intent.appReceiptId !== row.id + || receipt.environment !== row.payload.environment || receipt.operationId !== row.payload.operationId || receipt.operationId !== intent.operationId + || receipt.deliveryId !== intent.deliveryId || receipt.grantId !== row.payload.grantId || receipt.publicationId !== row.payload.publicationId + || receipt.interactionId !== row.payload.interactionId || receipt.userId !== row.payload.userId || receipt.outcome !== intent.outcome + || receipt.terminalDigest !== proof.terminalDigest || receipt.outcome !== proof.outcome || receipt.viewDigest !== viewDigest + || row.result?.phase !== (receipt.outcome === "no_effect" ? "not_opened" : receipt.outcome)) + fail("form_ack_terminal_identity_changed"); +} +export async function stageSlackFormNormalAck(tx: Tx, input: { companyId: string; instanceId: string; leaseId: string; deliveryIds: readonly string[] }): Promise { + const ids = [...new Set(input.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)))]; + if (!ids.length) return; + if (ids.length > 50 || !input.leaseId.length || input.leaseId.length > 200) return fail("form_ack_batch_invalid"); + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"), + inArray(chatActions.providerActionId, ids.map(id => `slack-form-session:${id.slice("slack_form_".length)}`)))).for("update"); + if (rows.length !== ids.length) return fail("form_ack_session_missing"); + for (const row of rows) { + const intent = record(row.result?.terminalIntent), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance); + if (row.companyId !== input.companyId || proof.binding.companyId !== input.companyId || proof.binding.instanceId !== input.instanceId || proof.appReceiptId !== row.id || !ids.includes(String(intent.deliveryId)) + || row.companyId !== proof.binding.companyId || row.endpointId !== proof.binding.endpointId + || !["submitted", "closed", "not_opened"].includes(String(row.result?.phase))) return fail("form_ack_authority_changed"); + if (row.result?.formAck !== undefined) { + const old = parseSlackFormAckWork(row.result.formAck); + if (old.deliveryId !== intent.deliveryId) return fail("form_ack_delivery_changed"); + if (old.phase !== "pending") continue; + } + const work: SlackFormAckWork = { version: 1, phase: "pending", leaseId: input.leaseId, deliveryId: String(intent.deliveryId), receiptJws: null, receipt: null }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const staged = await tx.update(chatActions).set({ result: { ...row.result, formAck: work, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (staged.length !== 1) return fail("form_ack_stage_changed"); + } +} +export async function acceptSlackFormAckReceipt(tx: Tx, input: { companyId: string; instanceId: string; leaseId: string; receiptJws: string }): Promise { + const receipt = verifyRetainedSlackAckReceipt(input.receiptJws); + if (receipt.binding.companyId !== input.companyId) return fail("form_ack_company_changed"); + if (receipt.binding.instanceId !== input.instanceId) return fail("form_ack_instance_changed"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, receipt.appReceiptId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, receipt.binding.endpointId), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row) return fail("form_ack_session_missing"); + assertSlackFormReceiptHistory(row, receipt); + const old = parseSlackFormAckWork(row.result?.formAck); + if (old.leaseId !== input.leaseId || old.deliveryId !== receipt.deliveryId) return fail("form_ack_lease_changed"); + if (old.receipt && !isDeepStrictEqual(old.receipt, receipt)) return fail("form_ack_receipt_changed"); + if (old.phase === "transferred" || old.phase === "gc_pending") return row.id; + const work: SlackFormAckWork = { ...old, phase: "accepted", receiptJws: input.receiptJws, receipt }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...row.result, formAck: work, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return fail("form_ack_acceptance_changed"); + return row.id; +} +export async function transferSlackFormAckReceipt(db: Db, input: { companyId: string; actionId: string; instanceId: string; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + let [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"))).limit(1); + if (!row) return fail("form_ack_session_missing"); + if (row.companyId !== input.companyId) return fail("form_ack_company_changed"); + let work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase === "transferred") return; + if (work.phase === "gc_pending") return releaseSlackAckFact(db, row, input); + if (work.phase !== "accepted" || !work.receipt || !work.receiptJws || work.receipt.binding.companyId !== input.companyId || work.receipt.binding.instanceId !== input.instanceId) + return fail("form_ack_acceptance_missing"); + assertSlackFormReceiptHistory(row, work.receipt); + const refreshResponse = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, operation: { + kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + refreshAckReceipt: { leaseId: work.leaseId, receiptDigest: slackFormFinalizedReceiptDigest(work.receipt), appReceiptId: row.id } } })); + if (Object.keys(refreshResponse).join() !== "ackReceiptJws" || typeof refreshResponse.ackReceiptJws !== "string") + throw new SlackCloudReceiptError("form_signature_refresh_unconfirmed", "retry"); + const currentJws = refreshResponse.ackReceiptJws; + const refreshed = verifyRetainedSlackAckReceipt(currentJws); + if (!isDeepStrictEqual(refreshed, work.receipt)) return fail("form_signature_canonical_changed"); + row = await replaceSlackFormReceiptSignature(db, row, "formAck", currentJws); + input.onSnapshot?.(row); + work = parseSlackFormAckWork(row.result?.formAck); + if (!work.receipt) return fail("form_ack_acceptance_missing"); + const originals = readSlackFormOriginalProofs(row); + const operation = { kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + acceptedAckReceipt: currentJws }; + let response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, + operation: { ...operation, originalOpening: originals.opening } })); + const digest = slackFormFinalizedReceiptDigest(work.receipt); + if (response.acceptedReceiptDigest !== digest || typeof response.needsTerminalOriginal !== "boolean") return fail("form_ack_transfer_receipt_changed"); + if (response.needsTerminalOriginal) { + if (!originals.terminal) return fail("form_terminal_original_proof_missing"); + response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, + operation: { ...operation, terminalOriginal: originals.terminal } })); + if (response.acceptedReceiptDigest !== digest || response.needsTerminalOriginal !== false) return fail("form_ack_transfer_receipt_changed"); + } + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return fail("form_ack_transfer_custody_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const retainedResult = Object.fromEntries(Object.entries(current.result ?? {}).filter(([key]) => key !== "openingProof")); + const completed = await tx.update(chatActions).set({ result: { ...retainedResult, formAck: { ...work, phase: "gc_pending" }, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (completed.length !== 1) throw new SlackCloudReceiptError("form_ack_transfer_custody_changed", "retry"); + }); +} + +export async function reconcileSlackFormNormalAck(db: Db, input: { companyId: string; actionId: string; instanceId: string; connector: unknown; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"))).limit(1); + if (!row) return fail("form_ack_session_missing"); + if (row.companyId !== input.companyId) return fail("form_ack_company_changed"); + const work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase === "transferred") return; + if (work.phase === "pending") { + const binding = parseSlackFormBinding(row.payload.binding); + if (binding.companyId !== input.companyId || binding.instanceId !== input.instanceId || !input.connector || typeof input.connector !== "object") return fail("form_ack_instance_changed"); + const acknowledge = Reflect.get(input.connector, "acknowledgeEventsWithReceipts"); + if (typeof acknowledge !== "function") return fail("form_ack_receipts_unsupported"); + const response = record(await Reflect.apply(acknowledge, input.connector, [{ subject: "github-bot-events", companyId: row.companyId, + leaseId: work.leaseId, deliveryIds: [work.deliveryId] }])); + if (!Array.isArray(response.formReceipts) || response.formReceipts.length !== 1) return fail("form_ack_receipt_missing"); + const receipt = record(response.formReceipts[0]); + if (receipt.deliveryId !== work.deliveryId || typeof receipt.receiptJws !== "string") return fail("form_ack_receipt_changed"); + await db.transaction(tx => acceptSlackFormAckReceipt(tx, { companyId: input.companyId, instanceId: input.instanceId, leaseId: work.leaseId, receiptJws: String(receipt.receiptJws) })); + } + await transferSlackFormAckReceipt(db, input); +} + +function verifyRetainedSlackAckReceipt(compactJws: string): SlackFormFinalizedReceipt { + try { return verifySlackFormFinalizedReceipt({ compactJws }); } + catch (error) { + if (error instanceof Error && ["PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is not configured", "PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is invalid", + "Cloud runtime identity uses an unknown signing key"].includes(error.message)) throw new SlackCloudReceiptError("form_ack_verifier_unavailable", "retry"); + return fail("form_ack_receipt_invalid"); + } +} +async function releaseSlackAckFact(db: Db, row: typeof chatActions.$inferSelect, input: { companyId: string; actionId: string; instanceId: string; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + const work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase !== "gc_pending" || !work.receipt || work.receipt.binding.companyId !== input.companyId || work.receipt.binding.instanceId !== input.instanceId) + return fail("form_ack_gc_authority_changed"); + assertSlackFormReceiptHistory(row, work.receipt); + const digest = slackFormFinalizedReceiptDigest(work.receipt); + const response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, operation: { + kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + releaseAckFact: { leaseId: work.leaseId, receiptDigest: digest, appReceiptId: row.id } } })); + if (Object.keys(response).join() !== "releasedAckDigest" || response.releasedAckDigest !== digest) + throw new SlackCloudReceiptError("form_ack_gc_unconfirmed", "retry"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) + throw new SlackCloudReceiptError("form_ack_gc_changed", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...current.result, formAck: { ...work, phase: "transferred" }, nextAttemptAt: null }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw new SlackCloudReceiptError("form_ack_gc_changed", "retry"); + }); +} diff --git a/server/src/services/chat-channels/slack/form-canonical-state.test.ts b/server/src/services/chat-channels/slack/form-canonical-state.test.ts new file mode 100644 index 00000000000..7262b53fe82 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-canonical-state.test.ts @@ -0,0 +1,26 @@ +import { expect, it } from "vitest"; +import { parseSlackFormAckWork } from "./form-ack-work.js"; +import { parseSlackFormFinalizedReceipt } from "./ack-receipt.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; + +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: "test", binding, + operationId: "c".repeat(64), deliveryId: "slack_form_" + "c".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-06T00:00:00.000Z" }); +it("keeps GC debt explicitly distinct from completed ACK history", () => { + const pending = parseSlackFormAckWork({ version: 1, phase: "gc_pending", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }); + expect(pending.phase).toBe("gc_pending"); expect(pending.leaseId).toBe("exact-lease"); expect(pending.receipt).toEqual(receipt); + expect(parseSlackFormAckWork({ ...pending, phase: "transferred" }).phase).toBe("transferred"); +}); +it("GC cannot be constructed from absent proof or a different delivery", () => { + const work = { version: 1, phase: "gc_pending", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }; + expect(() => parseSlackFormAckWork({ ...work, receipt: null })).toThrow(); + expect(() => parseSlackFormAckWork({ ...work, deliveryId: "slack_form_" + "e".repeat(64) })).toThrow(); + expect(() => parseSlackFormAckWork({ ...work, phase: "released_by_absence" })).toThrow(); +}); +it("ordinary old pending and completed states remain closed and do not acquire new authority", () => { + expect(parseSlackFormAckWork({ version: 1, phase: "pending", leaseId: "old", deliveryId: receipt.deliveryId, receipt: null, receiptJws: null }).phase).toBe("pending"); + expect(() => parseSlackFormAckWork({ version: 1, phase: "pending", leaseId: "old", deliveryId: receipt.deliveryId, receipt, receiptJws: "a.b.c" })).toThrow(); +}); diff --git a/server/src/services/chat-channels/slack/form-delivery.ts b/server/src/services/chat-channels/slack/form-delivery.ts new file mode 100644 index 00000000000..c03a709b792 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-delivery.ts @@ -0,0 +1,85 @@ +import { retainSlackFormOpeningProof } from "./form-opening-proof.js"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, toolConnections, type Db } from "@paperclipai/db"; +import type { chatCredentialMutationLease } from "../../chat-credential-mutation-lease.js"; +import type { createChatCredentials } from "../credentials.js"; +import { SlackCloudReceiptError, type SlackCloudEvent } from "./cloud-event.js"; +import { slackFormSetupRecord, currentSlackFormOwnerBinding, parseSlackFormOperation, sameSlackFormBinding, lookupSlackFormDelivery, withSlackFormOpenContext, + type SlackFormDeliveryAuthority } from "./cloud-forms.js"; +import { slackFormConnectorIdentity } from "./cloud-consumer.js"; +import { acceptSlackFormClosed, acceptSlackFormNoEffect, acceptSlackFormTerminalProof, readSlackFormTerminalIntent, readSlackFormTerminalRequest } from "./form-session.js"; +import type { SlackFormProtocol } from "./form-owner.js"; +import { resolveSlackCloudSigningSecret } from "./cloud-signing-secret.js"; + +export function createSlackFormDeliveryOwner(db: Db, options: { + connector(): unknown; protocol: SlackFormProtocol; + withCredentialMutationLease: ReturnType; + resolveCredentials: ReturnType["resolveCredentials"]; + webhookUrl(publicId: string): string; + handleWebhook(publicId: string, request: Request, generation: number, acceptedAt: number): Promise; +}) { + return async (event: SlackCloudEvent): Promise => { + const identity = slackFormConnectorIdentity(options.connector()); + if (!identity) throw new SlackCloudReceiptError("form_connector_identity_unavailable", "attention"); + const packet = event.payload.slackFormOpen; + if (!packet || typeof packet !== "object" || Array.isArray(packet)) throw new SlackCloudReceiptError("form_receipt_invalid", "invalid"); + const original = parseSlackFormOperation(Reflect.get(packet, "operation")), binding = original.binding; + if (binding.instanceId !== identity.instanceId || Reflect.get(packet, "environment") !== identity.environment) throw new SlackCloudReceiptError("form_instance_changed", "attention"); + const [endpoint] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId))).limit(1); + if (!endpoint || endpoint.provider !== "slack") throw new SlackCloudReceiptError("form_endpoint_custody_missing", "attention"); + let authority: SlackFormDeliveryAuthority; + const local = currentSlackFormOwnerBinding(slackFormSetupRecord(endpoint.setup)); + if (local && sameSlackFormBinding(local, binding) && ["active", "verifying", "attention"].includes(endpoint.status)) { + const signingSecret = await options.withCredentialMutationLease(endpoint, async lease => { + const [current] = await db.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints) + .innerJoin(toolConnections, and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, chatEndpoints.companyId))) + .where(and(eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId))).limit(1); + const owner = current && currentSlackFormOwnerBinding(slackFormSetupRecord(current.endpoint.setup)); + if (!current || !owner || !sameSlackFormBinding(owner, binding) || !["active", "verifying", "attention"].includes(current.endpoint.status) + || current.connection.status !== "active" || !current.connection.enabled) throw new SlackCloudReceiptError("form_current_authority_changed", "attention"); + const secret = await resolveSlackCloudSigningSecret(options.resolveCredentials, current.endpoint, db, ["signingSecret"]); + await lease.assertOwned(); return secret; + }); + authority = { kind: "current", binding, ...identity, signingSecret }; + } else { + const [owner] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + sql`${chatActions.kind} in ('slack_form_owner','slack_form_session')`, sql`${chatActions.payload}->'binding' = ${JSON.stringify(binding)}::jsonb`, + sql`${chatActions.payload}->>'environment' = ${identity.environment}`)).limit(1); + if (!owner) throw new SlackCloudReceiptError("form_retained_owner_missing", "attention"); + authority = { kind: "retained", binding, ...identity }; + } + const lookup = await lookupSlackFormDelivery(event, authority, values => options.protocol({ subject: "github-bot-events", companyId: binding.companyId, + operation: { kind: "receipt", ...values } })); + const op = lookup.operation; + let terminal = await readSlackFormTerminalRequest(db, binding, op.id); + const actions = Reflect.get(lookup.originalPayload, "actions"); + const actionId = Array.isArray(actions) && actions.length === 1 && actions[0] && typeof actions[0] === "object" ? Reflect.get(actions[0], "action_id") : null; + if (typeof actionId !== "string") throw new SlackCloudReceiptError("form_issued_action_invalid", "attention"); + if (!terminal && op.closed?.kind === "closed") await db.transaction(tx => acceptSlackFormClosed(tx, op, actionId, identity.environment)); + else if (!terminal && op.rejection) await db.transaction(tx => acceptSlackFormNoEffect(tx, op, actionId, identity.environment)); + else if (!terminal && authority.kind === "current" && op.phase === "confirmed" && !op.closed) { + const rawBody = Reflect.get(packet, "rawBody"), timestamp = Reflect.get(packet, "timestamp"), signature = Reflect.get(packet, "signature"); + if (typeof rawBody !== "string" || typeof timestamp !== "string" || typeof signature !== "string") throw new SlackCloudReceiptError("form_original_invalid", "invalid"); + const url = new URL(options.webhookUrl(binding.publicId)); + if (!url.pathname.endsWith(`/api/chat-webhooks/${binding.publicId}/slack`)) throw new SlackCloudReceiptError("form_webhook_url_changed", "attention"); + const request = new Request(url, { method: "POST", + headers: { "content-type": "application/x-www-form-urlencoded", "x-slack-request-timestamp": timestamp, "x-slack-signature": signature }, body: Buffer.from(rawBody, "base64") }); + const response = await withSlackFormOpenContext(op, binding, + () => options.handleWebhook(binding.publicId, request, binding.runtimeGeneration, Date.parse(op.acceptedAt)), identity.environment); + if (!response.ok) throw new SlackCloudReceiptError("form_action_callback_failed", "retry"); + } + terminal = await readSlackFormTerminalRequest(db, binding, op.id); + if (!terminal) throw new SlackCloudReceiptError(op.phase === "confirmed" ? "form_view_active" : "form_resource_unresolved", op.phase === "confirmed" ? "retry" : "attention"); + const intent = await readSlackFormTerminalIntent(db, op); + if (!intent || intent.deliveryId !== event.id) throw new SlackCloudReceiptError("form_terminal_intent_missing", "attention"); + const proof = await options.protocol({ subject: "github-bot-events", companyId: binding.companyId, operation: terminal }); + if (!proof || typeof proof !== "object" || Reflect.get(proof, "instanceId") !== identity.instanceId || Reflect.get(proof, "environment") !== identity.environment) + throw new SlackCloudReceiptError("form_terminal_proof_invalid", "attention"); + const proven = parseSlackFormOperation(Reflect.get(proof, "operation")); + await db.transaction(async tx => { + await acceptSlackFormTerminalProof(tx, { intent, operation: Reflect.get(proof, "operation"), currentInstanceId: identity.instanceId }); + await retainSlackFormOpeningProof(tx, { companyId: binding.companyId, appReceiptId: intent.appReceiptId, operation: proven, + original: { rawBody: Reflect.get(packet, "rawBody"), timestamp: Reflect.get(packet, "timestamp"), signature: Reflect.get(packet, "signature") } }); + }); + }; +} diff --git a/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts b/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts new file mode 100644 index 00000000000..6d7d03da359 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts @@ -0,0 +1,39 @@ +import { createHash } from "node:crypto"; +import { expect, it } from "vitest"; +import { parseSlackFormPublicEnrollment } from "./cloud-forms.js"; +import { slackFormEnrollmentActionId, slackFormEnrollmentCompatibleActionIds, isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; + +const enrollment = parseSlackFormPublicEnrollment({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", + publicId: "p".repeat(43), connectionId: "connection", runtimeGeneration: 1, bindingId: "slack-event:" + "a".repeat(64), teamId: "TTEST", appId: "ATEST" }); +const legacyId = (value: unknown) => `slack-form-owner:${createHash("sha256").update(JSON.stringify(value)).digest("hex")}`; +it("uses a complete typed tuple independent of input and JSONB object order", () => { + const reversed = parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(enrollment).reverse())); + const postgresOrder = parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(enrollment).sort(([a], [b]) => Buffer.byteLength(a) - Buffer.byteLength(b) || a.localeCompare(b)))); + expect(slackFormEnrollmentActionId(reversed)).toBe(slackFormEnrollmentActionId(enrollment)); + expect(slackFormEnrollmentActionId(postgresOrder)).toBe(slackFormEnrollmentActionId(enrollment)); + expect(legacyId(reversed)).not.toBe(legacyId(enrollment)); +}); +it("recognizes only the exact known legacy producer orders for the same full authority tuple", () => { + const e = enrollment; + const connectionFirst = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, endpointId: e.endpointId, + connectionId: e.connectionId, publicId: e.publicId, runtimeGeneration: e.runtimeGeneration, teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + const publicFirst = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, endpointId: e.endpointId, + publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + for (const previous of [enrollment, connectionFirst, publicFirst]) + expect(isSlackFormEnrollmentActionId(legacyId(previous), parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(previous).reverse())))).toBe(true); + expect(slackFormEnrollmentCompatibleActionIds(enrollment)).toHaveLength(4); + expect(isSlackFormEnrollmentActionId(legacyId(Object.fromEntries(Object.entries(enrollment).reverse())), enrollment)).toBe(false); + expect(isSlackFormEnrollmentActionId("slack-form-owner:" + "0".repeat(64), enrollment)).toBe(false); +}); +it("every authority field is bound for canonical and compatible identities", () => { + const variants = { instanceId: "other-instance", companyId: "other-company", endpointId: "other-endpoint", publicId: "q".repeat(43), + connectionId: "other-connection", runtimeGeneration: 2, bindingId: "slack-event:" + "b".repeat(64), teamId: "TOTHER", appId: "AOTHER" }; + for (const [key, value] of Object.entries(variants)) { + const changed = parseSlackFormPublicEnrollment({ ...enrollment, [key]: value }); + for (const id of slackFormEnrollmentCompatibleActionIds(enrollment)) expect(isSlackFormEnrollmentActionId(id, changed)).toBe(false); + } +}); +it("does not accept a different schema version or extra authority keys", () => { + expect(() => parseSlackFormPublicEnrollment({ ...enrollment, version: 2 })).toThrow(); + expect(() => parseSlackFormPublicEnrollment({ ...enrollment, unexpected: "authority" })).toThrow(); +}); diff --git a/server/src/services/chat-channels/slack/form-enrollment-identity.ts b/server/src/services/chat-channels/slack/form-enrollment-identity.ts new file mode 100644 index 00000000000..b5f893449d3 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-enrollment-identity.ts @@ -0,0 +1,27 @@ +import { createHash } from "node:crypto"; +import { parseSlackFormPublicEnrollment, type SlackFormPublicEnrollment } from "./cloud-forms.js"; + +function actionId(value: readonly unknown[] | SlackFormPublicEnrollment): string { + return `slack-form-owner:${createHash("sha256").update(JSON.stringify(value)).digest("hex")}`; +} +export function slackFormEnrollmentActionId(input: SlackFormPublicEnrollment): string { + const e = parseSlackFormPublicEnrollment(input); + return actionId(["slack-form-owner/v2", e.version, e.instanceId, e.companyId, e.endpointId, e.publicId, + e.connectionId, e.runtimeGeneration, e.bindingId, e.teamId, e.appId]); +} +export function slackFormEnrollmentCompatibleActionIds(input: SlackFormPublicEnrollment): readonly string[] { + const e = parseSlackFormPublicEnrollment(input); + const projected: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, + bindingId: e.bindingId, teamId: e.teamId, appId: e.appId }; + const bindingDerived: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, connectionId: e.connectionId, publicId: e.publicId, runtimeGeneration: e.runtimeGeneration, + teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + const publicFirstBinding: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, + teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + return [slackFormEnrollmentActionId(e), actionId(projected), actionId(bindingDerived), actionId(publicFirstBinding)]; +} +export function isSlackFormEnrollmentActionId(providerActionId: unknown, enrollment: SlackFormPublicEnrollment): boolean { + return typeof providerActionId === "string" && slackFormEnrollmentCompatibleActionIds(enrollment).includes(providerActionId); +} diff --git a/server/src/services/chat-channels/slack/form-idle-history-sql.ts b/server/src/services/chat-channels/slack/form-idle-history-sql.ts new file mode 100644 index 00000000000..5644e40b31e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-idle-history-sql.ts @@ -0,0 +1,47 @@ +import { sql, type SQL, type SQLWrapper } from "drizzle-orm"; + +type Columns = { id: SQLWrapper; companyId: SQLWrapper; endpointId: SQLWrapper; kind: SQLWrapper; status: SQLWrapper; payload: SQLWrapper; result: SQLWrapper }; +function objectKeys(value: SQL, keys: readonly string[]): SQL { + return sql`jsonb_typeof(${value}) = 'object' AND ${value} ?& ${[...keys]}::text[] + AND (SELECT count(*) FROM jsonb_object_keys(CASE WHEN jsonb_typeof(${value})='object' THEN ${value} ELSE '{}'::jsonb END)) = ${keys.length}`; +} +export function completedSlackFormSessionSql(action: Columns): SQL { + const p = sql`${action.payload}`, r = sql`${action.result}`, work = sql`(${r}->'formAck')`, receipt = sql`(${work}->'receipt')`, + binding = sql`(${p}->'binding')`, proof = sql`(${r}->'cloudProofAcceptance')`, intent = sql`(${r}->'terminalIntent')`, view = sql`(${intent}->'view')`; + const digest = sql`encode(sha256(convert_to('[' || to_json(${view}->>'id')::text || ',' || to_json(${view}->>'hash')::text || ',' || to_json(${view}->>'externalId')::text || ']', 'UTF8')), 'hex')`; + return sql`coalesce(${action.kind}='slack_form_session' AND (${action.status} IN ('processed','cancelled') OR (${action.status}='failed' AND ${receipt}->>'outcome'='no_effect')) + AND ${r}->'nextAttemptAt' = 'null'::jsonb + AND ${objectKeys(work, ['version','phase','leaseId','deliveryId','receiptJws','receipt'])} + AND ${work}->'version' = '1'::jsonb AND ${work}->>'phase'='transferred' + AND jsonb_typeof(${work}->'receiptJws')='string' AND length(${work}->>'receiptJws') BETWEEN 1 AND 12000 + AND ${work}->>'receiptJws' ~ '^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$' + AND ${objectKeys(receipt, ['version','kind','binding','environment','operationId','deliveryId','appReceiptId','grantId','publicationId','interactionId','userId','viewDigest','terminalDigest','outcome','finalizedAt'])} + AND ${receipt}->'version'='1'::jsonb AND ${receipt}->>'kind'='slack_form_finalized' + AND ${objectKeys(binding, ['version','instanceId','companyId','endpointId','publicId','connectionId','runtimeGeneration','teamId','appId','eventBindingId','credentialFingerprint'])} + AND ${binding}->'version'='1'::jsonb AND jsonb_typeof(${binding}->'runtimeGeneration')='number' + AND ${binding}->>'runtimeGeneration' ~ '^[1-9][0-9]{0,15}$' + AND (length(${binding}->>'runtimeGeneration') < 16 OR ${binding}->>'runtimeGeneration' <= '9007199254740991') + AND ${binding}->>'publicId' ~ '^[A-Za-z0-9_-]{43}$' AND ${binding}->>'teamId' ~ '^T[A-Z0-9]+$' AND ${binding}->>'appId' ~ '^A[A-Z0-9]+$' + AND ${binding}->>'eventBindingId' ~ '^slack-event:[a-f0-9]{64}$' AND ${binding}->>'credentialFingerprint' ~ '^[a-f0-9]{64}$' + AND NOT EXISTS (SELECT 1 FROM unnest(ARRAY['instanceId','companyId','endpointId','publicId','connectionId','teamId','appId','eventBindingId','credentialFingerprint']) key + WHERE jsonb_typeof(${binding}->key) IS DISTINCT FROM 'string' OR length(${binding}->>key) NOT BETWEEN 1 AND 200) + AND ${objectKeys(proof, ['version','binding','operationId','appReceiptId','outcome','terminalDigest','acceptedAt'])} + AND ${proof}->'version'='1'::jsonb AND ${receipt}->'binding'=${binding} AND ${proof}->'binding'=${binding} + AND ${proof}->>'acceptedAt' ~ '^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9][.][0-9]{3}Z$' + AND ${receipt}->>'finalizedAt' ~ '^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9][.][0-9]{3}Z$' + AND ${receipt}->>'environment' IN ('development','test','staging','production') + AND ${binding}->>'companyId'=${action.companyId}::text AND ${binding}->>'endpointId'=${action.endpointId}::text + AND ${receipt}->>'appReceiptId'=${action.id}::text AND ${proof}->>'appReceiptId'=${action.id}::text AND ${intent}->>'appReceiptId'=${action.id}::text + AND ${receipt}->'environment'=${p}->'environment' + AND ${receipt}->'operationId'=${p}->'operationId' AND ${receipt}->'operationId'=${intent}->'operationId' AND ${receipt}->'operationId'=${proof}->'operationId' + AND ${receipt}->>'operationId' ~ '^[a-f0-9]{64}$' AND ${receipt}->>'deliveryId'='slack_form_'||(${receipt}->>'operationId') + AND ${receipt}->'deliveryId'=${intent}->'deliveryId' AND ${receipt}->'deliveryId'=${work}->'deliveryId' + AND ${receipt}->'grantId'=${p}->'grantId' AND ${receipt}->'publicationId'=${p}->'publicationId' AND ${receipt}->'interactionId'=${p}->'interactionId' + AND ${receipt}->'userId'=${p}->'userId' AND ${receipt}->'outcome'=${intent}->'outcome' AND ${receipt}->'outcome'=${proof}->'outcome' + AND ${receipt}->'terminalDigest'=${proof}->'terminalDigest' AND ${receipt}->>'terminalDigest' ~ '^[a-f0-9]{64}$' + AND ${r}->>'phase'=CASE WHEN ${receipt}->>'outcome'='no_effect' THEN 'not_opened' ELSE ${receipt}->>'outcome' END + AND CASE WHEN ${intent}->>'outcome'='no_effect' THEN ${view}='null'::jsonb AND ${receipt}->'viewDigest'='null'::jsonb + WHEN ${intent}->>'outcome' IN ('submitted','closed') THEN ${objectKeys(view, ['id','hash','externalId'])} + AND jsonb_typeof(${view}->'id')='string' AND jsonb_typeof(${view}->'hash')='string' AND jsonb_typeof(${view}->'externalId')='string' + AND ${receipt}->>'viewDigest'=${digest} ELSE false END, false)`; +} diff --git a/server/src/services/chat-channels/slack/form-idle-history.ts b/server/src/services/chat-channels/slack/form-idle-history.ts new file mode 100644 index 00000000000..da10215d85b --- /dev/null +++ b/server/src/services/chat-channels/slack/form-idle-history.ts @@ -0,0 +1,85 @@ +import { slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { isDeepStrictEqual } from "node:util"; +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { matchSlackConfiguredCoverage, type SlackConfiguredRequirement } from "./configured-coverage.js"; +import type { chatActions } from "@paperclipai/db"; +import { parseSlackFormBinding, parseSlackFormPublicEnrollment, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormAckWork, assertSlackFormReceiptHistory } from "./form-ack-work.js"; + +export type SlackFormHistoryRow = Pick; +export type SlackFormHistoryCoverage = { requirements: readonly SlackConfiguredRequirement[]; verified: VerifiedIdleCapabilities; + expected: { instanceId: string; generation: string; environment: string }; currentOwners?: ReadonlyMap }; + +/** Prepare bounded current coverage once; history pages retain no protocol rows. + * Session-related actions are supplied from the same page or indexed DB joins. + */ +export function createSlackFormHistoryClassifier(coverage?: SlackFormHistoryCoverage) { + const matched = coverage ? matchSlackConfiguredCoverage(coverage.requirements, coverage.verified, coverage.expected) : undefined; + const endpoints = new Set(matched?.endpointIds), connections = new Set(matched?.connectionIds); + const bindings = new Map(coverage?.requirements.map(requirement => [ + `${requirement.binding.companyId}:${requirement.binding.endpointId}`, requirement.binding])); + return (rows: readonly SlackFormHistoryRow[], companyId: string, endpointId: string, check?: () => void): readonly string[] => { + const scoped: SlackFormHistoryRow[] = [], byId = new Map(); + for (const row of rows) { + check?.(); + if (row.companyId !== companyId || row.endpointId !== endpointId) continue; + const prior = byId.get(row.id); + if (prior && !isDeepStrictEqual(prior, row)) return []; + if (!prior) { scoped.push(row); byId.set(row.id, row); } + } + const completed = new Set(); + for (const row of scoped) { + check?.(); + if (!["slack_form_owner", "slack_form_grant"].includes(row.kind) || row.result?.nextAttemptAt !== null) continue; + try { + if (row.result?.phase === "transferred") { + if (slackFormRetirementHistoryReceipt(row)) completed.add(row.id); + continue; + } + const binding = parseSlackFormBinding(row.payload.binding); + const current = bindings.get(`${companyId}:${endpointId}`); + if (binding.companyId !== companyId || binding.endpointId !== endpointId || !coverage + || !endpoints.has(endpointId) || !connections.has(binding.connectionId) + || row.payload.environment !== coverage.expected.environment || !current || !sameSlackFormBinding(current, binding)) continue; + if (row.kind === "slack_form_owner" && row.result?.phase === "enrolled") { + // Owner uniqueness was certified separately. A new or changed owner + // between keyset passes remains unclassified for the fresh shared gate. + if (!coverage.currentOwners || isDeepStrictEqual(coverage.currentOwners.get(row.id), row)) completed.add(row.id); + } else if (row.kind === "slack_form_grant" && row.result?.phase === "published") completed.add(row.id); + } catch { continue; } + } + for (const session of scoped) { + check?.(); + if (session.kind !== "slack_form_session" || !["processed", "cancelled", "failed"].includes(session.status)) continue; + try { + const binding = parseSlackFormBinding(session.payload.binding), work = parseSlackFormAckWork(session.result?.formAck); + if (session.status === "failed" && work.receipt?.outcome !== "no_effect") continue; + if (binding.companyId !== companyId || binding.endpointId !== endpointId || work.phase !== "transferred" || !work.receiptJws || !work.receipt + || session.result?.nextAttemptAt !== null) continue; + assertSlackFormReceiptHistory(session, work.receipt); + completed.add(session.id); + for (const [kind, id] of [["question_form_open", session.payload.openActionRowId], ["question_form_submit", session.payload.submitActionRowId]] as const) { + const action = typeof id === "string" ? byId.get(id) : undefined; + if (action?.kind === kind && action.status === "processed" && action.conversationId === session.conversationId + && action.principalId === session.principalId && action.payload.interactionId === session.payload.interactionId + && action.payload.publicationId === session.payload.publicationId) completed.add(action.id); + } + } catch { continue; } + } + return [...completed].sort(); + }; +} + +export function completedSlackFormHistoryActionIds(rows: readonly SlackFormHistoryRow[], companyId: string, endpointId: string, coverage?: SlackFormHistoryCoverage): readonly string[] { + return createSlackFormHistoryClassifier(coverage)(rows, companyId, endpointId); +} + +export function completedSlackFormSessionReceipt(session: Pick) { + if (session.kind !== "slack_form_session" || !["processed", "cancelled", "failed"].includes(session.status)) return null; + const binding = parseSlackFormBinding(session.payload.binding), work = parseSlackFormAckWork(session.result?.formAck); + if (session.status === "failed" && work.receipt?.outcome !== "no_effect") return null; + if (binding.companyId !== session.companyId || binding.endpointId !== session.endpointId || work.phase !== "transferred" + || !work.receiptJws || !work.receipt || session.result?.nextAttemptAt !== null) return null; + assertSlackFormReceiptHistory(session, work.receipt); + return work.receipt; +} diff --git a/server/src/services/chat-channels/slack/form-normal-ack.ts b/server/src/services/chat-channels/slack/form-normal-ack.ts new file mode 100644 index 00000000000..8a3432b113e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-normal-ack.ts @@ -0,0 +1,37 @@ +import type { Db } from "@paperclipai/db"; +import type { PaperclipCloudConnector } from "../../paperclip-cloud-connector.js"; +import { stageSlackFormNormalAck, acceptSlackFormAckReceipt } from "./form-ack-work.js"; +import { slackFormConnectorIdentity } from "./cloud-consumer.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; + +export async function acknowledgeSlackFormDeliveries(db: Db, input: { connector: PaperclipCloudConnector; + values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }; companies: ReadonlyMap }): Promise { + const formIds = input.values.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)); + const values = input.values; + if (!formIds.length) await input.connector.acknowledgeEvents(values); + else { + const identity = slackFormConnectorIdentity(input.connector), detailed = Reflect.get(input.connector, "acknowledgeEventsWithReceipts"); + if (!identity || typeof detailed !== "function") throw new SlackCloudReceiptError("form_ack_receipts_unsupported", "attention"); + for (const deliveryId of formIds) { + const companyId = input.companies.get(deliveryId); + if (!companyId) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + await db.transaction(tx => stageSlackFormNormalAck(tx, { companyId, instanceId: identity.instanceId, leaseId: values.leaseId, deliveryIds: [deliveryId] })); + } + const response: unknown = await Reflect.apply(detailed, input.connector, [values]); + if (!response || typeof response !== "object" || Array.isArray(response) || !Array.isArray(Reflect.get(response, "formReceipts"))) + throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const receipts: unknown = Reflect.get(response, "formReceipts"); + if (!Array.isArray(receipts) || receipts.length !== formIds.length) throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const seen = new Set(); + for (const value of receipts) { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const deliveryId: unknown = Reflect.get(value, "deliveryId"), receiptJws: unknown = Reflect.get(value, "receiptJws"); + if (typeof deliveryId !== "string" || !formIds.includes(deliveryId) || seen.has(deliveryId) || typeof receiptJws !== "string") + throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + seen.add(deliveryId); + const companyId = input.companies.get(deliveryId); + if (!companyId) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + await db.transaction(tx => acceptSlackFormAckReceipt(tx, { companyId, instanceId: identity.instanceId, leaseId: values.leaseId, receiptJws })); + } + } +} diff --git a/server/src/services/chat-channels/slack/form-open-callback.ts b/server/src/services/chat-channels/slack/form-open-callback.ts new file mode 100644 index 00000000000..173cedc4593 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-open-callback.ts @@ -0,0 +1,43 @@ +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; +import { and, eq } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { sameSlackFormBinding, type SlackFormOpenContext } from "./cloud-forms.js"; +import { stageSlackFormSession, confirmSlackFormSessionUpdate } from "./form-session.js"; + +type Tx = Parameters[0]>[0]; +type Scope = { companyId: string; endpointId: string; conversationId: string; principalId: string }; +export async function stageSlackFormOpenCallback(tx: Tx, input: Scope & { + context: SlackFormOpenContext; resolved: ResolvedChatQuestionFormOpen; lease?: CredentialMutationLeaseGuard; + attemptProviderActionId: string; attemptActionId: string | null; +}): Promise<{ kind: "ready"; attemptActionId: string; resolved: ResolvedChatQuestionFormOpen } | { kind: "duplicate" }> { + const { context, lease } = input; + if (!lease || !context.environment) throw new SlackCloudReceiptError("form_credential_lease_unavailable", "retry"); + const held = await stageSlackFormSession(tx, { operation: context.operation, currentBinding: context.currentBinding, + principalId: input.principalId, resolved: input.resolved, lease, environment: context.environment }); + context.sessionActionId = held.actionId; + if (held.kind !== "update") return { kind: "duplicate" }; + if (input.attemptActionId) return { kind: "ready", attemptActionId: input.attemptActionId, resolved: input.resolved }; + const [attempt] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, input.endpointId), eq(chatActions.conversationId, input.conversationId), + eq(chatActions.principalId, input.principalId), eq(chatActions.kind, "question_form_open_attempt"), + eq(chatActions.providerActionId, input.attemptProviderActionId))).for("update"); + if (!attempt || attempt.status !== "processing") throw new SlackCloudReceiptError("form_open_attempt_unresolved", "attention"); + return { kind: "ready", attemptActionId: attempt.id, resolved: input.resolved }; +} +export async function confirmSlackFormOpenCallback(tx: Tx, input: Omit & { context: SlackFormOpenContext }): Promise { + const { context } = input; + if (!context.sessionActionId || !context.updatedView || !context.operation.view) throw new SlackCloudReceiptError("form_update_receipt_unavailable", "retry"); + const updated = await confirmSlackFormSessionUpdate(tx, { companyId: input.companyId, endpointId: input.endpointId, + principalId: input.principalId, operationId: context.operation.id }, context.operation.view, context.updatedView.hash); + if (!updated) throw new SlackCloudReceiptError("form_update_settlement_changed", "retry"); +} + +export function isRetryableSlackFormUpdate(context: SlackFormOpenContext | undefined, error: unknown): boolean { + return Boolean(context?.sessionActionId && context.updateAttempted === true && !context.updatedView + && context.operation.phase === "confirmed" && context.operation.view && !context.operation.closed && !context.operation.rejection + && sameSlackFormBinding(context.operation.binding, context.currentBinding) + && classifyChatPublicationError(error, 1).kind === "retry"); +} diff --git a/server/src/services/chat-channels/slack/form-opening-proof.ts b/server/src/services/chat-channels/slack/form-opening-proof.ts new file mode 100644 index 00000000000..903fa7dd566 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-opening-proof.ts @@ -0,0 +1,63 @@ +import { createHash } from "node:crypto"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import type { StoredSecretVersionMaterial } from "../../../secrets/types.js"; +import { createLocalEncryptedCipher } from "../../../secrets/local-encrypted-provider.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormOperation } from "./cloud-forms.js"; +type Tx = Pick; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return fail("form_opening_proof_invalid"); + return Object.fromEntries(Object.entries(value)); +} +function opening(value: unknown): { rawBody: string; timestamp: string; signature: string } { + const raw = object(value); + if (Object.keys(raw).sort().join() !== "rawBody,signature,timestamp" || typeof raw.rawBody !== "string" || raw.rawBody.length > 1398104 + || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(raw.rawBody) + || typeof raw.timestamp !== "string" || !/^[0-9]{1,16}$/.test(raw.timestamp) || typeof raw.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(raw.signature)) + return fail("form_opening_proof_invalid"); + return { rawBody: raw.rawBody, timestamp: raw.timestamp, signature: raw.signature }; +} +const digest = (raw: ReturnType) => createHash("sha256").update(JSON.stringify([raw.rawBody, raw.timestamp, raw.signature])).digest("hex"); +export function readSlackFormOriginalProofs(row: Pick): { opening: ReturnType; terminal: ReturnType | null } { + const saved = object(row.result?.openingProof); + if (row.kind !== "slack_form_session" || saved.version !== 1 || Object.keys(saved).sort().join() !== "material,version") return fail("form_opening_proof_invalid"); + let value: Record; + try { value = object(JSON.parse(createLocalEncryptedCipher("read").decrypt(object(saved.material)))); } + catch { return fail("form_opening_cipher_unavailable"); } + const binding = parseSlackFormBinding(row.payload.binding); + if (value.version !== 1 || value.kind !== "slack_form_original_opening" || value.appReceiptId !== row.id || value.companyId !== row.companyId || value.endpointId !== row.endpointId + || value.operationId !== row.payload.operationId || value.environment !== row.payload.environment + || !sameSlackFormBinding(parseSlackFormBinding(value.binding), binding)) return fail("form_opening_cipher_binding_changed"); + const raw = opening(value.original); + if (digest(raw) !== row.payload.requestDigest) return fail("form_opening_cipher_binding_changed"); + if (value.terminal === undefined) return fail("form_terminal_original_proof_missing"); + return { opening: raw, terminal: value.terminal === null ? null : opening(value.terminal) }; +} +export async function retainSlackFormOpeningProof(tx: Tx, input: { companyId: string; appReceiptId: string; operation: SlackFormOperation; original: unknown }): Promise { + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, input.appReceiptId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, input.operation.binding.endpointId), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row || row.payload.operationId !== input.operation.id || row.payload.requestDigest !== input.operation.requestDigest + || !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), input.operation.binding) || !row.result?.cloudProofAcceptance) + return fail("form_opening_proof_authority_changed"); + const original = opening(input.original); + if (digest(original) !== input.operation.requestDigest) return fail("form_opening_proof_changed"); + const terminal = input.operation.closed ? opening({ rawBody: input.operation.closed.evidence.rawBody, + timestamp: input.operation.closed.evidence.timestamp, signature: input.operation.closed.evidence.signature }) : null; + if (terminal && digest(terminal) !== input.operation.closed?.requestDigest) return fail("form_terminal_original_proof_changed"); + if (row.result.openingProof !== undefined) { + if (!isDeepStrictEqual(readSlackFormOriginalProofs(row), { opening: original, terminal })) return fail("form_opening_proof_changed"); + return; + } + let material: StoredSecretVersionMaterial; + try { material = createLocalEncryptedCipher("write").encrypt(JSON.stringify({ version: 1, kind: "slack_form_original_opening", appReceiptId: row.id, companyId: row.companyId, + endpointId: row.endpointId, operationId: input.operation.id, environment: row.payload.environment, binding: input.operation.binding, original, terminal })); } + catch { return fail("form_opening_cipher_unavailable"); } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const changed = await tx.update(chatActions).set({ result: { ...row.result, openingProof: { version: 1, material } }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (changed.length !== 1) return fail("form_opening_proof_custody_changed"); +} diff --git a/server/src/services/chat-channels/slack/form-owner-rearm.test.ts b/server/src/services/chat-channels/slack/form-owner-rearm.test.ts new file mode 100644 index 00000000000..b010d4bc6cb --- /dev/null +++ b/server/src/services/chat-channels/slack/form-owner-rearm.test.ts @@ -0,0 +1,21 @@ +import { expect, it } from "vitest"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { slackFormAttentionCanRearm } from "./form-owner.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const { eventBindingId, credentialFingerprint, ...identity } = binding; +const enrollment = { ...identity, bindingId: eventBindingId }; +it("allows a recoverable cipher failure only under the exact current configured authority", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding, enrollment })).toBe(true); +}); +it("refuses foreign company and replaced instance during explicit configuration rearm", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, companyId: "foreign" }, enrollment })).toBe(false); + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, instanceId: "foreign" }, enrollment })).toBe(false); +}); +it("does not rewrite stale generations or known authority conflicts into retryable work", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, runtimeGeneration: 2 }, enrollment })).toBe(false); + expect(slackFormAttentionCanRearm({ code: "form_owner_authority_unavailable", binding, enrollment })).toBe(false); +}); +it("unknown malformed attention stays blocked without aborting other eligible retries", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { malformed: true }, enrollment })).toBe(false); +}); diff --git a/server/src/services/chat-channels/slack/form-owner.ts b/server/src/services/chat-channels/slack/form-owner.ts new file mode 100644 index 00000000000..0f747eeffe5 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-owner.ts @@ -0,0 +1,320 @@ +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; +import { slackFormEnrollmentActionId, slackFormEnrollmentCompatibleActionIds } from "./form-enrollment-identity.js"; +import { acceptSlackFormRetirementLocally, transferSlackFormRetirement } from "./form-retirement-work.js"; +import { reconcileSlackFormNormalAck } from "./form-ack-work.js"; +import { HttpError } from "../../../errors.js"; +import { createPaperclipCloudConnector, paperclipCloudConnectorConfigFromEnv, PaperclipCloudConnectorError } from "../../paperclip-cloud-connector.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatPublications, issueThreadInteractions, toolConnections, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { resolveChatQuestionFormOpen, type ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import type { IssueThreadInteraction } from "@paperclipai/shared"; +import { chatInteractionForm } from "../../chat-interaction-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { slackFormSetupRecord, parseSlackFormBinding, parseSlackFormPublicEnrollment, projectSlackIssuedFormGrant, + type SlackFormPublicEnrollment, type SlackFormBinding } from "./cloud-forms.js"; +import { slackFormConnectorIdentity, type SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +type Tx = Pick; +export type SlackFormProtocol = (input: { subject: string; companyId: string; operation: Record }) => Promise; +function attention(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return attention("form_owner_response_invalid"); + return value as Record; +} +function matches(binding: SlackFormBinding, enrollment: SlackFormPublicEnrollment) { + const { credentialFingerprint: _, eventBindingId, ...identity } = binding; + return eventBindingId === enrollment.bindingId && isDeepStrictEqual({ ...identity, bindingId: eventBindingId }, enrollment); +} +export function projectSlackFormPublicEnrollment(endpoint: typeof chatEndpoints.$inferSelect): SlackFormPublicEnrollment | null { + const setup = slackFormSetupRecord(endpoint.setup), descriptor = setup.slackCloudEvents; + if (!descriptor || typeof descriptor !== "object" || Array.isArray(descriptor) + || typeof setup.slackCloudInstanceId !== "string") return null; + return parseSlackFormPublicEnrollment({ version: 1, instanceId: setup.slackCloudInstanceId, companyId: endpoint.companyId, + endpointId: endpoint.id, publicId: endpoint.publicId, connectionId: endpoint.connectionId, + runtimeGeneration: Reflect.get(descriptor, "runtimeGeneration"), bindingId: Reflect.get(descriptor, "bindingId"), + teamId: Reflect.get(descriptor, "teamId"), appId: Reflect.get(descriptor, "appId") }); +} +export async function stageSlackFormEnrollment(tx: Tx, input: { enrollment: SlackFormPublicEnrollment; subject: string; environment: string }): Promise { + const enrollment = parseSlackFormPublicEnrollment(input.enrollment); + if (!input.subject || !["development", "test", "staging", "production"].includes(input.environment)) return attention("form_owner_subject_invalid"); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, enrollment.endpointId), eq(chatEndpoints.companyId, enrollment.companyId))).for("update"); + const setup: Record | undefined = endpoint ? slackFormSetupRecord(endpoint.setup) : undefined; + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status) + || endpoint.publicId !== enrollment.publicId || endpoint.connectionId !== enrollment.connectionId + || setup?.runtimeGeneration !== enrollment.runtimeGeneration || setup?.slackCloudInstanceId !== enrollment.instanceId + || setup?.slackCloudSubject !== input.subject || !isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment)) return attention("form_owner_current_enrollment_changed"); + const providerActionId = slackFormEnrollmentActionId(enrollment); + const existingRows = await tx.select().from(chatActions).where(and(eq(chatActions.endpointId, endpoint.id), eq(chatActions.companyId, endpoint.companyId), + inArray(chatActions.providerActionId, [...slackFormEnrollmentCompatibleActionIds(enrollment)]))).for("update"); + if (existingRows.length > 1) return attention("form_owner_identity_ambiguous"); + const [existing] = existingRows; + const payload = { enrollment, subject: input.subject, environment: input.environment }; + if (existing) { + if (existing.kind !== "slack_form_owner" || !Object.keys(payload).every(key => isDeepStrictEqual(existing.payload[key], Reflect.get(payload, key)))) return attention("form_owner_custody_changed"); + return existing.id; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: endpoint.companyId, endpointId: endpoint.id, kind: "slack_form_owner", providerActionId, + payload, status: "issued", result: { phase: "staged", nextAttemptAt: new Date().toISOString() } }).returning({ id: chatActions.id }); + return row.id; +} +export async function reconcileSlackFormEnrollment(db: Db, input: { actionId: string; identity: SlackFormConnectorIdentity; + protocol: SlackFormProtocol; resolveBotToken(endpointId: string): Promise; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void }): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.kind, "slack_form_owner"))).limit(1); + if (!row || row.result?.phase === "transferred") return; + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), subject = row.payload.subject; + if (enrollment.companyId !== row.companyId || enrollment.endpointId !== row.endpointId || enrollment.instanceId !== input.identity.instanceId + || row.payload.environment !== input.identity.environment || typeof subject !== "string" || !subject) return attention("form_owner_identity_changed"); + if (row.result?.phase === "retirement_accepted" || row.result?.phase === "retirement_gc_pending") return transferSlackFormRetirement(db, row, input.identity, input.protocol, input.onSnapshot); + const [endpoint] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, row.endpointId), eq(chatEndpoints.companyId, row.companyId))).limit(1); + const endpointSetup: Record | undefined = endpoint ? slackFormSetupRecord(endpoint.setup) : undefined; + const live = row.result?.desiredActive !== false && endpoint && ["active", "verifying", "attention"].includes(endpoint.status) && endpointSetup?.runtimeGeneration === enrollment.runtimeGeneration + && endpointSetup?.slackCloudInstanceId === enrollment.instanceId && endpointSetup?.slackCloudSubject === subject && isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment); + let binding: SlackFormBinding; + if (row.payload.binding !== undefined) binding = parseSlackFormBinding(row.payload.binding); + else { + const response = object(await input.protocol({ subject, companyId: row.companyId, operation: live + ? { kind: "enroll", enrollment, botToken: await input.resolveBotToken(row.endpointId) } + : { kind: "owner-receipt", enrollment } })); + if (!live && response.kind === "retired-enrollment") { + if (typeof response.retirementReceiptJws !== "string") throw new SlackCloudReceiptError("form_retirement_receipt_missing", "retry"); + await acceptSlackFormRetirementLocally(db, row, input.identity, response.retirementReceiptJws); + return; + } + binding = parseSlackFormBinding(response.binding); + } + if (!matches(binding, enrollment)) return attention("form_owner_binding_changed"); + if (row.payload.binding === undefined) { + await db.transaction(async tx => { + const [current] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, row.endpointId), eq(chatEndpoints.companyId, row.companyId))).for("update"); + const [intent] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!intent || !isDeepStrictEqual(intent.payload, row.payload) || !isDeepStrictEqual(intent.result, row.result)) throw new SlackCloudReceiptError("form_owner_intent_changed", "retry"); + const currentSetup: Record | undefined = current ? slackFormSetupRecord(current.setup) : undefined; + const currentLive = intent.result?.desiredActive !== false && current && ["active", "verifying", "attention"].includes(current.status) && currentSetup?.runtimeGeneration === enrollment.runtimeGeneration + && currentSetup?.slackCloudInstanceId === enrollment.instanceId && currentSetup?.slackCloudSubject === subject && isDeepStrictEqual(projectSlackFormPublicEnrollment(current), enrollment); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ payload: { ...intent.payload, binding }, result: { phase: currentLive ? "enrolled" : "retired_cleanup", nextAttemptAt: currentLive ? null : new Date().toISOString() }, updatedAt: new Date() }).where(eq(chatActions.id, intent.id)); + if (currentLive) { + await tx.update(chatEndpoints).set({ setup: sql`jsonb_set(${chatEndpoints.setup}, '{slackCloudForms}', ${JSON.stringify(binding)}::jsonb)` }).where(and(eq(chatEndpoints.id, current.id), eq(chatEndpoints.companyId, current.companyId))); + await notifyChatPublicationWork(tx); + } + }); + return; + } + if (!live) { + await input.protocol({ subject, companyId: row.companyId, operation: { kind: "retire", binding } }); + const response = object(await input.protocol({ subject, companyId: row.companyId, operation: { kind: "cleanup", binding } })); + if (response.cleaned !== true) throw new SlackCloudReceiptError("form_owner_retained_resource", "retry"); + if (response.instanceId !== input.identity.instanceId || response.environment !== input.identity.environment + || !isDeepStrictEqual(response.binding, binding)) return attention("form_owner_retirement_receipt_changed"); + if (typeof response.retirementReceiptJws !== "string") throw new SlackCloudReceiptError("form_retirement_receipt_missing", "retry"); + await acceptSlackFormRetirementLocally(db, row, input.identity, response.retirementReceiptJws); + } +} +export async function stageSlackFormGrant(tx: Tx, input: { binding: SlackFormBinding; environment: string; subject: string; resolved: ResolvedChatQuestionFormOpen }) { + const grant = await projectSlackIssuedFormGrant(tx, input.binding, input.resolved); + if (!grant) return attention("form_grant_issued_authority_changed"); + const [owner] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.binding.companyId), eq(chatActions.endpointId, input.binding.endpointId), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(input.binding)}::jsonb`, sql`${chatActions.result}->>'phase' = 'enrolled'`)).for("update"); + if (!owner || owner.payload.environment !== input.environment || owner.payload.subject !== input.subject) return attention("form_grant_owner_changed"); + const providerActionId = `slack-form-grant:${grant.grantId}`; + const [existing] = await tx.select().from(chatActions).where(and(eq(chatActions.endpointId, input.binding.endpointId), eq(chatActions.companyId, input.binding.companyId), eq(chatActions.providerActionId, providerActionId))).for("update"); + if (existing) { if (!isDeepStrictEqual(existing.payload.grant, grant)) return attention("form_grant_custody_changed"); return existing.id; } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: input.binding.companyId, endpointId: input.binding.endpointId, conversationId: grant.conversationId, + kind: "slack_form_grant", providerActionId, payload: { grant, binding: input.binding, subject: input.subject, environment: input.environment }, + status: "issued", result: { phase: "staged", nextAttemptAt: new Date().toISOString() } }).returning({ id: chatActions.id }); + return row.id; +} +export async function publishSlackFormGrant(db: Db, actionId: string, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol) { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, actionId), eq(chatActions.kind, "slack_form_grant"))).limit(1); + if (!row || ["published", "transferred"].includes(String(row.result?.phase))) return; + const binding = parseSlackFormBinding(row.payload.binding); + if (binding.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || binding.companyId !== row.companyId + || binding.endpointId !== row.endpointId || typeof row.payload.subject !== "string") return attention("form_grant_identity_changed"); + const response = object(await protocol({ subject: row.payload.subject, companyId: row.companyId, operation: { kind: "grant", grant: row.payload.grant } })); + if (!isDeepStrictEqual(response.grant, row.payload.grant)) return attention("form_grant_receipt_changed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || !isDeepStrictEqual(current.payload, row.payload)) throw new SlackCloudReceiptError("form_grant_intent_changed", "retry"); + if (current.result?.phase === "transferred") return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "processed", result: { phase: "published", nextAttemptAt: null }, updatedAt: new Date() }).where(eq(chatActions.id, row.id)); + }); +} + +export async function stageSlackFormOwnerConfiguration(tx: Tx, endpointId: string, active: boolean, subject: string, identity: SlackFormConnectorIdentity) { + const [endpoint] = await tx.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)).for("update"); + if (!endpoint || endpoint.provider !== "slack") return; + const setup = slackFormSetupRecord(endpoint.setup); + if (active) { + const enrollment = projectSlackFormPublicEnrollment(endpoint); + if (!enrollment || enrollment.instanceId !== identity.instanceId || setup.slackCloudSubject !== subject) return attention("form_owner_event_enrollment_unsettled"); + await stageSlackFormEnrollment(tx, { enrollment, subject, environment: identity.environment }); + await rearmSlackFormAttention(tx, { enrollment, subject, environment: identity.environment }); + const previous = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"))).for("update"); + for (const row of previous) { + const prior = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (prior.companyId !== endpoint.companyId || prior.endpointId !== endpoint.id || prior.instanceId !== identity.instanceId + || row.payload.environment !== identity.environment || typeof row.payload.subject !== "string" || !row.payload.subject + || prior.runtimeGeneration >= enrollment.runtimeGeneration || ["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase))) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "issued", result: { ...row.result, desiredActive: false, + phase: row.payload.binding ? "retired_cleanup" : "staged", nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)); + } + return; + } + const [connection] = await tx.select().from(toolConnections).where(and(eq(toolConnections.id, endpoint.connectionId), eq(toolConnections.companyId, endpoint.companyId))).limit(1); + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"))).for("update"); + for (const row of rows) { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (enrollment.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || typeof row.payload.subject !== "string" + || enrollment.runtimeGeneration > Number(setup.runtimeGeneration) || ["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase))) continue; + if (["active", "verifying", "attention"].includes(endpoint.status) && connection?.status === "active" && connection.enabled + && enrollment.runtimeGeneration === setup.runtimeGeneration && setup.slackCloudRetiredGeneration !== enrollment.runtimeGeneration + && isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment)) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "issued", result: { ...row.result, desiredActive: false, + phase: row.payload.binding ? "retired_cleanup" : "staged", nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))); + } +} +export async function reconcilePendingSlackFormWork(db: Db, input: { identity: SlackFormConnectorIdentity | null; protocol: SlackFormProtocol; + resolveBotToken(endpointId: string): Promise; connector?(): unknown; report(code: string, actionId: string): void }, now = new Date()): Promise { + if (!input.identity) return null; + const eligible = and(sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`, + sql`${chatActions.payload}->>'environment' = ${input.identity.environment}`, + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${input.identity.instanceId}`, + sql`${chatActions.result}->>'nextAttemptAt' is not null`); + const [row] = await db.select().from(chatActions).where(eligible).orderBy(sql`${chatActions.result}->>'nextAttemptAt'`, chatActions.id).limit(1); + if (!row) return null; + const due = typeof row.result?.nextAttemptAt === "string" ? Date.parse(row.result.nextAttemptAt) : Number.NaN; + if (Number.isFinite(due) && due > now.getTime()) return due; + let failureSnapshot = row; + const onSnapshot = (snapshot: typeof chatActions.$inferSelect) => { failureSnapshot = snapshot; }; + try { + if (!Number.isFinite(due)) attention("form_owner_deadline_invalid"); + if (row.kind === "slack_form_owner") await reconcileSlackFormEnrollment(db, { ...input, identity: input.identity, actionId: row.id, onSnapshot }); + else if (row.kind === "slack_form_grant") await publishSlackFormGrant(db, row.id, input.identity, input.protocol); + else await reconcileSlackFormNormalAck(db, { companyId: row.companyId, actionId: row.id, instanceId: input.identity.instanceId, connector: input.connector?.(), protocol: input.protocol, onSnapshot }); + } catch (error) { + const credentialAttention = error instanceof HttpError && [400, 401, 403, 404, 422].includes(error.status); + const permanent = credentialAttention || error instanceof SlackCloudReceiptError && error.disposition !== "retry" + || error instanceof PaperclipCloudConnectorError && ([400, 401, 403, 404].includes(error.status ?? 0) + || error.status === 409 && ["SLACK_FORM_CUSTODY_MISSING", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_RETIREMENT_PROOF_MISSING", "SLACK_FORM_CUSTODY_CHANGED", "SLACK_FORM_CANONICAL_FACT_CHANGED", "SLACK_FORM_CANONICAL_FACT_BUDGET_CHANGED"].includes(error.brokerReason ?? "")); + const code = credentialAttention ? "form_bot_credential_unavailable" : error instanceof SlackCloudReceiptError ? error.code : permanent ? "form_owner_authority_unavailable" : "form_owner_broker_unavailable"; + try { + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || !isDeepStrictEqual(current, failureSnapshot)) return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const attempts = typeof current.result?.attempts === "number" ? Math.min(current.result.attempts + 1, 1000000) : 1; + await tx.update(chatActions).set({ status: permanent ? "attention" : current.status, result: { ...current.result, attempts, failureCode: code, + failureReason: error instanceof PaperclipCloudConnectorError ? error.brokerReason ?? null : null, + nextAttemptAt: permanent ? null : new Date(Date.now() + Math.min(300000, 5000 * 2 ** Math.min(attempts - 1, 6))).toISOString() }, updatedAt: new Date() }).where(eq(chatActions.id, current.id)); + }); + } catch (writeError) { throw new AggregateError([error, writeError], "Slack form owner failure and durable retry failure"); } + input.report(code, row.id); + } + const [next] = await db.select({ result: chatActions.result }).from(chatActions).where(eligible).orderBy(sql`${chatActions.result}->>'nextAttemptAt'`, chatActions.id).limit(1); + const nextDue = typeof next?.result?.nextAttemptAt === "string" ? Date.parse(next.result.nextAttemptAt) : Number.NaN; + return Number.isFinite(nextDue) ? nextDue : null; +} + +export async function stageSlackFormPublication(tx: Tx, publicationId: string, interaction: IssueThreadInteraction): Promise { + const [currentInteraction] = await tx.select().from(issueThreadInteractions).where(and(eq(issueThreadInteractions.id, interaction.id), + eq(issueThreadInteractions.companyId, interaction.companyId), eq(issueThreadInteractions.issueId, interaction.issueId), eq(issueThreadInteractions.status, "pending"))).limit(1); + if (!currentInteraction || !isDeepStrictEqual(currentInteraction.payload, interaction.payload)) return null; + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, publicationId), eq(chatPublications.companyId, interaction.companyId), eq(chatPublications.issueId, interaction.issueId))).limit(1); + if (!publication || publication.state !== "published" || !publication.providerMessageId) return null; + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, publication.endpointId), eq(chatEndpoints.companyId, publication.companyId))).limit(1); + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status)) return null; + const rawBinding = slackFormSetupRecord(endpoint.setup).slackCloudForms; + if (!rawBinding) return null; + const binding = parseSlackFormBinding(rawBinding), form = chatInteractionForm(interaction, "slack"); + if (!form || form.status !== "pending") return null; + const [owner] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(binding)}::jsonb`, sql`${chatActions.result}->>'phase' = 'enrolled'`)).limit(1); + if (!owner || typeof owner.payload.environment !== "string" || typeof owner.payload.subject !== "string") return attention("form_grant_owner_missing"); + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.conversationId, publication.conversationId), eq(chatActions.kind, "question_form_open"), eq(chatActions.status, "issued"), + sql`${chatActions.payload}->>'publicationId' = ${publication.id}`, sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`)).limit(1); + if (!open) return null; + const resolved = await resolveChatQuestionFormOpen(tx, { companyId: endpoint.companyId, endpointId: endpoint.id, + conversationId: publication.conversationId, interaction: form, openActionId: open.providerActionId }); + if (!resolved) return null; + return stageSlackFormGrant(tx, { binding, environment: owner.payload.environment, subject: owner.payload.subject, resolved }); +} + + +export function createSlackFormConnectorAccess() { + let cached: { config: NonNullable>; connector: ReturnType } | undefined; + const connector = () => { + let config: ReturnType; + try { config = paperclipCloudConnectorConfigFromEnv(); } catch { return null; } + if (!config) { cached = undefined; return null; } + if (!cached || !isDeepStrictEqual(cached.config, config)) cached = { config, connector: createPaperclipCloudConnector({ config }) }; + return cached.connector; + }; + const protocol: SlackFormProtocol = async input => { + const current = connector(), identity = slackFormConnectorIdentity(current); + if (!current || !identity) return attention("form_connector_unavailable"); + const operation = Reflect.get(current, "slackFormOperation"); + if (typeof operation !== "function") return attention("form_connector_unsupported"); + return Reflect.apply(operation, current, [input]); + }; + return { connector, protocol, identity: () => slackFormConnectorIdentity(connector()) }; +} + +const formWorkOwners = new WeakMap Promise>(); +export function registerSlackFormWorkOwner(db: Db, owner: (now: Date) => Promise) { + formWorkOwners.set(db, owner); + return () => { if (formWorkOwners.get(db) === owner) formWorkOwners.delete(db); }; +} +export function reconcileRegisteredSlackFormWork(db: Db, now: Date): Promise { + return formWorkOwners.get(db)?.(now) ?? Promise.resolve(null); +} + +export function slackFormAttentionCanRearm(input: { code: unknown; binding: unknown; enrollment: unknown }): boolean { + const recoverable = ["form_bot_credential_unavailable", "form_opening_cipher_unavailable", "form_submission_cipher_unavailable", + "form_connector_unsupported", "form_ack_receipts_unsupported", "form_ack_acceptance_missing", "form_owner_broker_unavailable"]; + if (typeof input.code !== "string" || !recoverable.includes(input.code)) return false; + try { + const binding = parseSlackFormBinding(input.binding), enrollment = parseSlackFormPublicEnrollment(input.enrollment); + return matches(binding, enrollment); + } catch { return false; } +} +export async function rearmSlackFormAttention(tx: Tx, input: { enrollment: SlackFormPublicEnrollment; subject: string; environment: string }): Promise { + const enrollment = parseSlackFormPublicEnrollment(input.enrollment); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, enrollment.endpointId), eq(chatEndpoints.companyId, enrollment.companyId))).for("update"); + if (!endpoint || endpoint.provider !== "slack" || !isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment) + || slackFormSetupRecord(endpoint.setup).slackCloudSubject !== input.subject) return 0; + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, enrollment.companyId), eq(chatActions.endpointId, enrollment.endpointId), + eq(chatActions.status, "attention"), sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`)).for("update"); + let count = 0; + for (const row of rows) { + if (row.payload.environment !== input.environment || row.result?.nextAttemptAt !== null) continue; + const binding = row.kind === "slack_form_owner" ? row.payload.binding : row.kind === "slack_form_grant" + ? row.payload.grant && typeof row.payload.grant === "object" ? Reflect.get(row.payload.grant, "binding") : undefined : row.payload.binding; + if (row.payload.subject !== undefined && row.payload.subject !== input.subject) continue; + if (row.kind === "slack_form_owner" && binding === undefined) { + if (row.result?.failureCode !== "form_bot_credential_unavailable") continue; + try { if (!isDeepStrictEqual(parseSlackFormPublicEnrollment(row.payload.enrollment), enrollment)) continue; } catch { continue; } + } else if (!slackFormAttentionCanRearm({ code: row.result?.failureCode, binding, enrollment })) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const phase = row.result?.phase; + const status = row.kind === "slack_form_session" && phase === "submitted" ? "processed" : row.kind === "slack_form_session" && phase === "not_opened" ? "failed" : row.kind === "slack_form_session" && phase === "closed" ? "cancelled" : "issued"; + const updated = await tx.update(chatActions).set({ status, result: { ...row.result, firstFailureCode: row.result?.firstFailureCode ?? row.result?.failureCode, + nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, enrollment.companyId), + eq(chatActions.endpointId, enrollment.endpointId), sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + count += updated.length; + } + return count; +} diff --git a/server/src/services/chat-channels/slack/form-publication.ts b/server/src/services/chat-channels/slack/form-publication.ts new file mode 100644 index 00000000000..d98fc46158a --- /dev/null +++ b/server/src/services/chat-channels/slack/form-publication.ts @@ -0,0 +1,82 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatPublications, toolConnections, type Db } from "@paperclipai/db"; +import type { ChatPublicationPreparationWait } from "../publications/preparation.js"; +import { chatInteractionForm } from "../../chat-interaction-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, slackFormSetupRecord } from "./cloud-forms.js"; +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; +import type { EndpointRow } from "../runtime-types.js"; +import { projectSlackFormPublicEnrollment, stageSlackFormPublication } from "./form-owner.js"; + +type Tx = Parameters[0]>[0]; +function object(value: unknown): Record | null { + return value && typeof value === "object" && !Array.isArray(value) ? slackFormSetupRecord(value) : null; +} +function managedFormsExpected(setup: Record): boolean { + return setup.slackCloudIntent !== undefined || setup.slackCloudForms !== undefined + || setup.slackCloudEvents !== undefined || typeof setup.slackCloudInstanceId === "string"; +} +function scheduled(value: unknown): boolean { + return typeof value === "string" && Number.isFinite(Date.parse(value)); +} +function unavailable(): never { + // Inconsistent, stopped and attention custody retains the ordinary failure + // policy and authorized Retry path; only an actual scheduled owner can wait. + throw new SlackCloudReceiptError("form_publication_enrollment_pending", "retry"); +} +const enrollmentWait: ChatPublicationPreparationWait = { + kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2_000, +}; +export function createSlackFormPublicationStage(db: Db) { + return async (endpoint: EndpointRow, publication: typeof chatPublications.$inferSelect): Promise<((tx: Tx) => Promise) | ChatPublicationPreparationWait | null> => { + if (endpoint.provider !== "slack" || !managedFormsExpected(slackFormSetupRecord(endpoint.setup)) || !publication.issueId + || typeof publication.payload.interactionId !== "string" || publication.companyId !== endpoint.companyId || publication.endpointId !== endpoint.id) return null; + const interaction = (await issueThreadInteractionService(db).listForIssue(publication.issueId)) + .find(value => value.id === publication.payload.interactionId && value.companyId === endpoint.companyId && value.status === "pending"); + if (!interaction || !chatInteractionForm(interaction, "slack")) return null; + const [open] = await db.select({ id: chatActions.id }).from(chatActions).where(and( + eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.conversationId, publication.conversationId), eq(chatActions.kind, "question_form_open"), eq(chatActions.status, "issued"), + sql`${chatActions.payload}->>'publicationId' = ${publication.id}`, sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`)).limit(1); + if (!open) return null; + const [current] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, endpoint.id), eq(chatEndpoints.companyId, endpoint.companyId))).limit(1); + if (!current || current.provider !== "slack" || !["active", "verifying", "attention"].includes(current.status) + || current.connectionId !== endpoint.connectionId || current.publicId !== endpoint.publicId) return unavailable(); + const [connection] = await db.select().from(toolConnections).where(and(eq(toolConnections.id, current.connectionId), eq(toolConnections.companyId, current.companyId))).limit(1); + if (!connection || connection.status !== "active" || !connection.enabled) return unavailable(); + const setup = slackFormSetupRecord(current.setup), generation = setup.runtimeGeneration; + if (generation !== slackFormSetupRecord(endpoint.setup).runtimeGeneration || !Number.isSafeInteger(generation) || Number(generation) < 1 + || Number.isSafeInteger(setup.slackCloudRetiredGeneration) && Number(setup.slackCloudRetiredGeneration) >= Number(generation)) return unavailable(); + const binding = setup.slackCloudForms === undefined ? undefined : parseSlackFormBinding(setup.slackCloudForms); + const mayWait = current.status !== "attention"; + const intent = object(setup.slackCloudIntent); + if (intent) { + // This is the initial events-enrollment responsibility, before onCommitted + // can atomically stage the forms owner. A null deadline means attention. + if (!mayWait || intent.version !== 1 || typeof intent.id !== "string" || !intent.id || intent.active !== true + || intent.companyId !== current.companyId || intent.endpointId !== current.id || intent.publicId !== current.publicId + || intent.connectionId !== current.connectionId || intent.runtimeGeneration !== generation || !/^[A-Za-z0-9_-]{43}$/.test(current.publicId) + || typeof intent.instanceId !== "string" || !intent.instanceId || typeof intent.subject !== "string" || !intent.subject + || typeof intent.teamId !== "string" || !/^T[A-Z0-9]+$/.test(intent.teamId) || intent.teamId !== current.providerAccountId + || typeof intent.appId !== "string" || !/^A[A-Z0-9]+$/.test(intent.appId) + || !scheduled(intent.nextAttemptAt)) return unavailable(); + return enrollmentWait; + } + const enrollment = projectSlackFormPublicEnrollment(current); + if (!enrollment || enrollment.runtimeGeneration !== generation || enrollment.instanceId !== setup.slackCloudInstanceId + || typeof setup.slackCloudSubject !== "string" || !setup.slackCloudSubject) return unavailable(); + const owners = await db.select().from(chatActions).where(and(eq(chatActions.companyId, current.companyId), eq(chatActions.endpointId, current.id), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'enrollment' = ${JSON.stringify(enrollment)}::jsonb`)).limit(2); + if (owners.length !== 1) return unavailable(); + const [owner] = owners; + if (!owner || owner.status !== "issued" || owner.result?.desiredActive === false || owner.payload.subject !== setup.slackCloudSubject + || typeof owner.payload.environment !== "string" || !["development", "test", "staging", "production"].includes(owner.payload.environment)) return unavailable(); + if (mayWait && owner.result?.phase === "staged" && owner.payload.binding === undefined && scheduled(owner.result.nextAttemptAt)) return enrollmentWait; + if (owner.result?.phase !== "enrolled" || !binding || !isDeepStrictEqual(owner.payload.binding, binding) + || !isDeepStrictEqual({ version: binding.version, instanceId: binding.instanceId, companyId: binding.companyId, endpointId: binding.endpointId, + publicId: binding.publicId, connectionId: binding.connectionId, runtimeGeneration: binding.runtimeGeneration, + teamId: binding.teamId, appId: binding.appId, bindingId: binding.eventBindingId }, enrollment)) return unavailable(); + return async tx => { await stageSlackFormPublication(tx, publication.id, interaction); }; + }; +} diff --git a/server/src/services/chat-channels/slack/form-retirement-work.ts b/server/src/services/chat-channels/slack/form-retirement-work.ts new file mode 100644 index 00000000000..a5e4758d12c --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retirement-work.ts @@ -0,0 +1,120 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { replaceSlackFormReceiptSignature } from "./form-signature-refresh.js"; +import { parseSlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { verifySlackFormRetirementReceipt } from "./receipt-authority.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { parseSlackFormPublicEnrollment, parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; +import type { SlackFormProtocol } from "./form-owner.js"; +import type { SlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +type Row = typeof chatActions.$inferSelect; +function retry(code: string): never { throw new SlackCloudReceiptError(code, "retry"); } +export function assertSlackFormRetirementWorkAuthority(row: Row, identity: SlackFormConnectorIdentity, receipt: SlackFormRetirementReceipt): void { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), binding = receipt.binding; + if (row.kind !== "slack_form_owner" || row.companyId !== binding.companyId || row.endpointId !== binding.endpointId + || row.payload.subject !== receipt.subject || row.payload.environment !== receipt.environment || receipt.environment !== identity.environment + || binding.instanceId !== identity.instanceId || enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(binding, key) !== Reflect.get(enrollment, key))) + throw new SlackCloudReceiptError("form_retirement_authority_changed", "attention"); + if (row.payload.binding !== undefined && !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), binding)) + throw new SlackCloudReceiptError("form_retirement_fingerprint_changed", "attention"); + if (row.result?.retirementReceipt !== undefined && !isDeepStrictEqual(row.result.retirementReceipt, receipt)) + throw new SlackCloudReceiptError("form_retirement_receipt_changed", "attention"); + if (!isSlackFormEnrollmentActionId(row.providerActionId, enrollment)) throw new SlackCloudReceiptError("form_retirement_owner_changed", "attention"); +} +export function verifyRetainedSlackFormRetirement(compactJws: string) { + try { return verifySlackFormRetirementReceipt({ compactJws }); } + catch (error) { + if (error instanceof Error && ["PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is not configured", "PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is invalid", + "Cloud runtime identity uses an unknown signing key"].includes(error.message)) return retry("form_retirement_verifier_unavailable"); + throw new SlackCloudReceiptError("form_retirement_receipt_invalid", "attention"); + } +} +export async function acceptSlackFormRetirementLocally(db: Db, row: Row, identity: SlackFormConnectorIdentity, compactJws: string): Promise { + const receipt = verifyRetainedSlackFormRetirement(compactJws); assertSlackFormRetirementWorkAuthority(row, identity, receipt); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_acceptance_changed"); + if (["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(current.result?.phase))) return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...current.result, phase: "retirement_accepted", + retirementReceiptJws: compactJws, retirementReceipt: receipt, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(current.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(current.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return retry("form_retirement_acceptance_changed"); + }); +} +export async function transferSlackFormRetirement(db: Db, snapshot: Row, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol, + onSnapshot?: (snapshot: Row) => void): Promise { + let row = snapshot; + if (row.result?.phase === "retirement_gc_pending") return releaseSlackRetirementFact(db, row, identity, protocol); + if (row.result?.phase !== "retirement_accepted" || typeof row.result.retirementReceiptJws !== "string") return retry("form_retirement_acceptance_missing"); + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt); assertSlackFormRetirementWorkAuthority(row, identity, receipt); + const refreshed = await protocol({ subject: receipt.subject, companyId: row.companyId, operation: { kind: "cleanup", binding: receipt.binding, + refreshRetirementReceipt: { receiptId: receipt.receiptId, ownerId: receipt.ownerId } } }); + if (!refreshed || typeof refreshed !== "object" || Array.isArray(refreshed) || Object.keys(refreshed).join() !== "retirementReceiptJws" + || typeof Reflect.get(refreshed, "retirementReceiptJws") !== "string") return retry("form_signature_refresh_unconfirmed"); + const compactJws = Reflect.get(refreshed, "retirementReceiptJws"); + if (typeof compactJws !== "string" || !isDeepStrictEqual(verifyRetainedSlackFormRetirement(compactJws), receipt)) + throw new SlackCloudReceiptError("form_signature_canonical_changed", "attention"); + row = await replaceSlackFormReceiptSignature(db, row, "retirementReceiptJws", compactJws); + onSnapshot?.(row); + const response = await protocol({ subject: receipt.subject, companyId: row.companyId, + operation: { kind: "cleanup", binding: receipt.binding, acceptedRetirementReceipt: compactJws } }); + if (!response || typeof response !== "object" || Array.isArray(response) || Object.keys(response).join() !== "acceptedRetirementDigest" + || Reflect.get(response, "acceptedRetirementDigest") !== receipt.receiptId) return retry("form_retirement_transfer_unconfirmed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_transfer_changed"); + const result = current.result ?? {}; + const retirement = { version: 1, binding: receipt.binding, instanceId: identity.instanceId, environment: identity.environment, + subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...result, phase: "retirement_gc_pending", nextAttemptAt: new Date().toISOString(), retirement }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(current.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(current.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return retry("form_retirement_transfer_changed"); + const grants = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), + eq(chatActions.kind, "slack_form_grant"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(receipt.binding)}::jsonb`)).for("update"); + for (const grant of grants) { + if (grant.payload.environment !== identity.environment || grant.payload.subject !== receipt.subject) + throw new SlackCloudReceiptError("form_grant_retirement_identity_changed", "attention"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const cancelled = await tx.update(chatActions).set({ status: "cancelled", result: { ...grant.result, phase: "transferred", nextAttemptAt: null, retirement }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, grant.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, grant.status), + sql`${chatActions.payload} = ${JSON.stringify(grant.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(grant.result)}::jsonb`)).returning({ id: chatActions.id }); + if (cancelled.length !== 1) return retry("form_grant_retirement_changed"); + } + }); +} + +export function assertSlackRetirementGCWork(row: Row, identity: SlackFormConnectorIdentity): SlackFormRetirementReceipt { + if (row.result?.phase !== "retirement_gc_pending") throw new SlackCloudReceiptError("form_retirement_gc_missing", "attention"); + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt); + assertSlackFormRetirementWorkAuthority(row, identity, receipt); + const expected = { version: 1, binding: receipt.binding, instanceId: identity.instanceId, environment: identity.environment, + subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + if (!isDeepStrictEqual(row.result.retirement, expected)) throw new SlackCloudReceiptError("form_retirement_gc_authority_changed", "attention"); + return receipt; +} +async function releaseSlackRetirementFact(db: Db, row: Row, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol): Promise { + const receipt = assertSlackRetirementGCWork(row, identity); + const response = await protocol({ subject: receipt.subject, companyId: row.companyId, + operation: { kind: "cleanup", binding: receipt.binding, releaseRetirementFact: { receiptId: receipt.receiptId } } }); + if (!response || typeof response !== "object" || Array.isArray(response) || Object.keys(response).join() !== "releasedRetirementDigest" + || Reflect.get(response, "releasedRetirementDigest") !== receipt.receiptId) return retry("form_retirement_gc_unconfirmed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_gc_changed"); + const { retirementReceiptJws: _, ...result } = current.result ?? {}; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const rows = await tx.update(chatActions).set({ status: "processed", result: { ...result, phase: "transferred", nextAttemptAt: null }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (rows.length !== 1) return retry("form_retirement_gc_changed"); + }); +} diff --git a/server/src/services/chat-channels/slack/form-retirement.test.ts b/server/src/services/chat-channels/slack/form-retirement.test.ts new file mode 100644 index 00000000000..32face47804 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retirement.test.ts @@ -0,0 +1,88 @@ +import { createHash, generateKeyPairSync, sign } from "node:crypto"; +import { afterEach, expect, it, vi } from "vitest"; +import type { chatActions } from "@paperclipai/db"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormRetirementReceipt, slackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { assertSlackFormRetirementWorkAuthority, verifyRetainedSlackFormRetirement } from "./form-retirement-work.js"; +import { verifySlackFormFinalizedReceipt, verifySlackFormRetirementReceipt } from "./receipt-authority.js"; +import { projectSlackFormRetainedOwner } from "./form-session.js"; + +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const { credentialFingerprint, eventBindingId, ...base } = binding; +const enrollment = { ...base, bindingId: eventBindingId }, identity = { instanceId: "instance", environment: "test" as const }; +const receipt = slackFormRetirementReceipt(binding, "test", "original-actor"); +function row(): typeof chatActions.$inferSelect { + return { id: "11111111-1111-4111-8111-111111111111", companyId: "company", endpointId: "endpoint", deliveryId: null, + conversationId: null, principalId: null, kind: "slack_form_owner", providerActionId: `slack-form-owner:${createHash("sha256").update(JSON.stringify(enrollment)).digest("hex")}`, + payload: { enrollment, binding, environment: "test", subject: "original-actor" }, status: "issued", result: { phase: "retired_cleanup", nextAttemptAt: null }, + createdAt: new Date(0), updatedAt: new Date(0) }; +} +function signed(value: unknown = receipt) { + const pair = generateKeyPairSync("ed25519"), kid = "retirement-unit"; + const env = { PAPERCLIP_CLOUD_STACK_ID: "instance", PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, use: "sig", alg: "EdDSA" }] }) }; + const enc = (input: unknown) => Buffer.from(JSON.stringify(input)).toString("base64url"); + const body = enc({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + enc({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: "instance", receipt: value }); + return { compactJws: body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"), env }; +} +afterEach(() => vi.unstubAllEnvs()); +it("verifies a real runtime signature without admitting the other receipt kind", () => { + const input = signed(); expect(verifySlackFormRetirementReceipt(input)).toEqual(receipt); + expect(() => verifySlackFormFinalizedReceipt(input)).toThrow(); +}); +it("rejects foreign instance, unknown signing key and tampered retirement payload", () => { + const input = signed(); + expect(() => verifySlackFormRetirementReceipt({ ...input, env: { ...input.env, PAPERCLIP_CLOUD_STACK_ID: "foreign" } })).toThrow(); + expect(() => verifySlackFormRetirementReceipt({ ...input, env: { ...input.env, PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: '{"keys":[]}' } })).toThrow(); + const parts = input.compactJws.split("."); parts[1] = Buffer.from("{}").toString("base64url"); + expect(() => verifySlackFormRetirementReceipt({ ...input, compactJws: parts.join(".") })).toThrow(); +}); +it("fails closed on changed canonical identity, extra keys and oversized original subject", () => { + expect(() => parseSlackFormRetirementReceipt({ ...receipt, subject: "replacement" })).toThrow(); + expect(() => parseSlackFormRetirementReceipt({ ...receipt, extra: true })).toThrow(); + expect(() => slackFormRetirementReceipt(binding, "test", "x".repeat(201))).toThrow(); +}); +it("requires exact local company, immutable original subject, fingerprint and owner action", () => { + const current = row(); expect(() => assertSlackFormRetirementWorkAuthority(current, identity, receipt)).not.toThrow(); + for (const changed of [{ ...current, companyId: "foreign" }, { ...current, payload: { ...current.payload, subject: "replacement" } }, + { ...current, payload: { ...current.payload, binding: { ...binding, credentialFingerprint: "c".repeat(64) } } }, { ...current, providerActionId: "foreign-action" }]) + expect(() => assertSlackFormRetirementWorkAuthority(changed, identity, receipt)).toThrow(); + expect(() => assertSlackFormRetirementWorkAuthority(current, { ...identity, instanceId: "foreign" }, receipt)).toThrow(); +}); +it("key outage remains retryable and invalid signature is visible attention", () => { + const input = signed(); vi.stubEnv("PAPERCLIP_CLOUD_STACK_ID", "instance"); vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", ""); + expect(() => verifyRetainedSlackFormRetirement(input.compactJws)).toThrow(expect.objectContaining({ code: "form_retirement_verifier_unavailable" })); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", input.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS); + const other = signed(); expect(() => verifyRetainedSlackFormRetirement(other.compactJws)).toThrow(expect.objectContaining({ code: "form_retirement_receipt_invalid" })); +}); +it("accepted retirement retains normal polling identity without granting new-use delivery authority", () => { + const current = row(), input = signed(); current.result = { phase: "retirement_accepted", retirementReceipt: receipt, retirementReceiptJws: input.compactJws, nextAttemptAt: new Date(0).toISOString() }; + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toEqual({ subject: "github-bot-events", companyId: current.companyId, binding, requiresDelivery: false }); + const missingBinding = { ...current, payload: { enrollment, environment: "test", subject: "original-actor" } }; + expect(projectSlackFormRetainedOwner(missingBinding, identity.instanceId, identity.environment)).not.toBeNull(); + expect(() => projectSlackFormRetainedOwner({ ...current, result: { ...current.result, retirementReceiptJws: "invalid" } }, identity.instanceId, identity.environment)).toThrow(); +}); +it("only exact committed transfer discharges history and its retained consumer", () => { + const current = row(); current.status = "processed"; + const retirement = { version: 1, binding, instanceId: identity.instanceId, environment: identity.environment, subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + current.result = { phase: "transferred", nextAttemptAt: null, retirement }; + expect(slackFormRetirementHistoryReceipt(current)).toEqual(receipt); + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toBeNull(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, companyId: "foreign" })).toThrow(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, result: { ...current.result, retirement: { ...retirement, receiptDigest: "0".repeat(64) } } })).toThrow(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, result: { phase: "transferred", nextAttemptAt: null } })).toThrow(); +}); + +it("preserves exact legacy custody through JSONB ordering for acceptance, polling and history", () => { + const current = row(); + current.payload = { ...current.payload, enrollment: Object.fromEntries(Object.entries(enrollment).sort(([a], [b]) => Buffer.byteLength(a) - Buffer.byteLength(b) || a.localeCompare(b))) }; + expect(() => assertSlackFormRetirementWorkAuthority(current, identity, receipt)).not.toThrow(); + const input = signed(); + current.result = { phase: "retirement_accepted", retirementReceipt: receipt, retirementReceiptJws: input.compactJws, nextAttemptAt: new Date(0).toISOString() }; + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)?.requiresDelivery).toBe(false); + current.status = "processed"; + current.result = { phase: "transferred", nextAttemptAt: null, retirement: { version: 1, binding, instanceId: identity.instanceId, + environment: identity.environment, subject: receipt.subject, receiptDigest: receipt.receiptId, receipt } }; + expect(slackFormRetirementHistoryReceipt(current)).toEqual(receipt); + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toBeNull(); +}); diff --git a/server/src/services/chat-channels/slack/form-retry.ts b/server/src/services/chat-channels/slack/form-retry.ts new file mode 100644 index 00000000000..d2fcebf36e9 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retry.ts @@ -0,0 +1,87 @@ +import { parseSlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { assertSlackFormRetirementWorkAuthority, assertSlackRetirementGCWork } from "./form-retirement-work.js"; +import { slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, type Db } from "@paperclipai/db"; +import type { ConnectionManagementContext } from "../../connection-management.js"; +import { logActivity } from "../../activity-log.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { badRequest, conflict, forbidden, notFound } from "../../../errors.js"; +import { parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormProofAcceptance } from "./form-session.js"; +import { parseSlackFormAckWork } from "./form-ack-work.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +export async function retrySlackFormAcknowledgement(db: Db, input: { companyId: string; endpointId: string; actionId: string; + identity(): SlackFormConnectorIdentity | null; management: ConnectionManagementContext }): Promise<{ actionId: string; state: "scheduled" | "already_scheduled" | "complete" }> { + if (!/^[a-f0-9-]{36}$/i.test(input.actionId)) throw badRequest("Invalid provider work ID"); + await input.management.reauthorize(); + const identity = input.identity(); + if (input.management.resource.companyId !== input.companyId || !identity || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current company and Cloud management authority are required"); + const result = await db.transaction(async tx => { + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, input.endpointId), eq(chatEndpoints.companyId, input.companyId))).for("update"); + if (!endpoint || endpoint.provider !== "slack") throw notFound("Slack endpoint not found"); + if (input.management.resource.connectionId && input.management.resource.connectionId !== endpoint.connectionId) throw forbidden("Connection authority changed"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, endpoint.id), inArray(chatActions.kind, ["slack_form_session", "slack_form_owner"]))).for("update"); + if (!row) throw notFound("Slack form work not found"); + if (row.kind === "slack_form_owner") { + if (row.result?.phase === "transferred") { + const retired = slackFormRetirementHistoryReceipt(row); + if (!retired || retired.binding.instanceId !== identity.instanceId || retired.environment !== identity.environment + || retired.binding.publicId !== endpoint.publicId || retired.binding.connectionId !== endpoint.connectionId) throw conflict("Retirement history authority changed"); + return { actionId: row.id, state: "complete" as const }; + } + if (!["retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase)) || typeof row.result?.retirementReceiptJws !== "string") throw conflict("No accepted retirement responsibility is available for retry"); + const receipt = row.result?.phase === "retirement_gc_pending" ? assertSlackRetirementGCWork(row, identity) : parseSlackFormRetirementReceipt(row.result.retirementReceipt); + assertSlackFormRetirementWorkAuthority(row, identity, receipt); + if (receipt.binding.publicId !== endpoint.publicId || receipt.binding.connectionId !== endpoint.connectionId + || ["SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_CUSTODY_CHANGED"].includes(String(row.result.failureReason))) + throw conflict("Retirement responsibility does not match current ownership"); + if (typeof row.result.nextAttemptAt === "string" && Number.isFinite(Date.parse(row.result.nextAttemptAt))) return { actionId: row.id, state: "already_scheduled" as const }; + if (row.status !== "attention" || row.result.nextAttemptAt !== null) throw conflict("Retirement work has no valid retry state"); + await input.management.reauthorize(); + const currentIdentity = input.identity(); + if (!currentIdentity || currentIdentity.instanceId !== identity.instanceId || currentIdentity.environment !== identity.environment + || input.management.resource.companyId !== input.companyId || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current Cloud or management authority changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...row.result, + firstFailureCode: row.result.firstFailureCode ?? row.result.failureCode, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw conflict("Retirement work changed before retry was committed"); + return { actionId: row.id, state: "scheduled" as const }; + } + const binding = parseSlackFormBinding(row.payload.binding), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance), work = parseSlackFormAckWork(row.result?.formAck); + if (binding.companyId !== input.companyId || binding.endpointId !== endpoint.id || binding.publicId !== endpoint.publicId || binding.connectionId !== endpoint.connectionId + || binding.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || !sameSlackFormBinding(proof.binding, binding) + || proof.appReceiptId !== row.id || proof.operationId !== row.payload.operationId || work.deliveryId !== `slack_form_${proof.operationId}` + || !["submitted", "closed", "not_opened"].includes(String(row.result?.phase))) throw conflict("Retained form responsibility does not match current Cloud authority"); + if (work.receipt && (work.receipt.appReceiptId !== row.id || work.receipt.operationId !== proof.operationId + || !sameSlackFormBinding(work.receipt.binding, binding) || work.receipt.terminalDigest !== proof.terminalDigest)) throw conflict("Retained form receipt changed"); + if (work.phase === "transferred") { + if (row.status === "attention" || row.result?.nextAttemptAt !== null) throw conflict("Completed provider work has inconsistent retained status"); + return { actionId: row.id, state: "complete" as const }; + } + if (typeof row.result?.nextAttemptAt === "string" && Number.isFinite(Date.parse(row.result.nextAttemptAt))) return { actionId: row.id, state: "already_scheduled" as const }; + if (row.status !== "attention" || row.result?.nextAttemptAt !== null) throw conflict("Provider work has no valid retained retry state"); + await input.management.reauthorize(); + const currentIdentity = input.identity(); + if (!currentIdentity || currentIdentity.instanceId !== identity.instanceId || currentIdentity.environment !== identity.environment + || input.management.resource.companyId !== input.companyId || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current Cloud or management authority changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const nextAttemptAt = new Date().toISOString(); + const phase = row.result?.phase, status = phase === "submitted" ? "processed" : phase === "not_opened" ? "failed" : "cancelled"; + const updated = await tx.update(chatActions).set({ status, result: { ...row.result, firstFailureCode: row.result?.firstFailureCode ?? row.result?.failureCode, + nextAttemptAt }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw conflict("Provider work changed before retry was committed"); + return { actionId: row.id, state: "scheduled" as const }; + }); + if (result.state === "scheduled") await logActivity(db, { ...input.management.audit, companyId: input.companyId, action: "chat_provider_work.retry_requested", + entityType: "tool_connection", entityId: input.management.resource.connectionId ?? input.endpointId, details: { endpointId: input.endpointId, actionId: input.actionId } }); + return result; +} diff --git a/server/src/services/chat-channels/slack/form-session.ts b/server/src/services/chat-channels/slack/form-session.ts new file mode 100644 index 00000000000..36c6c322a4b --- /dev/null +++ b/server/src/services/chat-channels/slack/form-session.ts @@ -0,0 +1,448 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { parseSlackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { createHash } from "node:crypto"; +import type { IssueThreadInteraction } from "@paperclipai/shared"; +import { createLocalEncryptedCipher } from "../../../secrets/local-encrypted-provider.js"; +import type { StoredSecretVersionMaterial } from "../../../secrets/types.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatExternalPrincipals, chatPublications, type Db } from "@paperclipai/db"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; +import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { assertSlackFormTerminalAck, parseSlackFormPublicEnrollment, parseSlackFormOperation, currentSlackFormOwnerBinding, parseSlackFormBinding, slackFormSubmissionEvidence, slackIssuedFormGrantId, projectSlackIssuedFormGrant, sameSlackFormBinding, type SlackFormBinding, type SlackFormOperation, type SlackFormView } from "./cloud-forms.js"; + +type FormTx = Parameters[0]>[0]; +export type SlackFormSessionScope = { companyId: string; endpointId: string; principalId: string; operationId: string }; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +export async function stageSlackFormSession(tx: FormTx, input: { + operation: SlackFormOperation; currentBinding: SlackFormBinding; principalId: string; + resolved: ResolvedChatQuestionFormOpen; lease: CredentialMutationLeaseGuard; environment: string; +}): Promise<{ kind: "update"; actionId: string; view: SlackFormView } | { kind: "closed" | "retained"; actionId: string }> { + const { operation: op, currentBinding: binding, resolved } = input; + await input.lease.assertOwned(tx); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.companyId, binding.companyId), eq(chatEndpoints.id, binding.endpointId))).for("update"); + const setup: Record | undefined = endpoint?.setup && typeof endpoint.setup === "object" && !Array.isArray(endpoint.setup) ? { ...endpoint.setup } : undefined; + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status) + || endpoint.publicId !== binding.publicId || endpoint.connectionId !== binding.connectionId + || endpoint.providerAccountId !== binding.teamId || setup?.runtimeGeneration !== binding.runtimeGeneration + || !sameSlackFormBinding(binding, op.binding)) fail("form_current_endpoint_changed"); + const localOwner = currentSlackFormOwnerBinding(setup ?? {}); + if (!localOwner || !sameSlackFormBinding(localOwner, binding)) fail("form_current_enrollment_changed"); + const [principal] = await tx.select().from(chatExternalPrincipals).where(and(eq(chatExternalPrincipals.id, input.principalId), + eq(chatExternalPrincipals.companyId, binding.companyId), eq(chatExternalPrincipals.provider, "slack"))).limit(1); + if (!principal || principal.externalId !== op.userId || principal.providerAccountId !== binding.teamId) fail("form_current_principal_changed"); + const grant = await projectSlackIssuedFormGrant(tx, binding, resolved); + if (!grant || grant.grantId !== op.grantId || grant.interactionId !== resolved.interactionId) fail("form_current_issued_authority_changed"); + const providerActionId = `slack-form-session:${op.id}`; + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.providerActionId, providerActionId))).for("update"); + const payload = { version: 1, binding, environment: input.environment, operationId: op.id, grantId: op.grantId, requestDigest: op.requestDigest, + userId: op.userId, publicationId: grant.publicationId, interactionId: grant.interactionId, + openActionRowId: grant.openActionRowId, submitActionRowId: grant.submitActionRowId, + view: op.view ?? op.closed?.view ?? null }; + if (old) { + if (old.kind !== "slack_form_session" || old.principalId !== principal.id || old.conversationId !== grant.conversationId + || !isDeepStrictEqual(old.payload, payload)) fail("form_session_identity_changed"); + if (op.closed) { + await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, operationId: op.id }, op.closed.view); + return { kind: "closed", actionId: old.id }; + } + if (old.status === "processing" && old.result?.phase === "update_attempted" && op.phase === "confirmed" && op.view + && isDeepStrictEqual(old.result.view, op.view)) { + await input.lease.assertOwned(tx); + return { kind: "update", actionId: old.id, view: op.view }; + } + return { kind: old.status === "cancelled" ? "closed" : "retained", actionId: old.id }; + } + if (["reserved", "attempted"].includes(op.phase) && !op.closed) throw new SlackCloudReceiptError("form_receipt_pending", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, + conversationId: grant.conversationId, kind: "slack_form_session", providerActionId, payload, + status: op.closed ? "cancelled" : op.phase === "confirmed" ? "processing" : "attention", + result: op.closed ? { phase: "closed", view: op.closed.view } : op.phase === "confirmed" + ? { phase: "update_attempted", view: op.view } : { phase: "provider_outcome_unresolved" } }) + .onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_session_claim_changed", "retry"); + await input.lease.assertOwned(tx); + if (op.closed) await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, operationId: op.id }, op.closed.view); + return op.closed ? { kind: "closed", actionId: row.id } : op.phase === "confirmed" && op.view + ? { kind: "update", actionId: row.id, view: op.view } : { kind: "retained", actionId: row.id }; +} +export async function confirmSlackFormSessionUpdate(tx: FormTx, scope: SlackFormSessionScope, view: SlackFormView, newHash: string): Promise { + if (!newHash || newHash.length > 200) fail("form_updated_hash_invalid"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), + eq(chatActions.principalId, scope.principalId), eq(chatActions.providerActionId, `slack-form-session:${scope.operationId}`), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row || row.status !== "processing" || row.result?.phase !== "update_attempted") return false; + const expected = row.result.view; + if (!expected || typeof expected !== "object" || Array.isArray(expected) || Reflect.get(expected, "id") !== view.id + || Reflect.get(expected, "hash") !== view.hash || Reflect.get(expected, "externalId") !== view.externalId) fail("form_updated_view_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { phase: "ready", view: { ...view, hash: newHash } }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, scope.companyId), eq(chatActions.status, "processing"), sql`${chatActions.result}->>'phase' = 'update_attempted'`)).returning({ id: chatActions.id }); + return updated.length === 1; +} +export async function closeSlackFormSession(tx: FormTx, scope: SlackFormSessionScope, view: SlackFormView): Promise { + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), + eq(chatActions.principalId, scope.principalId), eq(chatActions.providerActionId, `slack-form-session:${scope.operationId}`), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row) return false; + const current = row.result?.view; + if (!current || typeof current !== "object" || Array.isArray(current) || Reflect.get(current, "id") !== view.id + || Reflect.get(current, "externalId") !== view.externalId + || (Reflect.get(current, "hash") !== view.hash && !(row.status === "processing" && row.result?.phase === "update_attempted"))) fail("form_close_view_changed"); + if (row.status === "cancelled" && row.result?.phase === "closed" && row.result.terminalIntent) return true; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const identity = holdIdentity(row); + const terminalIntent: SlackFormTerminalIntent = { ...identity, view, outcome: "closed", submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ status: "cancelled", result: { phase: "closed", view, terminalIntent }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), eq(chatActions.principalId, scope.principalId))); + return true; +} + +export async function slackFormSessionsRequireDelivery(db: Pick): Promise { + const rows = await db.select({ id: chatActions.id }).from(chatActions).where(and(eq(chatActions.kind, "slack_form_session"), + sql`(${chatActions.status} in ('processing','attention') OR (${chatActions.result}->'terminalIntent' is not null AND ${chatActions.result}->'cloudProofAcceptance' is null))`)).limit(1); + return rows.length > 0; +} + +export type SlackFormTerminalIntent = { + version: 1; binding: SlackFormBinding; operationId: string; deliveryId: string; userId: string; grantId: string; + publicationId: string; interactionId: string; view: SlackFormView | null; appReceiptId: string; + outcome: "closed" | "submitted" | "no_effect"; submitReceiptId: string | null; rejectionDigest: string | null; + encryptedEvidence: StoredSecretVersionMaterial | null; +}; +export type SlackFormEvidenceCipher = ReturnType; +const digest = (value: string) => createHash("sha256").update(value).digest("hex"); +function bodyRecord(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) fail("form_terminal_invalid"); + return value as Record; +} +function holdIdentity(row: Pick) { + const p = row.payload, binding = parseSlackFormBinding(p.binding); + if (binding.companyId !== row.companyId || binding.endpointId !== row.endpointId || typeof p.operationId !== "string" + || !/^[a-f0-9]{64}$/.test(p.operationId) || row.providerActionId !== `slack-form-session:${p.operationId}` + || ["userId", "grantId", "publicationId", "interactionId"].some(k => typeof p[k] !== "string" || !(p[k] as string).length)) fail("form_hold_identity_changed"); + return { version: 1 as const, binding, operationId: p.operationId, deliveryId: `slack_form_${p.operationId}`, userId: String(p.userId), + grantId: String(p.grantId), publicationId: String(p.publicationId), interactionId: String(p.interactionId), appReceiptId: row.id }; +} +type TerminalIdentity = Pick; +export function prepareSlackFormSubmissionIntent(identity: TerminalIdentity, view: SlackFormView, submitReceiptId: string, + original: NonNullable>, cipher: SlackFormEvidenceCipher): SlackFormTerminalIntent { + let encryptedEvidence: StoredSecretVersionMaterial; + try { encryptedEvidence = cipher.encrypt(JSON.stringify({ ...identity, kind: "submission", view, submitReceiptId, original })); } + catch { return fail("form_submission_cipher_unavailable"); } + return { ...identity, view, outcome: "submitted", submitReceiptId, rejectionDigest: null, encryptedEvidence }; +} +export function openSlackFormSubmissionIntent(intent: SlackFormTerminalIntent, cipher: SlackFormEvidenceCipher): { + acceptedAt: string; rawBody: string; timestamp: string; signature: string; +} { + let decrypted: Record; + try { decrypted = bodyRecord(JSON.parse(cipher.decrypt(bodyRecord(intent.encryptedEvidence)))); } + catch { return fail("form_submission_cipher_unavailable"); } + if (intent.outcome !== "submitted" || decrypted.version !== 1 || decrypted.kind !== "submission" + || ["appReceiptId", "operationId", "deliveryId", "userId", "grantId", "publicationId", "interactionId", "submitReceiptId"].some(k => decrypted[k] !== Reflect.get(intent, k)) + || !sameSlackFormBinding(parseSlackFormBinding(decrypted.binding), intent.binding) || !isDeepStrictEqual(decrypted.view, intent.view)) fail("form_submission_cipher_binding_changed"); + const original = bodyRecord(decrypted.original); + if (original.endpointId !== intent.binding.endpointId || ["acceptedAt", "rawBody", "timestamp", "signature"].some(k => typeof original[k] !== "string")) + fail("form_submission_cipher_binding_changed"); + return { acceptedAt: String(original.acceptedAt), rawBody: String(original.rawBody), timestamp: String(original.timestamp), signature: String(original.signature) }; +} +export async function stageSlackFormSubmission(tx: FormTx, args: { + companyId: string; endpointId: string; conversationId: string; principalId: string; authorizedUserId: string; runtimeGeneration: number; + submitReceiptId: string; publicationId: string; resolved: IssueThreadInteraction; cipher?: SlackFormEvidenceCipher; +}): Promise { + const original = slackFormSubmissionEvidence(); + if (!original) return null; + if (original.endpointId !== args.endpointId) fail("form_submission_endpoint_changed"); + const payload = bodyRecord(JSON.parse(new URLSearchParams(Buffer.from(original.rawBody, "base64").toString("utf8")).get("payload") ?? "null")); + const view = bodyRecord(payload.view), user = bodyRecord(payload.user), team = bodyRecord(payload.team); + if (payload.type !== "view_submission" || typeof view.external_id !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(view.external_id)) fail("form_submission_identity_changed"); + const operationId = view.external_id.slice("pc-form-".length); + const [hold] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, args.companyId), eq(chatActions.endpointId, args.endpointId), + eq(chatActions.conversationId, args.conversationId), eq(chatActions.principalId, args.principalId), eq(chatActions.kind, "slack_form_session"), + eq(chatActions.providerActionId, `slack-form-session:${operationId}`))).for("update"); + if (!hold) fail("form_submission_hold_missing"); + const identity = holdIdentity(hold), expectedView = bodyRecord(hold.result?.view); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, args.endpointId), eq(chatEndpoints.companyId, args.companyId))).limit(1); + const setup: Record | undefined = endpoint?.setup && typeof endpoint.setup === "object" && !Array.isArray(endpoint.setup) ? { ...endpoint.setup } : undefined, owner = setup ? currentSlackFormOwnerBinding(setup) : null; + if (!endpoint || !owner || !sameSlackFormBinding(owner, identity.binding) || args.runtimeGeneration !== identity.binding.runtimeGeneration + || setup?.runtimeGeneration !== args.runtimeGeneration || endpoint.connectionId !== identity.binding.connectionId || endpoint.publicId !== identity.binding.publicId) + fail("form_submission_current_generation_changed"); + const [receipt] = await tx.select().from(chatActions).where(and(eq(chatActions.id, args.submitReceiptId), eq(chatActions.companyId, args.companyId), + eq(chatActions.endpointId, args.endpointId), eq(chatActions.conversationId, args.conversationId), eq(chatActions.principalId, args.principalId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.status, "processed"))).limit(1); + if (!receipt || receipt.result?.code !== "question_form_answered" || receipt.result.interactionId !== args.resolved.id + || receipt.payload.publicationId !== args.publicationId || receipt.providerActionId !== view.callback_id || hold.payload.submitActionRowId !== receipt.id + || args.resolved.companyId !== args.companyId || args.resolved.id !== identity.interactionId || args.resolved.status !== "answered" + || args.resolved.resolvedByUserId !== args.authorizedUserId || args.publicationId !== identity.publicationId + || user.id !== identity.userId || team.id !== identity.binding.teamId || payload.api_app_id !== identity.binding.appId || view.app_id !== identity.binding.appId + || view.id !== expectedView.id || view.external_id !== expectedView.externalId || typeof view.hash !== "string" + || (view.hash !== expectedView.hash && hold.result?.phase !== "update_attempted")) fail("form_submission_receipt_changed"); + if (hold.status === "processed" && hold.result?.phase === "submitted") { + const old = bodyRecord(hold.result.terminalIntent); + if (old.submitReceiptId !== receipt.id || old.appReceiptId !== hold.id || old.operationId !== operationId) fail("form_submission_receipt_changed"); + return hold.id; + } + if (hold.status !== "processing" || !["ready", "update_attempted"].includes(String(hold.result?.phase))) fail("form_submission_hold_changed"); + const currentView = { id: String(view.id), hash: String(view.hash), externalId: String(view.external_id) }; + let intent: SlackFormTerminalIntent; + try { intent = prepareSlackFormSubmissionIntent(identity, currentView, receipt.id, original, args.cipher ?? createLocalEncryptedCipher("read")); } + catch { return fail("form_submission_cipher_unavailable"); } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "processed", result: { phase: "submitted", view: currentView, terminalIntent: intent }, updatedAt: new Date() }).where(and( + eq(chatActions.id, hold.id), eq(chatActions.companyId, args.companyId), eq(chatActions.status, "processing"))); + return hold.id; +} +export async function acceptSlackFormNoEffect(tx: FormTx, operation: SlackFormOperation, originalActionId: string, environment?: string): Promise { + if (operation.phase !== "rejected" || !operation.rejection || operation.view || operation.closed) fail("form_no_effect_unproven"); + const binding = operation.binding; + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_open"), eq(chatActions.providerActionId, originalActionId))).limit(1); + const [submit] = open ? await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.providerActionId, String(open.payload.formActionId)), eq(chatActions.conversationId, open.conversationId!))).limit(1) : []; + if (!open || !submit || !open.conversationId || open.payload.version !== 1 || submit.payload.version !== 1 + || typeof open.payload.publicationId !== "string" || typeof open.payload.interactionId !== "string" + || submit.payload.publicationId !== open.payload.publicationId || submit.payload.interactionId !== open.payload.interactionId + || slackIssuedFormGrantId(binding, open.id, submit.id, open.payload.publicationId, open.payload.interactionId) !== operation.grantId) + fail("form_no_effect_issued_authority_changed"); + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, open.payload.publicationId), + eq(chatPublications.companyId, binding.companyId), eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.conversationId, open.conversationId))).limit(1); + if (!publication?.providerMessageId) fail("form_no_effect_publication_changed"); + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).for("update"); + if (old) { + const intent = bodyRecord(old.result?.terminalIntent); + if (old.status !== "failed" || old.result?.phase !== "not_opened" || intent.rejectionDigest !== operation.rejection.responseDigest + || intent.operationId !== operation.id || intent.appReceiptId !== old.id) fail("form_no_effect_conflicts_custody"); + return old.id; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, conversationId: open.conversationId, + kind: "slack_form_session", providerActionId: `slack-form-session:${operation.id}`, status: "failed", payload: { version: 1, binding, environment, operationId: operation.id, + grantId: operation.grantId, requestDigest: operation.requestDigest, userId: operation.userId, publicationId: open.payload.publicationId, + interactionId: open.payload.interactionId, openActionRowId: open.id, submitActionRowId: submit.id, view: null }, result: { phase: "not_opened" } }) + .onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_no_effect_claim_changed", "retry"); + const intent: SlackFormTerminalIntent = { version: 1, binding, operationId: operation.id, deliveryId: operation.deliveryId, userId: operation.userId, + grantId: operation.grantId, publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, appReceiptId: row.id, + outcome: "no_effect", submitReceiptId: null, rejectionDigest: operation.rejection.responseDigest, view: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ result: { phase: "not_opened", terminalIntent: intent }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId))); + const credentialFailure = ["invalid_auth", "token_revoked", "not_authed", "invalid_arguments"].includes(operation.rejection.code); + const lastError = credentialFailure ? "Slack could not open the form. Repair the connection, then open the form again." + : "The Slack form request expired before opening. Open the form again to create a fresh request."; + await tx.update(chatEndpoints).set({ ...(credentialFailure ? { status: "attention" } : {}), lastError, updatedAt: new Date() }).where(and( + eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId), eq(chatEndpoints.connectionId, binding.connectionId), + eq(chatEndpoints.publicId, binding.publicId), inArray(chatEndpoints.status, ["active", "verifying", "attention"]), + sql`${chatEndpoints.setup}->'slackCloudForms' = ${JSON.stringify(binding)}::jsonb`)); + return row.id; +} +export async function readSlackFormTerminalRequest(db: Pick, binding: SlackFormBinding, operationId: string, + cipher?: SlackFormEvidenceCipher): Promise | null> { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operationId}`))).limit(1); + if (!row || !["closed", "submitted", "not_opened"].includes(String(row.result?.phase))) return null; + const identity = holdIdentity(row), intent = bodyRecord(row.result?.terminalIntent); + if (!sameSlackFormBinding(identity.binding, binding) || intent.appReceiptId !== row.id || intent.operationId !== operationId + || intent.grantId !== identity.grantId || intent.publicationId !== identity.publicationId || intent.interactionId !== identity.interactionId + || intent.userId !== identity.userId || !sameSlackFormBinding(parseSlackFormBinding(intent.binding), binding)) fail("form_terminal_intent_changed"); + const view = intent.view === null ? null : bodyRecord(intent.view); + let terminal: Record; + if (intent.outcome === "submitted") { + if (intent.submitReceiptId !== row.payload.submitActionRowId) fail("form_submission_cipher_binding_changed"); + let original: ReturnType; + try { original = openSlackFormSubmissionIntent(intent as unknown as SlackFormTerminalIntent, cipher ?? createLocalEncryptedCipher("read")); } + catch (error) { if (error instanceof SlackCloudReceiptError) throw error; return fail("form_submission_cipher_unavailable"); } + terminal = { kind: "submitted", submitReceiptId: intent.submitReceiptId, evidence: original }; + } else if (intent.outcome === "no_effect") { + if (typeof intent.rejectionDigest !== "string" || !/^[a-f0-9]{64}$/.test(intent.rejectionDigest)) fail("form_terminal_intent_changed"); + terminal = { kind: "no_effect", responseDigest: intent.rejectionDigest }; + } else if (intent.outcome === "closed") terminal = { kind: "closed" }; + else fail("form_terminal_intent_changed"); + return { kind: "terminal-ack", binding, operationId, deliveryId: identity.deliveryId, userId: identity.userId, grantId: identity.grantId, + publicationId: identity.publicationId, interactionId: identity.interactionId, viewId: view?.id ?? null, viewHash: view?.hash ?? null, + externalId: view?.externalId ?? `pc-form-${operationId}`, appReceiptId: row.id, terminal: terminal! }; +} + +export type SlackFormProofAcceptance = { version: 1; binding: SlackFormBinding; operationId: string; appReceiptId: string; + outcome: SlackFormTerminalIntent["outcome"]; terminalDigest: string; acceptedAt: string }; +export function parseSlackFormProofAcceptance(value: unknown): SlackFormProofAcceptance { + const v = bodyRecord(value); + if (Object.keys(v).sort().join() !== ["version", "binding", "operationId", "appReceiptId", "outcome", "terminalDigest", "acceptedAt"].sort().join() + || v.version !== 1 || typeof v.operationId !== "string" || !/^[a-f0-9]{64}$/.test(v.operationId) + || typeof v.appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(v.appReceiptId) + || typeof v.terminalDigest !== "string" || !/^[a-f0-9]{64}$/.test(v.terminalDigest) + || typeof v.acceptedAt !== "string" || !Number.isFinite(Date.parse(v.acceptedAt)) + || v.outcome !== "closed" && v.outcome !== "submitted" && v.outcome !== "no_effect") fail("form_proof_acceptance_invalid"); + return { version: 1, binding: parseSlackFormBinding(v.binding), operationId: v.operationId, appReceiptId: v.appReceiptId, + outcome: v.outcome, terminalDigest: v.terminalDigest, acceptedAt: v.acceptedAt }; +} +export function prepareSlackFormProofAcceptance(intent: SlackFormTerminalIntent, value: unknown, currentInstanceId: string, + cipher?: SlackFormEvidenceCipher): SlackFormProofAcceptance { + const op = parseSlackFormOperation(value); + if (op.binding.instanceId !== currentInstanceId || !sameSlackFormBinding(op.binding, intent.binding) + || op.id !== intent.operationId || op.deliveryId !== intent.deliveryId || op.grantId !== intent.grantId || op.userId !== intent.userId) + fail("form_proof_identity_changed"); + assertSlackFormTerminalAck(op, intent.appReceiptId); + let terminalDigest: string; + if (intent.outcome === "no_effect") { + if (!op.rejection || op.closed || op.view || intent.view || op.rejection.responseDigest !== intent.rejectionDigest) fail("form_proof_outcome_changed"); + terminalDigest = op.rejection.responseDigest; + } else { + if (!op.closed || op.rejection || !intent.view || !isDeepStrictEqual(op.closed.view, intent.view) + || op.closed.kind !== (intent.outcome === "submitted" ? "submitted" : "closed")) fail("form_proof_outcome_changed"); + if (intent.outcome === "submitted") { + if (op.closed.evidence.submitReceiptId !== intent.submitReceiptId) fail("form_proof_submission_changed"); + const original = openSlackFormSubmissionIntent(intent, cipher ?? createLocalEncryptedCipher("read")); + if (digest(JSON.stringify([original.rawBody, original.timestamp, original.signature])) !== op.closed.requestDigest) + fail("form_proof_submission_changed"); + } + terminalDigest = op.closed.requestDigest; + } + if (!op.acknowledgement) fail("form_proof_missing"); + return { version: 1, binding: intent.binding, operationId: op.id, appReceiptId: intent.appReceiptId, outcome: intent.outcome, + terminalDigest, acceptedAt: op.acknowledgement.acceptedAt }; +} +export async function acceptSlackFormTerminalProof(tx: FormTx, input: { intent: SlackFormTerminalIntent; operation: unknown; + currentInstanceId: string; cipher?: SlackFormEvidenceCipher }): Promise { + const { intent } = input; + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, intent.appReceiptId), eq(chatActions.companyId, intent.binding.companyId), + eq(chatActions.endpointId, intent.binding.endpointId), eq(chatActions.kind, "slack_form_session"), + eq(chatActions.providerActionId, `slack-form-session:${intent.operationId}`))).for("update"); + if (!row || !["closed", "submitted", "not_opened"].includes(String(row.result?.phase)) + || !isDeepStrictEqual(row.result?.terminalIntent, intent)) fail("form_proof_intent_changed"); + const identity = holdIdentity(row); + if (identity.appReceiptId !== intent.appReceiptId || !sameSlackFormBinding(identity.binding, intent.binding) + || identity.grantId !== intent.grantId || identity.userId !== intent.userId || identity.publicationId !== intent.publicationId + || identity.interactionId !== intent.interactionId) fail("form_proof_intent_changed"); + const accepted = prepareSlackFormProofAcceptance(intent, input.operation, input.currentInstanceId, input.cipher); + if (row.result?.cloudProofAcceptance !== undefined) { + if (!isDeepStrictEqual(parseSlackFormProofAcceptance(row.result.cloudProofAcceptance), accepted)) fail("form_proof_acceptance_changed"); + return accepted; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...row.result, cloudProofAcceptance: accepted }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, intent.binding.companyId), + sql`${chatActions.result}->'terminalIntent' = ${JSON.stringify(intent)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw new SlackCloudReceiptError("form_proof_acceptance_changed", "retry"); + return accepted; +} +export type SlackFormRetainedRow = Pick; +export function projectSlackFormRetainedOwner(row: SlackFormRetainedRow, currentInstanceId: string, currentEnvironment?: string): { + subject: "github-bot-events"; companyId: string; binding: SlackFormBinding | null; requiresDelivery: boolean; +} | null { + if (["slack_form_owner", "slack_form_grant"].includes(row.kind) && row.result?.phase === "transferred" + && row.result.retirement && typeof row.result.retirement === "object" && Reflect.get(row.result.retirement, "receipt") !== undefined) { + if (!slackFormRetirementHistoryReceipt(row)) fail("form_retained_owner_changed"); + return null; + } + if (row.kind === "slack_form_owner" && (row.result?.phase === "retirement_accepted" || row.result?.phase === "retirement_gc_pending")) { + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt), enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), binding = receipt.binding; + if (receipt.binding.instanceId !== currentInstanceId || currentEnvironment !== undefined && receipt.environment !== currentEnvironment) return null; + if (row.companyId !== binding.companyId || row.endpointId !== binding.endpointId || row.payload.subject !== receipt.subject + || !isSlackFormEnrollmentActionId(row.providerActionId, enrollment) + || row.payload.environment !== receipt.environment || typeof row.result.retirementReceiptJws !== "string" + || Buffer.byteLength(row.result.retirementReceiptJws, "ascii") > 12000 + || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(row.result.retirementReceiptJws) + || enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(enrollment, key) !== Reflect.get(binding, key)) + || row.payload.binding !== undefined && !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), binding)) fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: false }; + } + if (row.kind === "slack_form_owner" && row.payload.binding === undefined) { + const rawEnrollment = row.payload.enrollment; + if (!rawEnrollment || typeof rawEnrollment !== "object" || Array.isArray(rawEnrollment) || Reflect.get(rawEnrollment, "instanceId") !== currentInstanceId) return null; + const enrollment = parseSlackFormPublicEnrollment(rawEnrollment); + if (currentEnvironment !== undefined && row.payload.environment !== currentEnvironment) fail("form_retained_environment_changed"); + if (enrollment.companyId !== row.companyId || enrollment.endpointId !== row.endpointId + || typeof row.payload.subject !== "string" || !row.payload.subject.length + || row.result?.phase !== "staged") fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding: null, requiresDelivery: true }; + } + const raw = row.payload.binding; + if (!raw || typeof raw !== "object" || Array.isArray(raw) || Reflect.get(raw, "instanceId") !== currentInstanceId) return null; + if (currentEnvironment !== undefined && row.payload.environment !== currentEnvironment) fail("form_retained_environment_changed"); + const binding = parseSlackFormBinding(raw); + if (binding.companyId !== row.companyId || binding.endpointId !== row.endpointId) fail("form_retained_owner_changed"); + if (row.kind === "slack_form_session") { + const identity = holdIdentity(row); + if (!sameSlackFormBinding(identity.binding, binding)) fail("form_retained_owner_changed"); + if (row.result?.cloudProofAcceptance !== undefined) { + const accepted = parseSlackFormProofAcceptance(row.result.cloudProofAcceptance), intent = bodyRecord(row.result.terminalIntent); + if (accepted.appReceiptId !== row.id || accepted.operationId !== identity.operationId || intent.appReceiptId !== row.id + || intent.operationId !== identity.operationId || !sameSlackFormBinding(parseSlackFormBinding(accepted.binding), binding) + || !["closed", "submitted", "not_opened"].includes(String(row.result.phase))) fail("form_retained_owner_changed"); + } + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.cloudProofAcceptance === undefined }; + } + if (row.kind === "slack_form_grant") { + const grant = bodyRecord(row.payload.grant); + if (!sameSlackFormBinding(parseSlackFormBinding(grant.binding), binding) || typeof row.payload.subject !== "string" + || !["staged", "published"].includes(String(row.result?.phase))) fail("form_retained_grant_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.phase === "staged" }; + } + if (row.kind !== "slack_form_owner" || typeof row.payload.subject !== "string" || !row.payload.subject.length + || !["staged", "enrolled", "retired_cleanup", "transferred"].includes(String(row.result?.phase))) fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.phase !== "transferred" }; +} +export async function readSlackFormRetainedOwner(db: Pick, currentInstanceId: string) { + const [row] = await db.select().from(chatActions).where(and(inArray(chatActions.kind, ["slack_form_session", "slack_form_owner"]), + sql`${chatActions.payload}->'binding'->>'instanceId' = ${currentInstanceId}`, + sql`${chatActions.payload}->'binding'->>'companyId' = ${chatActions.companyId}::text`, + sql`${chatActions.payload}->'binding'->>'endpointId' = ${chatActions.endpointId}::text`)) + .orderBy(sql`case when ${chatActions.result}->'cloudProofAcceptance' is null and coalesce(${chatActions.result}->>'phase','') <> 'transferred' then 0 else 1 end`, chatActions.id).limit(1); + return row ? projectSlackFormRetainedOwner(row, currentInstanceId) : null; +} + +export async function acceptSlackFormClosed(tx: FormTx, operation: SlackFormOperation, originalActionId: string, environment: string): Promise { + if (!operation.closed || operation.closed.kind !== "closed" || operation.rejection) fail("form_close_unproven"); + const binding = operation.binding; + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).for("update"); + if (old) { + if (!sameSlackFormBinding(parseSlackFormBinding(old.payload.binding), binding) || old.payload.userId !== operation.userId) fail("form_close_owner_changed"); + if (old.principalId) { + await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: old.principalId, operationId: operation.id }, operation.closed.view); + return old.id; + } + if (old.status !== "cancelled" || old.result?.phase !== "closed" || !isDeepStrictEqual(old.result.view, operation.closed.view)) fail("form_close_owner_changed"); + return old.id; + } + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_open"), eq(chatActions.providerActionId, originalActionId))).limit(1); + const [submit] = open?.conversationId ? await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.providerActionId, String(open.payload.formActionId)), eq(chatActions.conversationId, open.conversationId))).limit(1) : []; + if (!open || !submit || !open.conversationId || typeof open.payload.publicationId !== "string" || typeof open.payload.interactionId !== "string" + || submit.payload.publicationId !== open.payload.publicationId || submit.payload.interactionId !== open.payload.interactionId + || slackIssuedFormGrantId(binding, open.id, submit.id, open.payload.publicationId, open.payload.interactionId) !== operation.grantId) fail("form_close_issued_authority_changed"); + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, open.payload.publicationId), eq(chatPublications.companyId, binding.companyId), + eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.conversationId, open.conversationId))).limit(1); + if (!publication?.providerMessageId) fail("form_close_publication_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, conversationId: open.conversationId, + kind: "slack_form_session", providerActionId: `slack-form-session:${operation.id}`, status: "cancelled", payload: { version: 1, binding, environment, + operationId: operation.id, grantId: operation.grantId, requestDigest: operation.requestDigest, userId: operation.userId, + publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, openActionRowId: open.id, submitActionRowId: submit.id, view: operation.closed.view }, + result: { phase: "closed", view: operation.closed.view } }).onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_close_claim_changed", "retry"); + const intent: SlackFormTerminalIntent = { version: 1, binding, operationId: operation.id, deliveryId: operation.deliveryId, userId: operation.userId, + grantId: operation.grantId, publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, appReceiptId: row.id, + outcome: "closed", view: operation.closed.view, submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ result: { phase: "closed", view: operation.closed.view, terminalIntent: intent }, updatedAt: new Date() }).where(eq(chatActions.id, row.id)); + return row.id; +} +export async function readSlackFormTerminalIntent(db: Pick, operation: SlackFormOperation): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, operation.binding.companyId), eq(chatActions.endpointId, operation.binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).limit(1); + if (!row?.result?.terminalIntent) return null; + const raw = bodyRecord(row.result.terminalIntent), identity = holdIdentity(row); + if (!sameSlackFormBinding(identity.binding, operation.binding) || identity.operationId !== operation.id || identity.grantId !== operation.grantId + || identity.userId !== operation.userId || Object.keys(identity).some(k => !isDeepStrictEqual(raw[k], Reflect.get(identity, k)))) fail("form_terminal_intent_changed"); + if (!["closed", "submitted", "no_effect"].includes(String(raw.outcome)) || (raw.submitReceiptId !== null && typeof raw.submitReceiptId !== "string") + || (raw.rejectionDigest !== null && typeof raw.rejectionDigest !== "string")) fail("form_terminal_intent_changed"); + const view = raw.view === null ? null : bodyRecord(raw.view); + if (view && (typeof view.id !== "string" || typeof view.hash !== "string" || typeof view.externalId !== "string")) fail("form_terminal_intent_changed"); + return { ...identity, outcome: raw.outcome as SlackFormTerminalIntent["outcome"], submitReceiptId: raw.submitReceiptId, + rejectionDigest: raw.rejectionDigest, view: view ? { id: String(view.id), hash: String(view.hash), externalId: String(view.externalId) } : null, + encryptedEvidence: raw.encryptedEvidence === null ? null : bodyRecord(raw.encryptedEvidence) }; +} diff --git a/server/src/services/chat-channels/slack/form-signature-refresh.ts b/server/src/services/chat-channels/slack/form-signature-refresh.ts new file mode 100644 index 00000000000..5c47a4936cf --- /dev/null +++ b/server/src/services/chat-channels/slack/form-signature-refresh.ts @@ -0,0 +1,29 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; + +type Row = typeof chatActions.$inferSelect; +export async function replaceSlackFormReceiptSignature(db: Db, row: Row, field: "retirementReceiptJws" | "formAck", compactJws: string): Promise { + const old = field === "retirementReceiptJws" ? row.result?.retirementReceiptJws + : row.result?.formAck && typeof row.result.formAck === "object" ? Reflect.get(row.result.formAck, "receiptJws") : undefined; + if (typeof old !== "string" || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(compactJws) || Buffer.byteLength(compactJws, "ascii") > 12000) + throw new SlackCloudReceiptError("form_signature_refresh_invalid", "attention"); + if (old === compactJws) return row; + return db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || current.kind !== row.kind || current.providerActionId !== row.providerActionId + || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) + throw new SlackCloudReceiptError("form_signature_refresh_changed", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const result = field === "retirementReceiptJws" + ? sql`jsonb_set(${chatActions.result}, '{retirementReceiptJws}', ${JSON.stringify(compactJws)}::jsonb, false)` + : sql`jsonb_set(${chatActions.result}, '{formAck,receiptJws}', ${JSON.stringify(compactJws)}::jsonb, false)`; + const [updated] = await tx.update(chatActions).set({ result, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), + eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), eq(chatActions.kind, row.kind), eq(chatActions.providerActionId, row.providerActionId), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning(); + if (!updated) throw new SlackCloudReceiptError("form_signature_refresh_changed", "retry"); + return updated; + }); +} diff --git a/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts b/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts new file mode 100644 index 00000000000..ef7f9e3f13c --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts @@ -0,0 +1,30 @@ +import { expect, it } from "vitest"; +import { PgDialect } from "drizzle-orm/pg-core"; +import { idleChatHistoryChecks, idleChatHistoryChecksForOwner } from "../idle-history-proof.js"; +it("absence retains the exact existing chat history query objects", () => { + expect(idleChatHistoryChecksForOwner()).toBe(idleChatHistoryChecks); + expect(idleChatHistoryChecksForOwner([])).toBe(idleChatHistoryChecks); +}); +it("the same validated bound action identity appears in action and delivery queries with normal graph checks", () => { + const id = "11111111-1111-4111-8111-111111111111", checks = idleChatHistoryChecksForOwner([id, id]); + for (const reason of ["chat_actions", "chat_deliveries"]) { + const check = checks.find(check => check.reason === reason); if (!check || typeof check.query === "string") throw new Error("Parameterized owner query missing"); + const rendered = new PgDialect().sqlToQuery(check.query); + expect(rendered.params).toEqual([[id]]); + expect(rendered.sql).not.toContain(id); + expect(rendered.sql).toContain("e.company_id = a.company_id"); + expect(rendered.sql).toContain("d.company_id = a.company_id"); + expect(rendered.sql).toContain("unnest($1::uuid[])"); + } + expect(() => idleChatHistoryChecksForOwner(["foreign' OR true"])).toThrow(); + +}); + +it("retained completion identity counts do not become SQL literal or parameter ceilings",()=>{ + const ids=Array.from({length:65536},(_,index)=>`11111111-1111-4111-8111-${String(index).padStart(12,"0")}`); + const check=idleChatHistoryChecksForOwner(ids).find(value=>value.reason==="chat_actions"); + if(!check||typeof check.query==="string")throw new Error("Parameterized owner query absent"); + const rendered=new PgDialect().sqlToQuery(check.query); + expect(rendered.params).toEqual([ids]);expect(rendered.sql).not.toContain(ids[0]); + expect(rendered.sql).toContain("unnest($1::uuid[])"); +}); diff --git a/server/src/services/chat-channels/slack/idle-incoming.test.ts b/server/src/services/chat-channels/slack/idle-incoming.test.ts new file mode 100644 index 00000000000..777ce540fde --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-incoming.test.ts @@ -0,0 +1,64 @@ +import { slackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { expect, it } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { CAPABILITIES } from "../provider-catalog.js"; +import { verifyCloudControlAssertion, CLOUD_CONTROL_AUDIENCE, CLOUD_CONTROL_JWS_TYPE } from "../../cloud-runtime-identity.js"; +import type { IdleCapabilities, VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { slackConfiguredBindingsFingerprint } from "./configured-coverage.js"; +import { readSlackIdleIncomingProof, projectSlackIdleIncomingRows, type SlackIdleEndpoint, type SlackIdleConnection } from "./idle-incoming.js"; +import type { SlackFormHistoryRow } from "./form-idle-history.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "11111111-1111-4111-8111-111111111111", endpointId: "22222222-2222-4222-8222-222222222222", publicId: "p".repeat(43), connectionId: "33333333-3333-4333-8333-333333333333", runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +function verifiedPartial(): VerifiedIdleCapabilities { + const key = generateKeyPairSync("ed25519"), kid = "slack-idle-unit", now = Math.floor(Date.now() / 1000), generation = "c".repeat(64); + const caps: IdleCapabilities = { incomingWake: { version: 1, ownerId: randomUUID(), generation, routes: ["slack-broker-events-v1"], + slackBroker: { version: 1, instanceId: binding.instanceId, generation, bindingsFingerprint: slackConfiguredBindingsFingerprint([binding]), entries: [{ binding, lanes: ["events", "forms", "modal_close", "modal_submit"] }] } } }; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const body = encode({ alg: "EdDSA", typ: CLOUD_CONTROL_JWS_TYPE, kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: CLOUD_CONTROL_AUDIENCE, + sub: binding.instanceId, action: "task-drain:read", requestId: randomUUID(), iat: now, exp: now + 60, idleCapabilities: caps }); + const claims = verifyCloudControlAssertion({ compactJws: body + "." + sign(null, Buffer.from(body), key.privateKey).toString("base64url"), expectedAction: "task-drain:read", + env: { PAPERCLIP_CLOUD_STACK_ID: binding.instanceId, PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...key.publicKey.export({ format: "jwk" }), kid, use: "sig", alg: "EdDSA" }] }) } }); + if (!claims.idleCapabilities) throw new Error("Actual signed idle capability absent"); + return claims.idleCapabilities as VerifiedIdleCapabilities; +} +function rows() { + const { credentialFingerprint: _, eventBindingId, ...base } = binding, enrollment = { ...base, bindingId: eventBindingId }; + const endpoint: SlackIdleEndpoint = { id: binding.endpointId, companyId: binding.companyId, connectionId: binding.connectionId, publicId: binding.publicId, + provider: "slack", status: "active", capabilities: CAPABILITIES.slack, setup: { step: "complete", runtimeGeneration: 1, slackCloudInstanceId: binding.instanceId, + slackCloudSubject: "actor", slackCloudForms: binding, slackCloudEvents: { runtimeGeneration: 1, bindingId: binding.eventBindingId, teamId: binding.teamId, appId: binding.appId } } }; + const connection: SlackIdleConnection = { id: binding.connectionId, companyId: binding.companyId, status: "active", enabled: true, transport: "chat_sdk", connectionPurpose: "channel" }; + const owner: SlackFormHistoryRow = { id: "44444444-4444-4444-8444-444444444444", companyId: binding.companyId, endpointId: binding.endpointId, kind: "slack_form_owner", principalId: null, conversationId: null, + status: "issued", providerActionId: "slack-form-owner:" + "d".repeat(64), payload: { enrollment, binding, subject: "actor", environment: "test" }, result: { phase: "enrolled", nextAttemptAt: null } }; + return { endpoints: [{ endpoint, connection }], actions: [owner] }; +} +it("absent signed Slack proof performs zero query and budget operations", async () => { + let calls = 0; + const tx: Pick = { select: () => { calls++; throw new Error("No absent-feature query permitted"); } }; + expect(await readSlackIdleIncomingProof(tx, { receiverBound: true, beforeRead: async () => { calls++; }, remaining: () => { calls++; return 1; }, registeredOptions: () => false })).toBeUndefined(); + expect(calls).toBe(0); + expect(await readSlackIdleIncomingProof(tx, { capabilities: verifiedPartial(), identity: { instanceId: binding.instanceId, environment: "test" }, receiverBound: false, + beforeRead: async () => { calls++; }, remaining: () => { calls++; return 1; }, registeredOptions: () => false })).toBeUndefined(); + expect(calls).toBe(0); +}); +it("genuinely signed forms-events-only coverage keeps default commands and general actions blocked", () => { + const result = projectSlackIdleIncomingRows({ ...rows(), capabilities: verifiedPartial(), identity: { instanceId: binding.instanceId, environment: "test" }, registeredOptions: () => false }); + expect(result).toEqual({ endpointIds: [], connectionIds: [], completedActionIds: [] }); +}); +it("foreign company and duplicate current owners never become current Slack completion authority", () => { + const current = rows(), capabilities = verifiedPartial(), common = { capabilities, identity: { instanceId: binding.instanceId, environment: "test" as const }, registeredOptions: () => false }; + expect(projectSlackIdleIncomingRows({ ...current, ...common, endpoints: [{ ...current.endpoints[0]!, connection: { ...current.endpoints[0]!.connection, companyId: "foreign" } }] })).toBeUndefined(); + expect(projectSlackIdleIncomingRows({ ...current, ...common, actions: Array.from({ length: 1001 }, () => current.actions[0]!) })).toBeUndefined(); +}); + +it("retained completed receipts exceed one history page without enlarging the current endpoint cap", () => { + const current=rows(),receipt=slackFormRetirementReceipt(binding,"test","actor"); + const history:SlackFormHistoryRow[]=Array.from({length:1005},()=>({id:randomUUID(),companyId:binding.companyId,endpointId:binding.endpointId, + kind:"slack_form_grant",principalId:null,conversationId:null,status:"cancelled",providerActionId:"slack-form-grant:"+randomUUID(), + payload:{binding,subject:"actor",environment:"test"},result:{phase:"transferred",nextAttemptAt:null, + retirement:{version:1,binding,instanceId:binding.instanceId,environment:"test",subject:"actor",receiptDigest:receipt.receiptId,receipt}}})); + const common={capabilities:verifiedPartial(),identity:{instanceId:binding.instanceId,environment:"test" as const},registeredOptions:()=>false}; + const result=projectSlackIdleIncomingRows({...current,...common,actions:[...current.actions,...history]}); + expect(result?.completedActionIds).toEqual(history.map(row=>row.id).sort()); + expect(projectSlackIdleIncomingRows({...current,...common,endpoints:Array.from({length:1001},()=>current.endpoints[0]!)})).toBeUndefined(); +}); diff --git a/server/src/services/chat-channels/slack/idle-incoming.ts b/server/src/services/chat-channels/slack/idle-incoming.ts new file mode 100644 index 00000000000..262416c7e1d --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-incoming.ts @@ -0,0 +1,175 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, gt, inArray, sql } from "drizzle-orm"; +import { alias } from "drizzle-orm/pg-core"; +import { chatActions, chatEndpoints, toolConnections, type Db } from "@paperclipai/db"; +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { parseSlackFormBinding, parseSlackFormPublicEnrollment, slackFormSetupRecord, sameSlackFormBinding } from "./cloud-forms.js"; +import { matchSlackConfiguredCoverage, type SlackConfiguredRequirement } from "./configured-coverage.js"; +import { createSlackFormHistoryClassifier, type SlackFormHistoryRow } from "./form-idle-history.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +export type SlackIdleEndpoint = Pick & { setup: unknown; capabilities: unknown }; +export type SlackIdleConnection = Pick; +export type SlackIdleIncomingProjection = Readonly<{ endpointIds: readonly string[]; connectionIds: readonly string[]; completedActionIds: readonly string[] }>; +export const slackHasRegisteredOptionsAtCurrentBuild = (_endpointId: string): boolean => false; +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid Slack idle object"); + return Object.fromEntries(Object.entries(value)); +} +function flag(value: Record, key: string): boolean { + const v = value[key]; if (typeof v !== "boolean") throw new Error("Unknown Slack configured capability"); return v; +} +type SlackIncomingRowsInput = { + endpoints: readonly { endpoint: SlackIdleEndpoint; connection: SlackIdleConnection }[]; actions: readonly SlackFormHistoryRow[]; + capabilities: VerifiedIdleCapabilities; identity: SlackFormConnectorIdentity; registeredOptions(endpointId: string): boolean; check?(): void; +}; +function slackIncomingContext(input: SlackIncomingRowsInput) { + if (input.endpoints.length > 1000 || !input.capabilities.incomingWake?.routes.includes("slack-broker-events-v1")) return; + const requirements: SlackConfiguredRequirement[] = [], endpointIds = new Set(), currentOwners = new Map(); + const groups = groupHistory(input.actions); + try { + for (const { endpoint, connection } of input.endpoints) { + input.check?.(); + if (endpoint.provider !== "slack" || endpoint.companyId !== connection.companyId || endpoint.connectionId !== connection.id || connection.transport !== "chat_sdk" || connection.connectionPurpose !== "channel" || endpointIds.has(endpoint.id)) return; + endpointIds.add(endpoint.id); + if (endpoint.status !== "active" || connection.status !== "active" || !connection.enabled) continue; + const setup = slackFormSetupRecord(endpoint.setup), binding = parseSlackFormBinding(setup.slackCloudForms); + const capabilities = object(endpoint.capabilities), registeredOptions = input.registeredOptions(endpoint.id); + if (typeof registeredOptions !== "boolean" || setup.step !== "complete" || setup.slackCloudInstanceId !== binding.instanceId || setup.runtimeGeneration !== binding.runtimeGeneration + || binding.instanceId !== input.identity.instanceId || binding.companyId !== endpoint.companyId || binding.endpointId !== endpoint.id + || binding.connectionId !== connection.id || binding.publicId !== endpoint.publicId || typeof setup.slackCloudSubject !== "string") return; + const descriptor = object(setup.slackCloudEvents), enrollment = parseSlackFormPublicEnrollment({ version: 1, instanceId: input.identity.instanceId, + companyId: endpoint.companyId, endpointId: endpoint.id, publicId: endpoint.publicId, connectionId: endpoint.connectionId, + runtimeGeneration: descriptor.runtimeGeneration, bindingId: descriptor.bindingId, teamId: descriptor.teamId, appId: descriptor.appId }); + const { credentialFingerprint: _, eventBindingId, ...base } = binding; + if (!isDeepStrictEqual({ ...base, bindingId: eventBindingId }, enrollment)) return; + const owners = (groups.get(`${endpoint.companyId}:${endpoint.id}`) ?? []).filter(row => row.companyId === endpoint.companyId && row.endpointId === endpoint.id && row.kind === "slack_form_owner" + && /^slack-form-owner:[a-f0-9]{64}$/.test(row.providerActionId) && row.result?.phase === "enrolled" && row.result.nextAttemptAt === null && row.result.desiredActive !== false && ["issued", "processed"].includes(row.status) + && row.payload.environment === input.identity.environment && row.payload.subject === setup.slackCloudSubject + && isDeepStrictEqual(row.payload.binding, binding) && isDeepStrictEqual(row.payload.enrollment, enrollment)); + if (owners.length !== 1) return; + currentOwners.set(owners[0]!.id, owners[0]!); + requirements.push({ binding, capabilities: { actions: flag(capabilities, "actions"), modals: flag(capabilities, "modals"), slashCommands: flag(capabilities, "slashCommands") }, registeredOptions }); + } + const expected = { ...input.identity, generation: input.capabilities.incomingWake.generation }; + const matched = matchSlackConfiguredCoverage(requirements, input.capabilities, expected); + return { requirements, expected, matched, currentOwners }; + } catch { return; } +} + +function groupHistory(rows: readonly SlackFormHistoryRow[]) { + const groups = new Map(); + for (const row of rows) { + const key = `${row.companyId}:${row.endpointId}`, group = groups.get(key); + if (group) group.push(row); else groups.set(key, [row]); + } + return groups; +} +function projection(context: NonNullable>, completed: Set): SlackIdleIncomingProjection { + return Object.freeze({ endpointIds: Object.freeze([...(context.matched?.endpointIds ?? [])]), + connectionIds: Object.freeze([...(context.matched?.connectionIds ?? [])]), completedActionIds: Object.freeze([...completed].sort()) }); +} +export function projectSlackIdleIncomingRows(input: SlackIncomingRowsInput): SlackIdleIncomingProjection | undefined { + try { + const context = slackIncomingContext(input); + if (!context) return; + const classify = createSlackFormHistoryClassifier({ requirements: context.requirements, verified: input.capabilities, expected: context.expected, currentOwners: context.currentOwners }); + const groups = groupHistory(input.actions), completed = new Set(); + for (const { endpoint } of input.endpoints) { + input.check?.(); + for (const id of classify(groups.get(`${endpoint.companyId}:${endpoint.id}`) ?? [], endpoint.companyId, endpoint.id, input.check)) { + if (!uuid.test(id)) return; completed.add(id); + } + } + return projection(context, completed); + } catch { return; } +} +const uuid = /^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$/i; +const pageSize = 1000; +const historyOpen = alias(chatActions, "slack_idle_history_open"), historySubmit = alias(chatActions, "slack_idle_history_submit"); +const relatedId = (key: "openActionRowId" | "submitActionRowId") => sql`CASE + WHEN ${chatActions.payload}->>${key} ~* '^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$' + THEN (${chatActions.payload}->>${key})::uuid ELSE NULL END`; + +export async function readSlackIdleIncomingProof(tx: Pick, input: { + capabilities?: VerifiedIdleCapabilities; identity?: SlackFormConnectorIdentity; receiverBound: boolean; + beforeRead(): Promise; remaining(): number; registeredOptions(endpointId: string): boolean; +}): Promise { + if (!input.capabilities?.incomingWake?.routes.includes("slack-broker-events-v1") || !input.identity || !input.receiverBound) return; + const broker = input.capabilities.incomingWake.slackBroker; + if (!broker || broker.instanceId !== input.identity.instanceId || broker.generation !== input.capabilities.incomingWake.generation) return; + await input.beforeRead(); + const endpoints = await tx.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints).innerJoin(toolConnections, + and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, chatEndpoints.companyId))) + .where(and(eq(chatEndpoints.provider, "slack"), eq(toolConnections.transport, "chat_sdk"), eq(toolConnections.connectionPurpose, "channel"), + sql`exists (select 1 from companies where id=${chatEndpoints.companyId} and status <> 'archived')`)).orderBy(chatEndpoints.id).limit(1001); + input.remaining(); if (endpoints.length > 1000) return; + if (!endpoints.length && broker.entries.length) return; + if (!endpoints.length) return Object.freeze({ endpointIds: [], connectionIds: [], completedActionIds: [] }); + // Retain only bounded current-owner candidates. Old enrolled generations do + // not become current coverage, and duplicate exact owners on ANY page fail. + const endpointsById = new Map(endpoints.map(value => [value.endpoint.id, value])); + const currentOwners = new Map(); + let ownerCursor: string | undefined; + for (;;) { + await input.beforeRead(); + const page = await tx.select({ action: chatActions }).from(chatActions).innerJoin(chatEndpoints, + and(eq(chatEndpoints.id, chatActions.endpointId), eq(chatEndpoints.companyId, chatActions.companyId))) + .where(and(eq(chatEndpoints.provider, "slack"), inArray(chatActions.endpointId, endpoints.map(value => value.endpoint.id)), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.result}->>'phase'='enrolled'`, + ownerCursor ? gt(chatActions.id, ownerCursor) : undefined)) + .orderBy(chatActions.id).limit(pageSize); + input.remaining(); + for (const { action } of page) { + input.remaining(); + const scope = endpointsById.get(action.endpointId); + if (!scope || action.companyId !== scope.endpoint.companyId || scope.endpoint.status !== "active" + || scope.connection.status !== "active" || !scope.connection.enabled) continue; + try { + const setup = slackFormSetupRecord(scope.endpoint.setup), binding = parseSlackFormBinding(setup.slackCloudForms); + if (!/^slack-form-owner:[a-f0-9]{64}$/.test(action.providerActionId) || !["issued", "processed"].includes(action.status) + || action.result?.nextAttemptAt !== null || action.result.desiredActive === false || action.payload.environment !== input.identity.environment + || action.payload.subject !== setup.slackCloudSubject || !sameSlackFormBinding(parseSlackFormBinding(action.payload.binding), binding)) continue; + const enrollment = parseSlackFormPublicEnrollment(action.payload.enrollment); + const { credentialFingerprint: _, eventBindingId, ...base } = binding; + if (!isDeepStrictEqual({ ...base, bindingId: eventBindingId }, enrollment)) continue; + const owners = currentOwners.get(action.endpointId) ?? []; + owners.push(action); if (owners.length > 1) return; + currentOwners.set(action.endpointId, owners); + } catch { continue; } + } + if (page.length < pageSize) break; + ownerCursor = page.at(-1)!.action.id; + } + const common = { endpoints, capabilities: input.capabilities, identity: input.identity, + registeredOptions: input.registeredOptions, check: () => { input.remaining(); } }; + const context = slackIncomingContext({ ...common, actions: [...currentOwners.values()].flat() }); + if (!context) return; + const classify = createSlackFormHistoryClassifier({ requirements: context.requirements, verified: input.capabilities, expected: context.expected, currentOwners: context.currentOwners }); + const completed = new Set(); let cursor: string | undefined; + for (;;) { + await input.beforeRead(); + // Indexed primary-key joins supply only the two referenced actions. A + // session/page boundary never requires an all-history JSON accumulator or + // one extra read per historical session. + const page = await tx.select({ action: chatActions, open: historyOpen, submit: historySubmit }).from(chatActions) + .innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatActions.endpointId), eq(chatEndpoints.companyId, chatActions.companyId))) + .leftJoin(historyOpen, and(eq(chatActions.kind, "slack_form_session"), eq(historyOpen.kind, "question_form_open"), eq(historyOpen.id, relatedId("openActionRowId")), eq(historyOpen.companyId, chatActions.companyId), eq(historyOpen.endpointId, chatActions.endpointId))) + .leftJoin(historySubmit, and(eq(chatActions.kind, "slack_form_session"), eq(historySubmit.kind, "question_form_submit"), eq(historySubmit.id, relatedId("submitActionRowId")), eq(historySubmit.companyId, chatActions.companyId), eq(historySubmit.endpointId, chatActions.endpointId))) + .where(and(eq(chatEndpoints.provider, "slack"), inArray(chatActions.endpointId, endpoints.map(value => value.endpoint.id)), + inArray(chatActions.kind, ["slack_form_owner", "slack_form_grant", "slack_form_session", "question_form_open", "question_form_submit"]), + cursor ? gt(chatActions.id, cursor) : undefined)).orderBy(chatActions.id).limit(pageSize); + input.remaining(); + const groups = groupHistory(page.flatMap(row => [row.action, ...(row.open ? [row.open] : []), ...(row.submit ? [row.submit] : [])])); + for (const rows of groups.values()) { + input.remaining(); const first = rows[0]!; + for (const id of classify(rows, first.companyId, first.endpointId, () => { input.remaining(); })) { + if (!uuid.test(id)) return; completed.add(id); + } + } + if (page.length < pageSize) break; + cursor = page.at(-1)!.action.id; + } + input.remaining(); + return projection(context, completed); +} diff --git a/server/src/services/chat-channels/slack/receipt-authority.ts b/server/src/services/chat-channels/slack/receipt-authority.ts new file mode 100644 index 00000000000..52e1c4db625 --- /dev/null +++ b/server/src/services/chat-channels/slack/receipt-authority.ts @@ -0,0 +1,11 @@ +import { verifyCloudConnectorReceipt } from "../../cloud-runtime-identity.js"; +import { parseSlackFormFinalizedReceipt, type SlackFormFinalizedReceipt } from "./ack-receipt.js"; +import { parseSlackFormRetirementReceipt, type SlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; + +type ReceiptVerificationInput = { compactJws: string; env?: NodeJS.ProcessEnv }; +export function verifySlackFormFinalizedReceipt(input: ReceiptVerificationInput): SlackFormFinalizedReceipt { + return verifyCloudConnectorReceipt(input, parseSlackFormFinalizedReceipt); +} +export function verifySlackFormRetirementReceipt(input: ReceiptVerificationInput): SlackFormRetirementReceipt { + return verifyCloudConnectorReceipt(input, parseSlackFormRetirementReceipt); +} diff --git a/server/src/services/chat-publication-errors.test.ts b/server/src/services/chat-publication-errors.test.ts index a2337931264..56abb236e21 100644 --- a/server/src/services/chat-publication-errors.test.ts +++ b/server/src/services/chat-publication-errors.test.ts @@ -475,4 +475,17 @@ describe("chat publication error classification", () => { }); }, ); + it("retries an unknown preparation failure only with a local before-invocation fact", () => { + const error = new Error("form_publication_enrollment_pending"); + expect(classifyChatPublicationError(error, 1, "not_started")).toEqual({ + kind: "retry", retryAfterMs: 2000, reason: error.message, + }); + expect(classifyChatPublicationError(error, 1)).toEqual({ kind: "delivery_unknown", reason: error.message }); + expect(classifyChatPublicationError(error, 1, "possible")).toEqual({ kind: "delivery_unknown", reason: error.message }); + }); + it("preserves definite provider dispositions with before-invocation context", () => { + expect(classifyChatPublicationError(Object.assign(new Error("rejected"), { status: 400 }), 1, "not_started")).toMatchObject({ kind: "failed" }); + expect(classifyChatPublicationError(Object.assign(new Error("slow down"), { status: 429 }), 1, "not_started")).toMatchObject({ kind: "retry", providerRateLimit: true }); + }); + }); diff --git a/server/src/services/chat-publication-errors.ts b/server/src/services/chat-publication-errors.ts index 24847917fa0..04f4c1c23d8 100644 --- a/server/src/services/chat-publication-errors.ts +++ b/server/src/services/chat-publication-errors.ts @@ -145,6 +145,7 @@ function structuredProviderErrors(error: unknown): StructuredProviderError[] { export function classifyChatPublicationError( error: unknown, attempt: number, + providerInvocation: "possible" | "not_started" = "possible", ): ChatPublicationErrorDisposition { const values = structuredProviderErrors(error); const firstNumber = (select: (value: StructuredProviderError) => unknown) => @@ -399,5 +400,8 @@ export function classifyChatPublicationError( // Adapter NetworkError and ordinary fetch/transport errors are ambiguous: // the request may have reached the provider even when no response reached // Paperclip. An operator must inspect the native conversation before replay. + if (providerInvocation === "not_started") { + return { kind: "retry", retryAfterMs: Math.min(60_000, 2 ** Math.max(0, attempt) * 1_000), reason }; + } return { kind: "delivery_unknown", reason }; } diff --git a/server/src/services/cloud-runtime-identity.ts b/server/src/services/cloud-runtime-identity.ts index b1f37dbea87..8d2280a599a 100644 --- a/server/src/services/cloud-runtime-identity.ts +++ b/server/src/services/cloud-runtime-identity.ts @@ -1,5 +1,6 @@ +import { DEFAULT_JSON_BODY_LIMIT_BYTES } from "../http/body-limits.js"; import { parseIdleCapabilities, type IdleCapabilities } from "./idle-incoming-wake-capability.js"; -import { createPublicKey, timingSafeEqual, verify, type JsonWebKey } from "node:crypto"; +import { createHash, createPublicKey, timingSafeEqual, verify, type JsonWebKey } from "node:crypto"; import { eq } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; import { instanceSettings } from "@paperclipai/db"; @@ -449,6 +450,7 @@ export const CLOUD_CONTROL_JWS_TYPE = "paperclip-cloud-control+jwt"; export const CLOUD_CONTROL_ACTIONS = [ "ephemeral-enrollment:prepare", "task-drain:read", + "task-drain:read-body", "task-drain:start", "task-drain:stop", "lifecycle:read", @@ -496,6 +498,7 @@ export type CloudControlClaims = { action: CloudControlAction; requestId: string; idleCapabilities?: IdleCapabilities; + bodySha256?: string; iat: number; exp: number; }; @@ -509,6 +512,7 @@ export type CloudControlClaims = { */ export function verifyCloudControlAssertion(input: { compactJws: string; + rawBody?: Uint8Array; expectedAction: CloudControlAction; env?: NodeJS.ProcessEnv; now?: Date; @@ -577,9 +581,27 @@ export function verifyCloudControlAssertion(input: { ) { throw new Error("Cloud control assertion is expired or has an invalid lifetime"); } - if (payload.idleCapabilities !== undefined) { - if (payload.action !== "task-drain:read") throw new Error("Idle capabilities require task-drain:read"); - payload.idleCapabilities = parseIdleCapabilities(payload.idleCapabilities); + if (payload.action === "task-drain:read-body") { + if (payload.idleCapabilities !== undefined || typeof payload.bodySha256 !== "string" || !/^[a-f0-9]{64}$/.test(payload.bodySha256) + || !input.rawBody || input.rawBody.byteLength === 0 || input.rawBody.byteLength > DEFAULT_JSON_BODY_LIMIT_BYTES) { + throw new Error("Invalid Cloud idle status body binding"); + } + const raw = Buffer.from(input.rawBody); + if (createHash("sha256").update(raw).digest("hex") !== payload.bodySha256) throw new Error("Cloud idle status body digest changed"); + const text = new TextDecoder("utf-8", { fatal: true }).decode(raw); + let body: unknown; + try { body = JSON.parse(text); } + catch { throw new Error("Invalid Cloud idle status body JSON"); } + if (!body || typeof body !== "object" || Array.isArray(body) || Object.keys(body).join() !== "idleCapabilities") { + throw new Error("Invalid Cloud idle status body"); + } + payload.idleCapabilities = parseIdleCapabilities(Reflect.get(body, "idleCapabilities")); + } else { + if (payload.bodySha256 !== undefined) throw new Error("Cloud body binding requires task-drain:read-body"); + if (payload.idleCapabilities !== undefined) { + if (payload.action !== "task-drain:read") throw new Error("Idle capabilities require task-drain:read"); + payload.idleCapabilities = parseIdleCapabilities(payload.idleCapabilities); + } } // Consumed LAST, only after every other check passed: a rejected // assertion must not burn its request id, or an attacker could deny a @@ -589,3 +611,24 @@ export function verifyCloudControlAssertion(input: { } return payload as CloudControlClaims; } + +export function verifyCloudConnectorReceipt(input: { compactJws: string; env?: NodeJS.ProcessEnv }, parseReceipt: (value: unknown) => T): T { + const env = input.env ?? process.env; + if (Buffer.byteLength(input.compactJws, "ascii") > 12_000 || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(input.compactJws)) + throw new Error("Invalid connector ACK receipt encoding"); + const [headerPart, payloadPart, signaturePart] = input.compactJws.split("."); + const header = decodeJsonPart(headerPart, "connector receipt header"); + if (Object.keys(header).sort().join() !== "alg,kid,typ" || header.alg !== "EdDSA" + || header.typ !== "paperclip-connector-ack-receipt+jwt" || typeof header.kid !== "string" || !header.kid) + throw new Error("Invalid connector ACK receipt header"); + const key = cloudRuntimePublicKeyForKid(env, header.kid); + if (!verify(null, Buffer.from(`${headerPart}.${payloadPart}`, "ascii"), key, Buffer.from(signaturePart, "base64url"))) + throw new Error("Invalid connector ACK receipt signature"); + const claims = decodeJsonPart(payloadPart, "connector receipt claims"); + if (Object.keys(claims).sort().join() !== "aud,iss,receipt,sub,v" || claims.v !== 1 || claims.iss !== CLOUD_RUNTIME_IDENTITY_ISSUER + || claims.aud !== "paperclip-connector-ack-receipt/v1" || claims.sub !== nonEmpty(env.PAPERCLIP_CLOUD_STACK_ID)) + throw new Error("Invalid connector ACK receipt claims"); + const receipt = parseReceipt(claims.receipt); + if (receipt.binding.instanceId !== claims.sub) throw new Error("Connector ACK receipt instance changed"); + return receipt; +} diff --git a/server/src/services/github-connection-events.ts b/server/src/services/github-connection-events.ts index e92c27d9953..e095c52c1a0 100644 --- a/server/src/services/github-connection-events.ts +++ b/server/src/services/github-connection-events.ts @@ -1,3 +1,7 @@ +import { parseSlackFormOperation } from "./chat-channels/slack/cloud-forms.js"; +import { acknowledgeSlackFormDeliveries } from "./chat-channels/slack/form-normal-ack.js"; +import { readSlackFormRetainedConsumer } from "./chat-channels/slack/cloud-consumer.js"; +import { reconcileRegisteredSlackFormWork } from "./chat-channels/slack/form-owner.js"; import { SlackCloudReceiptError } from "./chat-channels/slack/cloud-event.js"; import { reconcilePendingSlackCloudEnrollments } from "./chat-channels/slack/cloud-enrollment.js"; import { notifyExternalObjectRefresh } from "./external-object-refresh-notifications.js"; @@ -522,6 +526,11 @@ export function githubConnectionEventService( logger.warn({ reason: "enrollment_reconcile_unavailable" }, "Slack enrollment responsibility retained"); } } + let formsAt: number | null = null; + try { formsAt = await reconcileRegisteredSlackFormWork(db, now()); } + catch { formsAt = now().getTime() + 5_000; } + const next = Math.min(nextEnrollmentAt ?? Number.POSITIVE_INFINITY, formsAt ?? Number.POSITIVE_INFINITY); + nextEnrollmentAt = Number.isFinite(next) ? next : null; } } @@ -531,7 +540,25 @@ export function githubConnectionEventService( const slackBots = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.provider, "slack"), sql`${chatEndpoints.setup}->'slackCloudEvents' is not null`, sql`${chatEndpoints.status} in ('active','verifying','attention')`, sql`exists (select 1 from companies where id = ${chatEndpoints.companyId} and status <> 'archived')`)); const bots = [...githubBots, ...slackBots]; let identity: { subject: string; companyId: string } | null = bindings[0] ?? (bots[0] ? { subject: "github-bot-events", companyId: bots[0].companyId } : null); - const hasConsumer = bindings.length > 0 || bots.length > 0; + let hasConsumer = bindings.length > 0 || bots.length > 0; + if (!identity) { + try { + const retained = await readSlackFormRetainedConsumer(db, connector); + identity = retained.owner; + hasConsumer ||= retained.owner !== null; + const time = now().getTime(); + if (retained.attention.length && time >= nextEnrollmentDiagnosticAt) { + nextEnrollmentDiagnosticAt = time + 300_000; + logger.warn({ reason: "form_retained_owner_invalid", count: retained.attention.length }, "Slack form custody requires attention"); + } + } catch { + const time = now().getTime(); + if (time >= nextEnrollmentDiagnosticAt) { + nextEnrollmentDiagnosticAt = time + 300_000; + logger.warn({ reason: "form_retained_owner_unavailable" }, "Slack form custody requires attention"); + } + } + } if (!identity) { const [history] = await db.select({ grant: connectionGrants, connection: toolConnections }).from(connectionGrants) .innerJoin(toolConnections, and(eq(toolConnections.id, connectionGrants.connectionId), eq(toolConnections.companyId, connectionGrants.companyId))) @@ -567,9 +594,18 @@ export function githubConnectionEventService( failed: lease.failedEventIds?.length ?? 0, }; const acknowledge: string[] = []; + const formCompanies = new Map(); for (const event of lease.events) { if (event.provider === "slack") { - try { await dispatchSlackCloudEvent(db, event); acknowledge.push(event.id); result.processed += 1; } + try { + await dispatchSlackCloudEvent(db, event); + if (/^slack_form_[a-f0-9]{64}$/.test(event.id)) { + const packet = event.payload.slackFormOpen; + if (!packet || typeof packet !== "object" || Array.isArray(packet)) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + formCompanies.set(event.id, parseSlackFormOperation(Reflect.get(packet, "operation")).binding.companyId); + } + acknowledge.push(event.id); result.processed += 1; + } catch (error) { const code = error instanceof SlackCloudReceiptError ? error.code : "owner_or_admission_unavailable"; const disposition = error instanceof SlackCloudReceiptError ? error.disposition : "retry"; @@ -610,12 +646,8 @@ export function githubConnectionEventService( } } if (acknowledge.length > 0) { - await connector.acknowledgeEvents({ - subject: first.subject, - companyId: first.companyId, - leaseId: lease.leaseId, - deliveryIds: acknowledge, - }); + await acknowledgeSlackFormDeliveries(db, { connector, + values: { subject: first.subject, companyId: first.companyId, leaseId: lease.leaseId, deliveryIds: acknowledge }, companies: formCompanies }); } for (const deliveryId of lease.failedEventIds ?? []) { logger.warn({ deliveryId, reason: "inner_event_authentication_failed" }, diff --git a/server/src/services/idle-incoming-wake-capability.ts b/server/src/services/idle-incoming-wake-capability.ts index a0f26fff61c..00c086adf36 100644 --- a/server/src/services/idle-incoming-wake-capability.ts +++ b/server/src/services/idle-incoming-wake-capability.ts @@ -1,12 +1,14 @@ +import { parseSlackConfiguredCoverage, type SlackConfiguredCoverage } from "./chat-channels/slack/configured-coverage.js"; /** Closed Cloud-owned ingress catalog. No credential material belongs in this proof. */ export interface IdleIncomingWakeCapability { version: 1; ownerId: string; generation: string; - routes: ("tenant-api-http-v2" | "github-broker-events-v1" | "mcp-oauth-http-v1")[]; + routes: ("tenant-api-http-v2" | "github-broker-events-v1" | "slack-broker-events-v1" | "mcp-oauth-http-v1")[]; apiKeyProof?: { version: 1; challenge: string }; mcpOAuthProof?: { version: 1; challenge: string }; githubBroker?: { instanceId: string; bindingsFingerprint: string }; + slackBroker?: SlackConfiguredCoverage; } export interface IdleCapabilities { incomingWake?: IdleIncomingWakeCapability; @@ -40,6 +42,7 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { "generation", "routes", "githubBroker", + "slackBroker", "apiKeyProof", "mcpOAuthProof", ]) || @@ -51,9 +54,9 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { !digest(v.generation) || !Array.isArray(v.routes) || v.routes.length < 1 || - v.routes.length > 3 || + v.routes.length > 4 || v.routes.some( - (r) => !["tenant-api-http-v2", "github-broker-events-v1", "mcp-oauth-http-v1"].includes(r), + (r) => !["tenant-api-http-v2", "github-broker-events-v1", "slack-broker-events-v1", "mcp-oauth-http-v1"].includes(r), ) || new Set(v.routes).size !== v.routes.length || [...v.routes].sort().join() !== v.routes.join() @@ -82,10 +85,18 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { } else if (v.githubBroker !== undefined) throw new Error("Unexpected GitHub wake binding"); } + if (value.incomingWake && typeof value.incomingWake === "object" && !Array.isArray(value.incomingWake)) { + const incoming = Object.fromEntries(Object.entries(value.incomingWake)); + if (Array.isArray(incoming.routes) && incoming.routes.includes("slack-broker-events-v1")) { + const proof = parseSlackConfiguredCoverage(incoming.slackBroker); + if (proof.generation !== incoming.generation) throw new Error("Slack capability generation changed"); + } else if (incoming.slackBroker !== undefined) throw new Error("Unexpected Slack wake binding"); + } // Return a detached immutable snapshot; request callers cannot mutate a verified claim. const parsed = JSON.parse(JSON.stringify(value)) as IdleCapabilities; if (parsed.incomingWake) { Object.freeze(parsed.incomingWake.routes); + if (parsed.incomingWake.slackBroker) parsed.incomingWake.slackBroker = parseSlackConfiguredCoverage(parsed.incomingWake.slackBroker); if (parsed.incomingWake.apiKeyProof) Object.freeze(parsed.incomingWake.apiKeyProof); if (parsed.incomingWake.mcpOAuthProof) Object.freeze(parsed.incomingWake.mcpOAuthProof); if (parsed.incomingWake.githubBroker) diff --git a/server/src/services/idle-incoming-wake.ts b/server/src/services/idle-incoming-wake.ts index f611552c93c..977179ecb05 100644 --- a/server/src/services/idle-incoming-wake.ts +++ b/server/src/services/idle-incoming-wake.ts @@ -1,3 +1,5 @@ +import { idleChatHistoryChecksForOwner } from "./chat-channels/idle-history-proof.js"; +import { readSlackIdleIncomingProof, slackHasRegisteredOptionsAtCurrentBuild } from "./chat-channels/slack/idle-incoming.js"; import { readIdleMcpSleepProof, type IdleMcpSleepProof } from "./public-mcp/idle-sleep-proof.js"; import { idleMcpWorkQuery } from "./public-mcp/idle-work-query.js"; import { idleExternalAgentIncomingWorkQuery } from "../modules/external-agents/index.js"; @@ -22,6 +24,7 @@ import { hasConnectionEventWakeReceiver } from "./connection-event-wake.js"; export interface IdleIncomingWakeProof { capability: IdleIncomingWakeCapability; apiWakeAdmission?: ApiWakeAdmission; + completedActionIds?: readonly string[]; mcp?: IdleMcpSleepProof; connectionIds: string[]; endpointIds: string[]; @@ -102,6 +105,18 @@ export async function readIdleIncomingWakeProof( expiresAt: row.expiresAt === null ? null : new Date(row.expiresAt).toISOString() })), }); } + if (capability.routes.includes("slack-broker-events-v1")) { + const config = paperclipCloudConnectorConfigFromEnv(); + const slack = await readSlackIdleIncomingProof(tx, { capabilities: input.capabilities, + identity: config ? { instanceId: config.instanceId, environment: config.environment } : undefined, + receiverBound: hasConnectionEventWakeReceiver(input.receiverOwner), beforeRead, remaining, + registeredOptions: slackHasRegisteredOptionsAtCurrentBuild }); + if (slack) { + proof.endpointIds.push(...slack.endpointIds); + proof.connectionIds.push(...slack.connectionIds); + if (slack.completedActionIds.length) proof.completedActionIds = slack.completedActionIds; + } + } if (capability.routes.includes("mcp-oauth-http-v1")) { const stackId = process.env.PAPERCLIP_CLOUD_STACK_ID; if (!stackId || !capability.mcpOAuthProof) throw new Error("MCP wake challenge is missing"); @@ -306,6 +321,11 @@ export function incomingWakeWorkQuery( continuityConnectionIds.includes(id), ) ?? []), ]; + if ((reason === "chat_actions" || reason === "chat_deliveries") && proof?.completedActionIds?.length) { + const check = idleChatHistoryChecksForOwner(proof.completedActionIds).find(value => value.reason === reason); + if (!check) throw new Error("Chat history owner query missing"); + return typeof check.query === "string" ? sql.raw(check.query) : check.query; + } if (reason === "agent_api_keys") return proof?.apiWakeAdmission !== undefined ? sql`SELECT 1 WHERE false` diff --git a/server/src/services/paperclip-cloud-connector.test.ts b/server/src/services/paperclip-cloud-connector.test.ts index 4d7c594153b..e8e3767cd1f 100644 --- a/server/src/services/paperclip-cloud-connector.test.ts +++ b/server/src/services/paperclip-cloud-connector.test.ts @@ -5,6 +5,7 @@ import { generateKeyPairSync, hkdfSync, randomBytes, + sign, type KeyObject, } from "node:crypto"; import { SLACK_MANAGER_SCOPES } from "@paperclipai/shared"; @@ -22,6 +23,9 @@ import { type PaperclipCloudConnectorConfig, } from "./paperclip-cloud-connector.js"; +import { parseSlackFormFinalizedReceipt } from "./chat-channels/slack/ack-receipt.js"; +import { verifySlackFormFinalizedReceipt } from "./chat-channels/slack/receipt-authority.js"; + const instanceId = "inst_test"; const companyId = "company_test"; const subject = "user_test"; @@ -860,3 +864,56 @@ describe("sealed Slack event connector transport", () => { expect(JSON.parse(String(init.body))).toMatchObject({ enrollment: JSON.stringify(enrollment) }); }); }); + + +describe("Slack form normal connector terminal reply contracts", () => { + it("returns current signed refresh and GC-only replies through the actual signed connector", async () => { + const keys = config(), pair = generateKeyPairSync("ed25519"), kid = "connector-terminal-reply"; + const canonical = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: keys.config.environment, + binding: { version: 1, instanceId, companyId, endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "b".repeat(64), credentialFingerprint: "c".repeat(64) }, + operationId: "a".repeat(64), deliveryId: "slack_form_" + "a".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-07T00:00:00.000Z" }); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const unsigned = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: instanceId, receipt: canonical }); + const compactJws = unsigned + "." + sign(null, Buffer.from(unsigned), pair.privateKey).toString("base64url"); + const operations: Record[] = []; + let reply: unknown = { ackReceiptJws: compactJws }; + const request = vi.fn(async (_url, init) => { + const body = JSON.parse(String(init?.body)); + const parts = String(body.request).split("."); expect(parts).toHaveLength(3); + const claims = JSON.parse(Buffer.from(parts[1]!, "base64url").toString("utf8")); + expect(claims.sub).toBe(subject); expect(claims.cid).toBe(companyId); + expect(claims.sh).toBe(createHash("sha256").update(body.operation).digest("base64url")); + operations.push(JSON.parse(body.operation)); + return Response.json({ slackForm: reply }); + }); + const connector = createPaperclipCloudConnector({ config: keys.config, request }); + const refresh = { kind: "receipt", binding: { instanceId }, operationId: canonical.operationId, refreshAckReceipt: true }; + await expect(connector.slackFormOperation({ subject, companyId, operation: refresh })).resolves.toEqual({ ackReceiptJws: compactJws }); + const digest = "d".repeat(64); + reply = { releasedAckDigest: digest }; + const release = { kind: "receipt", binding: { instanceId }, operationId: canonical.operationId, releaseAckFact: true }; + await expect(connector.slackFormOperation({ subject, companyId, operation: release })).resolves.toEqual({ releasedAckDigest: digest }); + expect(operations).toEqual([refresh, release]); expect(request).toHaveBeenCalledTimes(2); + expect(verifySlackFormFinalizedReceipt({ compactJws, env: { PAPERCLIP_CLOUD_STACK_ID: instanceId, + PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] }) } })).toEqual(canonical); + }); + it("does not confuse refresh, release, acceptance or sealed operation responses", async () => { + const keys = config(); let reply: unknown; + const connector = createPaperclipCloudConnector({ config: keys.config, request: vi.fn(async () => Response.json({ slackForm: reply })) }); + const call = (operation: Record) => connector.slackFormOperation({ subject, companyId, operation: { kind: "receipt", ...operation } }); + for (const invalid of [{ releasedAckDigest: "d".repeat(64) }, { ackReceiptJws: "missing.signature" }, + { ackReceiptJws: "a.b.c", extra: true }, { acceptedReceiptDigest: "d".repeat(64), needsTerminalOriginal: false }, { sealed: {} }]) { + reply = invalid; await expect(call({ refreshAckReceipt: true })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + } + for (const invalid of [{ ackReceiptJws: "a.b.c" }, { releasedAckDigest: "invalid" }, { releasedAckDigest: "d".repeat(64), extra: true }]) { + reply = invalid; await expect(call({ releaseAckFact: true })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + } + reply = { releasedAckDigest: "d".repeat(64) }; + await expect(call({ acceptedAckReceipt: "a.b.c" })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + await expect(call({})).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + }); +}); diff --git a/server/src/services/paperclip-cloud-connector.ts b/server/src/services/paperclip-cloud-connector.ts index 368d8708f8a..b1f9f087133 100644 --- a/server/src/services/paperclip-cloud-connector.ts +++ b/server/src/services/paperclip-cloud-connector.ts @@ -36,7 +36,7 @@ export const GMAIL_CONNECTOR_SCOPES = [ export { GOOGLE_WORKSPACE_CONNECTOR_PROFILES }; export type PaperclipCloudConnectorEnvironment = "development" | "test" | "staging" | "production"; -export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events" | "mcp-client-mirror" | "mcp-revocation-index"; +export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events" | "slack-forms" | "mcp-client-mirror" | "mcp-revocation-index"; export type PaperclipCloudConnectorProfileId = GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId | "slack.manager" | "github.bot"; export type PaperclipCloudConnectorProvider = "google" | "github" | "asana" | "slack"; @@ -87,7 +87,7 @@ export type SealedConnectorEvents = { }>; }; -type SealedEnvelopeProfileId = PaperclipCloudConnectorProfileId | "slack.events.v1"; +type SealedEnvelopeProfileId = PaperclipCloudConnectorProfileId | "slack.events.v1" | "slack.form-open.v1" | "slack.callbacks.v1"; type SealedEnvelope = { v: 1; @@ -117,8 +117,10 @@ type ConnectorResponse = { leaseId?: unknown; events?: unknown; acknowledged?: unknown; + formReceipts?: unknown; githubApps?: { version: number }; slackEvents?: { version: number }; + slackForm?: unknown; enrollment?: unknown; retirement?: unknown; eventWakeVersion?: unknown; @@ -128,6 +130,7 @@ const ENDPOINTS: Record = { "mcp-client-mirror": "/v1/connector/mcp-client-mirror", "mcp-revocation-index": "/v1/connector/mcp-revocation-index", "slack-events": "/v1/connector/slack-events", + "slack-forms": "/v1/connector/slack-forms", "github-app": "/v1/connector/github-apps", status: "/v1/connector/instance-status", session: "/v1/connector/sessions", @@ -159,6 +162,8 @@ const BROKER_REJECTION_REASONS = new Set([ "SLACK_ENROLLMENT_GENERATION_CHANGED", "SLACK_ENROLLMENT_IDENTITY_CHANGED", "SLACK_ENROLLMENT_CHANGED", "SLACK_ENROLLMENT_UNSETTLED", "SLACK_ENROLLMENT_DISABLED", "SLACK_COMPANY_MISMATCH", "SLACK_EVENTS_UNAVAILABLE", + "SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", + "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_RETIREMENT_PROOF_MISSING", "SLACK_FORM_CUSTODY_CHANGED", "SLACK_FORM_RETIREMENT_SIGNER_UNAVAILABLE", "SLACK_FORM_CANONICAL_FACT_CHANGED", "SLACK_FORM_CANONICAL_FACT_BUDGET_CHANGED", "SLACK_FORM_CANONICAL_FACT_PENDING", ]); const UNKNOWN_BROKER_REASON = "UNKNOWN_BROKER_ERROR"; @@ -293,7 +298,7 @@ export function createPaperclipCloudConnector(input: { async function call( operation: PaperclipCloudConnectorOperation, claims: { subject: string; companyId: string; profile?: PaperclipCloudConnectorProfileId; scopes?: string[]; returnUri?: string; returnState?: string; claimId?: string; redemptionId?: string; eventWakeVersion?: 0 | 1 }, - secret?: { field: "refreshToken" | "token" | "binding" | "acknowledgement" | "enrollment"; value: string }, + secret?: { field: "refreshToken" | "token" | "binding" | "acknowledgement" | "enrollment" | "operation"; value: string }, ): Promise { const endpoint = new URL(ENDPOINTS[operation], `${config.baseUrl}/`).toString(); const issuedAt = Math.floor(now() / 1000); @@ -398,6 +403,7 @@ export function createPaperclipCloudConnector(input: { } return { + instanceIdentity: () => Object.freeze({ instanceId: config.instanceId, environment: config.environment }), async syncMcpClientMirror(serialized: string): Promise { const response = await call("mcp-client-mirror", {subject:"instance-client-metadata",companyId:"instance-client-metadata"}, {field:"binding",value:serialized}); return (response as ConnectorResponse & {mcpClientMirror?:unknown}).mcpClientMirror; @@ -411,6 +417,50 @@ export function createPaperclipCloudConnector(input: { const response = await call("status", { subject: "instance-capabilities", companyId: "instance-capabilities" }); return response.active === true && response.slackEvents?.version === 1; }, + async slackFormOperation(values: { subject: string; companyId: string; operation: Record }) { + const response = await call("slack-forms", values, { field: "operation", value: JSON.stringify(values.operation) }); + if (!isRecord(response.slackForm)) throw badEnvelope(); + if (values.operation.kind === "receipt" && values.operation.refreshAckReceipt !== undefined) { + if (values.operation.acceptedAckReceipt !== undefined || values.operation.releaseAckFact !== undefined + || Object.keys(response.slackForm).join() !== "ackReceiptJws" || typeof response.slackForm.ackReceiptJws !== "string" + || Buffer.byteLength(response.slackForm.ackReceiptJws, "ascii") > 12000 + || !/^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$/.test(response.slackForm.ackReceiptJws)) throw badEnvelope(); + return response.slackForm; + } + if (values.operation.kind === "receipt" && values.operation.releaseAckFact !== undefined) { + if (values.operation.acceptedAckReceipt !== undefined || Object.keys(response.slackForm).join() !== "releasedAckDigest" + || typeof response.slackForm.releasedAckDigest !== "string" || !/^[a-f0-9]{64}$/.test(response.slackForm.releasedAckDigest)) throw badEnvelope(); + return response.slackForm; + } + if (values.operation.kind === "receipt" && values.operation.acceptedAckReceipt !== undefined) { + if (Object.keys(response.slackForm).sort().join() !== "acceptedReceiptDigest,needsTerminalOriginal" || typeof response.slackForm.acceptedReceiptDigest !== "string" + || !/^[a-f0-9]{64}$/.test(response.slackForm.acceptedReceiptDigest) || typeof response.slackForm.needsTerminalOriginal !== "boolean") throw badEnvelope(); + return response.slackForm; + } + if (!["receipt", "terminal-ack"].includes(String(values.operation.kind))) return response.slackForm; + const envelope = parseEnvelope(response.slackForm.sealed, "events", "slack", "slack.form-open.v1"); + const receipt = decryptEnvelope(envelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(receipt) || receipt.version !== 1 || receipt.instanceId !== config.instanceId || receipt.environment !== config.environment) throw badEnvelope(); + if (!isRecord(receipt.operation)) throw badEnvelope(); + if (receipt.operation.closed !== null) { + if (!isRecord(receipt.operation.closed)) throw badEnvelope(); + const closeEnvelope = parseEnvelope(receipt.operation.closed.evidence, "events", "slack", "slack.form-open.v1"); + const evidence = decryptEnvelope(closeEnvelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(evidence) || evidence.version !== 1 || !["close", "submission"].includes(String(evidence.kind)) || evidence.instanceId !== config.instanceId + || evidence.environment !== config.environment || evidence.operationId !== receipt.operation.id) throw badEnvelope(); + receipt.operation.closed = { ...receipt.operation.closed, evidence }; + } + if (receipt.operation.rejection !== null) { + if (!isRecord(receipt.operation.rejection)) throw badEnvelope(); + const rejectionEnvelope = parseEnvelope(receipt.operation.rejection.evidence, "events", "slack", "slack.form-open.v1"); + const evidence = decryptEnvelope(rejectionEnvelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(evidence) || evidence.version !== 1 || evidence.kind !== "rejection" || evidence.instanceId !== config.instanceId + || evidence.environment !== config.environment || evidence.operationId !== receipt.operation.id + || evidence.responseDigest !== receipt.operation.rejection.responseDigest) throw badEnvelope(); + receipt.operation.rejection = { ...receipt.operation.rejection, evidence }; + } + return receipt; + }, async slackEventEnrollment(values: { subject: string; companyId: string; enrollment: Record }) { const response = await call("slack-events", values, { field: "enrollment", value: JSON.stringify(values.enrollment) }); return values.enrollment.active === false ? response.retirement : response.enrollment; @@ -586,6 +636,24 @@ export function createPaperclipCloudConnector(input: { event.payload = { githubApp: delivery }; } if (event.provider === "slack") { + if (event.payload.slackCallback !== undefined) { + const packet = event.payload.slackCallback; + if (!isRecord(packet) || typeof packet.bindingId !== "string" || packet.sealed === undefined) throw badEnvelope(); + const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.callbacks.v1"); + const delivery = decryptEnvelope(sealed, sealKey, config.instanceId, config.environment, "slack", "slack.callbacks.v1", []); + if (!isRecord(delivery) || delivery.version !== 1 || delivery.instanceId !== config.instanceId || delivery.environment !== config.environment + || delivery.bindingId !== packet.bindingId || delivery.acceptedAt !== event.createdAt || delivery.lane !== event.event + || !["command", "action"].includes(String(delivery.lane))) throw badEnvelope(); + event.payload = { slackCallback: delivery }; + } else if (event.payload.slackFormOpen !== undefined) { + const packet = event.payload.slackFormOpen; + if (!isRecord(packet) || typeof packet.operationId !== "string" || packet.bindingId !== `slack-form-open:${packet.operationId}`) throw badEnvelope(); + const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.form-open.v1"); + const delivery = decryptEnvelope(sealed, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(delivery) || delivery.version !== 1 || delivery.instanceId !== config.instanceId || delivery.environment !== config.environment + || !isRecord(delivery.operation) || delivery.operation.id !== packet.operationId || delivery.acceptedAt !== event.createdAt) throw badEnvelope(); + event.payload = { slackFormOpen: delivery }; + } else { const packet = event.payload.slackEvent; if (!isRecord(packet) || typeof packet.bindingId !== "string" || packet.sealed === undefined) throw badEnvelope(); const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.events.v1"); @@ -594,6 +662,7 @@ export function createPaperclipCloudConnector(input: { || delivery.environment !== config.environment || delivery.bindingId !== packet.bindingId || delivery.acceptedAt !== event.createdAt) throw badEnvelope(); event.payload = { slackEvent: delivery }; + } } events.push(event); } catch { @@ -602,6 +671,20 @@ export function createPaperclipCloudConnector(input: { } return { leaseId: opened.leaseId, events, ...(failedEventIds.length ? { failedEventIds } : {}) }; }, + async acknowledgeEventsWithReceipts(values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }): Promise<{ acknowledged: number; formReceipts: { deliveryId: string; receiptJws: string }[] }> { + const response = await call("event-ack", values, { field: "acknowledgement", value: JSON.stringify({ leaseId: values.leaseId, deliveryIds: values.deliveryIds }) }); + if (typeof response.acknowledged !== "number" || !Number.isSafeInteger(response.acknowledged) || response.acknowledged < 0 + || !Array.isArray(response.formReceipts) || response.formReceipts.length > 50) throw badEnvelope(); + const seen = new Set(); + const formReceipts = response.formReceipts.map((value: unknown) => { + if (!isRecord(value) || Object.keys(value).sort().join() !== "deliveryId,receiptJws" || typeof value.deliveryId !== "string" + || !/^slack_form_[a-f0-9]{64}$/.test(value.deliveryId) || !values.deliveryIds.includes(value.deliveryId) || seen.has(value.deliveryId) + || typeof value.receiptJws !== "string" || Buffer.byteLength(value.receiptJws, "ascii") > 12000) throw badEnvelope(); + seen.add(value.deliveryId); return { deliveryId: value.deliveryId, receiptJws: value.receiptJws }; + }); + if (values.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)).some(id => !seen.has(id))) throw badEnvelope(); + return { acknowledged: response.acknowledged, formReceipts }; + }, async acknowledgeEvents(values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }): Promise { const acknowledgement = JSON.stringify({ leaseId: values.leaseId, deliveryIds: values.deliveryIds }); const response = await call("event-ack", values, { field: "acknowledgement", value: acknowledgement }); @@ -633,7 +716,10 @@ function parseCloudHandoff(value: unknown): { kind: "paperclip_cloud"; session: return { kind: "paperclip_cloud", session }; } -export type PaperclipCloudConnector = ReturnType; +export type PaperclipCloudConnector = Omit, "instanceIdentity" | "slackFormOperation"> & { + instanceIdentity?: () => Readonly<{ instanceId: string; environment: PaperclipCloudConnectorEnvironment }>; + slackFormOperation?: ReturnType["slackFormOperation"]; +}; export type PaperclipCloudGoogleWorkspaceConnector = PaperclipCloudConnector; /** Accept persisted Paperclip ID-era records while all new records use the Cloud strategy. */ diff --git a/tests/runner-e2e/source-sdk-evidence.test.ts b/tests/runner-e2e/source-sdk-evidence.test.ts index 188d370a163..99803867d83 100644 --- a/tests/runner-e2e/source-sdk-evidence.test.ts +++ b/tests/runner-e2e/source-sdk-evidence.test.ts @@ -5,6 +5,8 @@ import os from "node:os"; import path from "node:path"; import { promisify } from "node:util"; import { existsSync } from "node:fs"; +import { prepareReviewedSdkReference } from "./source-sdk-reviewed-fixture.js"; +import pins from "./source-sdk-public-inputs.json" with { type: "json" }; import { afterEach, expect, it } from "vitest"; import { stagePrivateDiagnostics } from "./private-diagnostics.js"; import { prepareSourceServerDependencies } from "./source-startup.js"; @@ -110,11 +112,24 @@ it("retains declarations and source maps under names the private result sealer a for (const file of receipt.files) expect(hash(await readFile(path.join(uploadDir, SOURCE_SDK_DIAGNOSTICS, file.retainedPath)))).toBe(file.sha256); await expect(sealManualResultEvidence(context, [], results)).resolves.not.toThrow(); }); -// Needs the built SDK (scripts/ensure-plugin-build-deps.mjs) and the pinned public source. +// Build a genuine OWN normal producer-certified reviewed reference. The live +// CI lock can be regenerated for unrelated PR patches and must not be relabeled reviewed. const builtSdk = ["packages/shared/dist", "packages/plugins/sdk/dist"].every(target => existsSync(new URL(`../../${target}/.paperclip-build-complete`, import.meta.url))); -it.skipIf(!builtSdk)("seals the real built SDK upload, keeping proven key-shaped declarations out", async () => { - const repositoryRoot = await realpath(path.resolve(import.meta.dirname, "../..")); +it("seals a normal producer-certified reviewed-reference SDK, keeping key-shaped declarations out", async () => { + const originalRoot=await realpath(path.resolve(import.meta.dirname,"../..")); + const repositoryRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-reviewed-build-")));roots.push(repositoryRoot); + await prepareReviewedSdkReference(originalRoot,repositoryRoot); + // Existing normal Shared then SDK producer, exact original configs/options. + // Fresh dist directories and OWN cache/Shared resolution; no donor outputs. + expect(existsSync(path.join(repositoryRoot,"packages/shared/dist"))).toBe(false); + expect(existsSync(path.join(repositoryRoot,"packages/plugins/sdk/dist"))).toBe(false); + await prepareSourceServerDependencies(repositoryRoot,{...process.env},false); + for(const directory of ["packages/shared/dist","packages/plugins/sdk/dist"]){ + expect(await realpath(path.join(repositoryRoot,directory))).toBe(path.join(repositoryRoot,directory)); + expect(JSON.parse(await readFile(path.join(repositoryRoot,directory,".paperclip-build-complete"),"utf8"))) + .toMatchObject({sources:expect.stringMatching(/^[a-f0-9]{64}$/),outputs:expect.stringMatching(/^[a-f0-9]{64}$/)}); + } const root = await realpath(await mkdtemp(path.join(os.tmpdir(), "source-sdk-real-unit-"))); roots.push(root); const temporaryRoot = path.join(root, "attempt"), results = path.join(root, "results"), uploadDir = path.join(results, "campaign", "attempt-1"); await mkdir(path.join(temporaryRoot, "artifacts-private"), { recursive: true, mode: 0o700 }); await mkdir(path.join(temporaryRoot, "selected-private-inputs"), { recursive: true, mode: 0o700 }); @@ -126,3 +141,35 @@ it.skipIf(!builtSdk)("seals the real built SDK upload, keeping proven key-shaped const context = { root, protocol: MANUAL_PRIVATE_PROTOCOL, private: true, workflowSha: "a".repeat(40), targetSha: "b".repeat(40), runId: "123", attempt: "1", executionIds: [MANUAL_PRIVATE_EXECUTIONS[0]!] } as ManualPrivateHandoff; await expect(sealManualResultEvidence(context, secrets, results)).resolves.not.toThrow(); }, 120_000); + +it("withholds real unreviewed-build SDK bytes instead of authenticating the CI-resolved lock",async()=>{ + const originalRoot=await realpath(path.resolve(import.meta.dirname,"../..")); + let repositoryRoot=originalRoot,restoreLock:(()=>Promise)|undefined; + const actual=await readFile(path.join(originalRoot,"pnpm-lock.yaml")); + if(hash(actual)===pins.lockSha256){ + // On a reviewed local checkout, make the same refusal observable in an + // OWN genuine-built reference by invalidating only its owning lock input. + repositoryRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-unreviewed-build-")));roots.push(repositoryRoot); + await prepareReviewedSdkReference(originalRoot,repositoryRoot); + await prepareSourceServerDependencies(repositoryRoot,{...process.env},false); + const file=path.join(repositoryRoot,"pnpm-lock.yaml"),reviewed=await readFile(file); + await writeFile(file,Buffer.concat([reviewed,Buffer.from("\n# unreviewed unit input mutation\n")])); + restoreLock=()=>writeFile(file,reviewed); + }else if(!builtSdk)throw new Error("Unreviewed CI build requires its actual normal producer outputs"); + const root=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-unreviewed-refusal-")));roots.push(root); + const temporaryRoot=path.join(root,"attempt"); + await mkdir(path.join(temporaryRoot,"selected-private-inputs"),{recursive:true,mode:0o700}); + stagePrivateDiagnostics(temporaryRoot,[executionId],["synthetic-unit-provider-credential-0123456789"]); + try{ + await expect(retainPreparedSourceSdk({repositoryRoot,temporaryRoot,executionIds:[executionId],sourceRevision,installed:false})) + .rejects.toThrow("Source SDK evidence privacy check failed; bytes withheld"); + const file=path.join(temporaryRoot,"artifacts-private/cases/plugins.install-plugin/snapshots/source-sdk-privacy-observations.json"); + const observation=JSON.parse(await readFile(file,"utf8")); + expect(observation.complete).toBe(false);expect(observation.audit.publicSource).toBeNull(); + expect(observation.audit.bounded).toBe(true);expect(observation.audit.suppressedFindings).toBe(0); + expect(observation.audit.observations).toEqual(expect.arrayContaining([expect.objectContaining({ + pathSha256:hash("packages/shared/dist/validators/agent.d.ts"),detector:3,classification:"unresolved_context"})])); + await expect(stat(path.join(temporaryRoot,"artifacts-private/source-sdk-build/receipt.json"))).rejects.toThrow(); + await expect(stat(path.join(temporaryRoot,"artifacts-private/source-sdk-build/packages"))).rejects.toThrow(); + }finally{await restoreLock?.();} +},120_000); diff --git a/tests/runner-e2e/source-sdk-public-type.test.ts b/tests/runner-e2e/source-sdk-public-type.test.ts index 552d5fdc7f6..7678df7e7e6 100644 --- a/tests/runner-e2e/source-sdk-public-type.test.ts +++ b/tests/runner-e2e/source-sdk-public-type.test.ts @@ -3,6 +3,7 @@ import { copyFile, mkdir, mkdtemp, readFile, realpath, rm, symlink, writeFile } import path from "node:path"; import os from "node:os"; import { beforeAll, afterAll, expect, it } from "vitest"; +import { prepareReviewedSdkReference } from "./source-sdk-reviewed-fixture.js"; import pins from "./source-sdk-public-inputs.json" with { type: "json" }; import { createSdkArtifactScanner, sdkPublicTypeSyntax, verifySdkPublicSource } from "./source-sdk-public-type.js"; import { findSecretLeakMatch, normalizedSecrets } from "./redaction.js"; @@ -12,16 +13,18 @@ const blob = (v: string | Buffer) => createHash("sha1").update(`blob ${Buffer.by const originalRoot = path.resolve(import.meta.dirname, "../.."), file = "packages/shared/dist/validators/synthetic-unit.d.ts", inputFile = file.replace("/dist/", "/src/").replace(/\.d\.ts$/, ".ts"); const source = 'import {z} from "zod";export const schema=z.object({daytona_api_key:z.string().min(1)});'; const declaration = 'import {z} from "zod";export declare const schema:{daytona_api_key:z.ZodOptional};'; -let root: string; let fixtureLockSha256: string; +let root: string; let reviewedRoot: string; let fixtureLockSha256: string; const fixtureInputs: Record = {}; beforeAll(async () => { root = await realpath(await mkdtemp(path.join(os.tmpdir(), "sdk-public-type-synthetic-"))); - for (const [relative] of Object.entries(pins.inputs)) { const dest = path.join(root, relative); await mkdir(path.dirname(dest), { recursive: true }); await copyFile(path.join(originalRoot, relative), dest); fixtureInputs[relative] = blob(await readFile(dest)); } - await copyFile(path.join(originalRoot, "pnpm-lock.yaml"), path.join(root, "pnpm-lock.yaml")); fixtureLockSha256 = hash(await readFile(path.join(root, "pnpm-lock.yaml"))); - await symlink(await realpath(path.join(originalRoot, "node_modules")), path.join(root, "node_modules")); - await symlink(await realpath(path.join(originalRoot, "packages/shared/node_modules")), path.join(root, "packages/shared/node_modules")); + await prepareReviewedSdkReference(originalRoot,root); + reviewedRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"sdk-public-reviewed-reference-"))); + await prepareReviewedSdkReference(originalRoot,reviewedRoot); + for(const relative of Object.keys(pins.inputs))fixtureInputs[relative]=blob(await readFile(path.join(root,relative))); + fixtureLockSha256=hash(await readFile(path.join(root,"pnpm-lock.yaml"))); + }); -afterAll(async () => { if (root) await rm(root, { recursive: true, force: true }); }); +afterAll(async () => { for(const directory of [root,reviewedRoot])if(directory)await rm(directory,{recursive:true,force:true}); }); async function fixture(sourceText = source) { await mkdir(path.dirname(path.join(root, inputFile)), { recursive: true }); await writeFile(path.join(root, inputFile), sourceText); return { ...pins, lockSha256: fixtureLockSha256, inputs: { ...fixtureInputs, [inputFile]: blob(sourceText) } }; @@ -29,13 +32,7 @@ async function fixture(sourceText = source) { it("default public proof accepts the reviewed build lock and still rejects a changed lock", async () => { const directory = await realpath(await mkdtemp(path.join(os.tmpdir(), "sdk-reviewed-runtime-"))); try { - for (const relative of Object.keys(pins.inputs)) { - const destination = path.join(directory, relative); - await mkdir(path.dirname(destination), { recursive: true }); - await copyFile(path.join(originalRoot, relative), destination); - } - await symlink(await realpath(path.join(originalRoot, "node_modules")), path.join(directory, "node_modules")); - await symlink(await realpath(path.join(originalRoot, "packages/shared/node_modules")), path.join(directory, "packages/shared/node_modules")); + await prepareReviewedSdkReference(originalRoot,directory); const reviewed = JSON.parse(await readFile(path.join(originalRoot, "tests/runner-e2e/report-lock.json"), "utf8")); const lock = await readFile(path.join(directory, "pnpm-lock.yaml")); expect(hash(lock)).toBe(reviewed.outputSha256); @@ -56,7 +53,7 @@ it("classifies the shipped agent env schema declaration and keeps a literal at t // Plugin cells retain packages/shared/dist/validators/agent.d.ts, whose // `OPENROUTER_API_KEY: z.ZodOptional` matches the OpenRouter key // shape. Stale default pins left it unresolved and blocked web-server startup. - const repository = await realpath(originalRoot), agentFile = "packages/shared/dist/validators/agent.d.ts"; + const repository = reviewedRoot, agentFile = "packages/shared/dist/validators/agent.d.ts"; const shipped = 'import { z } from "zod";\nexport declare const env: z.ZodObject<{\n OPENROUTER_API_KEY: z.ZodOptional;\n}>;\n'; const scanner = createSdkArtifactScanner(repository, []); expect(await scanner.scan(agentFile, Buffer.from(shipped))).toBe(true); expect((await scanner.audit()).observations.map(row => row.classification)).toEqual(["public_schema_type"]); diff --git a/tests/runner-e2e/source-sdk-reviewed-fixture.ts b/tests/runner-e2e/source-sdk-reviewed-fixture.ts new file mode 100644 index 00000000000..cfdbaafdccf --- /dev/null +++ b/tests/runner-e2e/source-sdk-reviewed-fixture.ts @@ -0,0 +1,72 @@ +// Test-only reviewed SDK reference. Actual CI builds may use an unreviewed +// resolved lock; neither this fixture nor production admits that as public proof. +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { copyFile, mkdir, readFile, readdir, realpath, symlink, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; +import path from "node:path"; +import pins from "./source-sdk-public-inputs.json" with { type: "json" }; +import { prepareReportLock } from "./prepare-report-lock.mjs"; +import { verifySdkPublicSource } from "./source-sdk-public-type.js"; +const sha=(bytes:Buffer)=>createHash("sha256").update(bytes).digest("hex"); +const blob=(bytes:Buffer)=>createHash("sha1").update(`blob ${bytes.length}\0`).update(bytes).digest("hex"); +const gitFile=(root:string,file:string)=>execFileSync("git",["show",`HEAD:${file}`],{cwd:root,maxBuffer:8*1024*1024,stdio:["ignore","pipe","pipe"]}); + +// Create owned directories, link only read-only installed third-party inputs. +// Never link workspace compiled exports or donor node_modules/.cache: the +// normal producer's lock and every Shared/SDK emit stay in this fixture root. +async function thirdPartyLinks(from:string,to:string,installedStore:string){ + const link=async(source:string,destination:string)=>{ + const target=await realpath(source); + // A local workspace may have an unscoped name too. Admit only installed + // third-party store entries, never another checkout/package's exports. + if(target!==installedStore&&!target.startsWith(installedStore+path.sep))return; + await symlink(target,destination); + }; + await mkdir(to,{recursive:true}); + for(const entry of await readdir(from,{withFileTypes:true})){ + if([".cache",".bin","@paperclipai"].includes(entry.name))continue; + const source=path.join(from,entry.name),destination=path.join(to,entry.name); + if(entry.name.startsWith("@")){ + await mkdir(destination,{recursive:true}); + for(const child of await readdir(source))await link(path.join(source,child),path.join(destination,child)); + }else await link(source,destination); + } +} +export async function prepareReviewedSdkReference(original:string,directory:string){ + if(await realpath(directory)!==directory||(await readdir(directory)).length!==0)throw new Error("Reviewed SDK reference needs a fresh owned directory"); + const policyFile="tests/runner-e2e/report-lock.json",patchFile="tests/runner-e2e/report-lock.patch"; + const policyBytes=gitFile(original,policyFile),policy=JSON.parse(policyBytes.toString("utf8")); + if(!(await readFile(path.join(original,policyFile))).equals(policyBytes)||policy.outputSha256!==pins.lockSha256) + throw new Error("Reviewed SDK reference policy does not match public proof"); + const committed=gitFile(original,"pnpm-lock.yaml"); + if(![policy.inputSha256,policy.outputSha256].includes(sha(committed)))throw new Error("Committed SDK reference lock is not reviewed"); + for(const [file,expected]of Object.entries(pins.inputs)){ + const bytes=file==="pnpm-lock.yaml"?committed:await readFile(path.join(original,file)); + if(file!=="pnpm-lock.yaml"&&blob(bytes)!==expected)throw new Error("Reviewed SDK reference source changed"); + const target=path.join(directory,file);await mkdir(path.dirname(target),{recursive:true}); + if(file==="pnpm-lock.yaml")await writeFile(target,bytes);else await copyFile(path.join(original,file),target); + } + for(const file of [policyFile,patchFile]){ + const bytes=gitFile(original,file); + if(!(await readFile(path.join(original,file))).equals(bytes))throw new Error("Reviewed SDK lock policy changed"); + const target=path.join(directory,file);await mkdir(path.dirname(target),{recursive:true}); + await writeFile(target,bytes); + } + // Same existing authenticated input->output correction. No hash refresh, + // changed-lock admission, install, resolver or compiler-option substitution. + await prepareReportLock(directory); + if(sha(await readFile(path.join(directory,"pnpm-lock.yaml")))!==policy.outputSha256)throw new Error("Reviewed SDK reference lock changed"); + const installedStore=await realpath(path.join(original,"node_modules/.pnpm")); + await thirdPartyLinks(path.join(original,"node_modules"),path.join(directory,"node_modules"),installedStore); + for(const name of ["shared","plugins/sdk"])await thirdPartyLinks(path.join(original,"packages",name,"node_modules"),path.join(directory,"packages",name,"node_modules"),installedStore); + for(const modules of ["node_modules","packages/plugins/sdk/node_modules"]){ + const scope=path.join(directory,modules,"@paperclipai");await mkdir(scope,{recursive:true}); + await symlink(path.join(directory,"packages/shared"),path.join(scope,"shared")); + } + const require=createRequire(path.join(directory,"packages/plugins/sdk/package.json")); + const sharedEntry=await realpath(require.resolve("@paperclipai/shared")); + if(!sharedEntry.startsWith(path.join(directory,"packages/shared")+path.sep))throw new Error("SDK reference resolved a donor workspace"); + // Authenticate original compiler/parser/Zod/source pins BEFORE normal build. + await verifySdkPublicSource(directory); +}