diff --git a/doc/idle-sleep-safety.md b/doc/idle-sleep-safety.md index 8cf330272b2..84a53f96f46 100644 --- a/doc/idle-sleep-safety.md +++ b/doc/idle-sleep-safety.md @@ -5,6 +5,16 @@ instance-wide `idleSleepSafety` report for instance administrators and signed Cloud control callers. Ordinary task drains keep their existing behavior. Company members and agents cannot read this report. +Cloud callers carrying full idle catalogs use `POST /api/instance/task-drain/status` +with the same read query and `{ "idleCapabilities": ... }` JSON body. A short +`task-drain:read-body` control JWT binds the exact raw-body SHA256. The existing +10 MiB JSON limit, fatal UTF8 decoding, strict catalog parsing, stack/action/expiry +checks and exact byte digest all apply before consuming the single-use request ID +or installing any capability. Embedded catalogs and body digests cannot be mixed. +Unsigned bodies and browser authority alone cannot authorize this Cloud-only read. +GET remains compatible; POST drain start and DELETE stop are unchanged. + + Instance administrators and signed Cloud control callers may add `idleSleepDiagnostics=1` to an owned safety request for fixed-key blocker diagnostics. This explicit one-shot read evaluates all current predicates in @@ -48,7 +58,7 @@ and `Retry-After: 1`. Health checks and authenticated task-drain control remain available. Ordinary deploy drains do not enable this HTTP gate. The host must handle retry/wake at its ingress before enabling automatic sleep. Health and control mutations remain tracked through completion, including -signed bootstrap writes. Only the task-drain read skips request tracking. +signed bootstrap writes. Only the exact GET task-drain read and POST task-drain/status body read skip request tracking. Its authentication finishes under a separate work token before entering the report, so it counts concurrent writes without blocking on its own scan. diff --git a/doc/testing/slack-cloud-qualification.md b/doc/testing/slack-cloud-qualification.md new file mode 100644 index 00000000000..46198df67f9 --- /dev/null +++ b/doc/testing/slack-cloud-qualification.md @@ -0,0 +1,66 @@ +# Local Slack App/Cloud qualification + +This opt-in qualification uses the normal App owners, a companion Cloud checkout, +real PostgreSQL, and controlled Slack/Codex protocol boundaries. It does not call a +model or Slack, deploy a tenant, or prove Neon suspension. Ordinary tests skip this +one case unless the explicitly named fixture URL and owned state file are present. +The runner supplies the state file; do not construct a receipt or a trusted key set. + +Use Node 24 for App, Node 22 for Cloud, and Python 3.9 or newer. Install and build +both checkouts through their normal repository commands first. In the companion +Cloud checkout, `npm run build:test` must emit its own current source. App needs its +normal workspace prerequisites and server typecheck (`pnpm --filter +@paperclipai/server typecheck`). The runtime helper does not install dependencies, +compile a substitute program, copy another checkout's output, or start PostgreSQL. + +Create one fresh, caller-owned App/Cloud test database pair on the same loopback +PostgreSQL cluster. The names must be `sleep2_slack_form__app_test` and +`sleep2_slack_form__cloud_test`. Provide the matching `/postgres` admin URL. +The URLs must use `postgres://`, an explicit port, and test-only credentials. Never +point the helper at a production database. Normal migration owners apply the full +retained executable migration history; this helper neither substitutes a schema +nor drops databases. + +Run from the App checkout with these environment variables set: + +```sh +export PAPERCLIP_CLOUD_REPO_ROOT=/path/to/owned/cloud-checkout +export PAPERCLIP_SLACK_NODE24=/path/to/node24/bin/node +export PAPERCLIP_SLACK_NODE22=/path/to/node22/bin/node +export PAPERCLIP_SLACK_TEST_DATABASE_URL=postgres://paperclip@127.0.0.1:55448/sleep2_slack_form_local_app_test +export SLACK_EVENT_TEST_DATABASE_URL=postgres://paperclip@127.0.0.1:55448/sleep2_slack_form_local_cloud_test +export PAPERCLIP_TEST_POSTGRES_ADMIN_URL=postgres://paperclip@127.0.0.1:55448/postgres +python3 server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py /path/to/fresh-output-directory +``` + +The runner starts the actual Cloud owner host and exactly one whole normal Vitest +case. The controlled CLI uses only the minted current-run token to check out its +admitted task, post its attributed reply, and complete the task through ordinary +APIs. Its canonical CLI final message lets normal presentation reuse that comment. +The controlled Slack endpoint must receive the exact useful text and provider ID; +a database publication or synthetic receipt cannot replace that observation. + +Each phase keeps the endpoint and connection enabled. After normal work settles, +the case waits for actual used App-worker PostgreSQL sockets to close with the +normal 60-second idle default, inside the existing 95-second settlement budget. +An already-required useful/query call proves the PG write counter is live. It +samples scalar counters immediately before and after the original 30-second window, +with the end sample taken before later inspection queries. The window requires +zero PG writes, connect attempts/connections/closes, live/peak PG sockets, outbound +requests and non-PG connection attempts. It then admits fresh signed work plus its +duplicate, requires a second distinct useful result, and repeats the same window. +No query text, wire payload, token, or private key is recorded by the counters. + +The plaintext same-worker hooks do not establish a cross-process or OS-wide socket +census. Missing Vitest-worker EXIT evidence is reported explicitly, even when its +final PG counters and sampled process-group absence are known. Real detached useful +helpers and version probes require their own entry and EXIT 0 evidence. The runner +joins owned groups and closes the normal Cloud/App owners before returning. It +retains failed evidence and does not mark a failed attempt green after a retry. + +Keep the output directory private: it contains diagnostic logs and source/owner +identity receipts. State containing synthetic fixture credentials stays in a +mode-0700 temporary root and is not copied into the report. A caller may inspect +and remove successful fixture databases only after all owners close. This test +does not qualify managed JWKS distribution/rotation, Photon, Discord, Cloud's own +PostgreSQL quiet, deployed sleep, or physical suspension. diff --git a/patches/@chat-adapter__slack@4.39.0.patch b/patches/@chat-adapter__slack@4.39.0.patch index 55a55b02f10..e44eaeb7ba1 100644 --- a/patches/@chat-adapter__slack@4.39.0.patch +++ b/patches/@chat-adapter__slack@4.39.0.patch @@ -1,5 +1,4 @@ diff --git a/dist/index.js b/dist/index.js -index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909a36c143d 100644 --- a/dist/index.js +++ b/dist/index.js @@ -2154,6 +2154,7 @@ @@ -117,7 +116,37 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 }; } catch (error) { this.handleSlackError(error); -@@ -4183,7 +4203,7 @@ +@@ -4157,8 +4177,10 @@ + this.handleSlackError(error); + } + } +- async updateModal(viewId, modal) { +- const view = modalToSlackView(modal); ++ async updateModal(viewId, modal, options) { ++ const metadata = options?.contextId ? encodeModalMetadata({ contextId: options.contextId, privateMetadata: modal.privateMetadata }) : void 0; ++ const view = modalToSlackView(modal, metadata); ++ if (options) view.external_id = options.externalId; + this.logger.debug("Slack API: views.update", { + viewId, + callbackId: modal.callbackId +@@ -4167,6 +4189,7 @@ + const result = await this._client.views.update( + await this.withToken({ + view_id: viewId, ++ ...(options ? { hash: options.hash } : {}), + view + }) + ); +@@ -4174,7 +4197,7 @@ + viewId: result.view?.id, + ok: result.ok + }); +- return { viewId: result.view?.id }; ++ return options ? { viewId: result.view?.id, viewHash: result.view?.hash } : { viewId: result.view?.id }; + } catch (error) { + this.handleSlackError(error); + } +@@ -4183,7 +4206,7 @@ * Upload files to Slack and share them to a channel. * Returns the file IDs of uploaded files. */ @@ -126,7 +155,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 const bufferResults = await Promise.all( files.map(async (file) => { try { -@@ -4205,7 +4225,9 @@ +@@ -4205,7 +4228,9 @@ (result2) => result2 !== null ); if (fileUploads.length === 0) { @@ -137,7 +166,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 } this.logger.debug("Slack API: files.uploadV2 (batch)", { fileCount: fileUploads.length, -@@ -4219,15 +4241,158 @@ +@@ -4219,14 +4244,157 @@ const result = await this._client.files.uploadV2(uploadArgs); this.logger.debug("Slack API: files.uploadV2 response", { ok: result.ok }); const fileIds = []; @@ -191,7 +220,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + ); + } + return { fileIds, messageId }; - } ++ } + async paperclipResolveFileUploadReceipt(fileIds, threadId) { + if (!Array.isArray(fileIds) || fileIds.length === 0 || fileIds.length > 20 || new Set(fileIds).size !== fileIds.length || fileIds.some((fileId) => typeof fileId !== "string" || !/^F[A-Z0-9]{1,254}$/.test(fileId))) { + const error = new Error("Slack file upload receipt is invalid"); @@ -296,11 +325,10 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + (messageId) => rest.every((messageIds) => messageIds.has(messageId)) + ); + return sharedMessageIds.length === 1 ? sharedMessageIds[0] : null; -+ } + } async editMessage(threadId, messageId, _message) { const message = await this.resolveMessageMentions(_message, threadId); - const ephemeral = this.decodeEphemeralMessageId(messageId); -@@ -4622,6 +4787,8 @@ +@@ -4622,6 +4790,8 @@ this.logger.debug("Slack: starting stream", { channel, threadTs }); const token = await this.getToken(); const streamer = this._client.chatStream({ @@ -309,7 +337,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 channel, thread_ts: threadTs, ...options?.recipientUserId && { -@@ -4667,6 +4834,72 @@ +@@ -4667,6 +4837,72 @@ } }; const fallback = { message: null, mode: "native", nativeRendered: false }; @@ -382,7 +410,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 const updateIntervalMs = options?.updateIntervalMs ?? 1e3; let fallbackSent = ""; let lastFallbackEditAt = 0; -@@ -4709,17 +4942,11 @@ +@@ -4709,17 +4945,11 @@ return; } try { @@ -403,7 +431,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 } switchToFallback(error); await flushFallback(force); -@@ -4736,12 +4963,15 @@ +@@ -4736,12 +4966,15 @@ return; } try { @@ -424,13 +452,26 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 structuredChunksSupported = false; this.logger.warn( "Structured streaming chunk failed, falling back to text-only streaming. Ensure your Slack app manifest includes the agent/assistant feature and the assistant:write scope", -@@ -4749,22 +4979,28 @@ +@@ -4749,22 +4982,28 @@ ); } }; - for await (const chunk of textStream) { - if (options?.signal?.aborted) { - break; +- } +- if (typeof chunk === "string") { +- renderer.push(chunk); +- await flushCommitted(); +- } else if (chunk.type === "markdown_text") { +- renderer.push(chunk.text); +- await flushCommitted(); +- } else { +- await sendStructuredChunk(chunk); +- } +- } +- renderer.finish(); +- await flushCommitted(true); + try { + for await (const chunk of textStream) { + if (options?.signal?.aborted) { @@ -445,29 +486,18 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 + } else { + await sendStructuredChunk(chunk); + } - } -- if (typeof chunk === "string") { -- renderer.push(chunk); -- await flushCommitted(); -- } else if (chunk.type === "markdown_text") { -- renderer.push(chunk.text); -- await flushCommitted(); -- } else { -- await sendStructuredChunk(chunk); -- } ++ } + renderer.finish(); + await flushCommitted(true); + } catch (error) { + // Iterator/renderer/mention lookup failures also follow an irreversible + // accepted native prefix; never classify them as safe whole-send retry. + rethrowNativeFailure(error); - } -- renderer.finish(); -- await flushCommitted(true); ++ } if (fallback.mode === "fallback") { if (options?.stopBlocks || this.feedbackButtons) { this.logger.warn( -@@ -4775,7 +5011,7 @@ +@@ -4775,7 +5014,7 @@ this.logger.debug("Slack: fallback stream complete", { messageId: fallback.message?.id }); @@ -476,7 +506,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 return fallback.message; } const stopBlocks = [ -@@ -4784,21 +5020,27 @@ +@@ -4784,21 +5023,27 @@ ]; let result; try { @@ -507,7 +537,7 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 return fallback.message; } const messageTs = result.message?.ts ?? result.ts; -@@ -5636,7 +5878,8 @@ +@@ -5636,7 +5881,8 @@ const response = await fetch(responseUrl, { method: "POST", headers: { "Content-Type": "application/json" }, @@ -517,3 +547,17 @@ index b80888b6d87e8ad429d18cc955c4c5048a55722a..081e3b1664d3e65d1decf82bbf314909 }); if (!response.ok) { const errorText = await response.text(); +diff --git a/dist/index.d.ts b/dist/index.d.ts +--- a/dist/index.d.ts ++++ b/dist/index.d.ts +@@ -1179,8 +1179,9 @@ + openModal(triggerId: string, modal: ModalElement, contextId?: string): Promise<{ + viewId: string; + }>; +- updateModal(viewId: string, modal: ModalElement): Promise<{ ++ updateModal(viewId: string, modal: ModalElement, options?: { hash: string; externalId: string; contextId?: string }): Promise<{ + viewId: string; ++ viewHash?: string; + }>; + /** + * Upload files to Slack and share them to a channel. diff --git a/server/src/__tests__/chat-run-milestone-work.test.ts b/server/src/__tests__/chat-run-milestone-work.test.ts index 40d3ce01195..1fec371eed3 100644 --- a/server/src/__tests__/chat-run-milestone-work.test.ts +++ b/server/src/__tests__/chat-run-milestone-work.test.ts @@ -1,9 +1,9 @@ import { documentService } from "../services/documents.js"; import { createChatMilestoneFixtures } from "./helpers/chat-milestone-fixtures.js"; import { randomUUID } from "node:crypto"; -import { eq } from "drizzle-orm"; +import { eq, sql } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; -import { agents, agentWakeupRequests, caseHandoffs, cases, chatConversations, chatEndpoints, chatPublications, completionContracts, createDb, nativeRunFinalizations, nativeRunResults, statusDecisions, statusDecisionEffects, workAssessments, subscribeDatabaseWork, heartbeatRuns, issueComments, issueQuestionResponseDeliveries, issueThreadInteractions, issues } from "@paperclipai/db"; +import { applyPendingMigrations, agents, agentWakeupRequests, caseHandoffs, cases, chatConversations, chatEndpoints, chatPublications, completionContracts, createDb, nativeRunFinalizations, nativeRunResults, statusDecisions, statusDecisionEffects, workAssessments, subscribeDatabaseWork, heartbeatRuns, issueComments, issueQuestionResponseDeliveries, issueThreadInteractions, issues } from "@paperclipai/db"; import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js"; @@ -25,6 +25,11 @@ import { NATIVE_STATUS_ARBITER_POLICY_VERSION, type NativeStatusDecision } from import { claimQueuedNativeReviewRun } from "../services/native-runtime/native-review-dispatch.js"; import { createChatPublicationResolution } from "../services/chat-channels/publications/resolution.js"; import { createChatPublicationWorker } from "../services/chat-channels/publications/worker.js"; +import { chatCredentialMutationLease } from "../services/chat-credential-mutation-lease.js"; +import { createSlackFormPublicationStage } from "../services/chat-channels/slack/form-publication.js"; +import { stageSlackFormEnrollment, reconcileSlackFormEnrollment } from "../services/chat-channels/slack/form-owner.js"; +import { parseSlackFormBinding } from "../services/chat-channels/slack/cloud-forms.js"; +import { chatActions } from "@paperclipai/db"; import { CAPABILITIES } from "../services/chat-channels/provider-catalog.js"; import { createChatPublicationFailures } from "../services/chat-channels/publications/failures.js"; import { logger } from "../middleware/logger.js"; @@ -46,7 +51,8 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu beforeAll(async () => { if (!external) temporary = await startEmbeddedPostgresTestDatabase("chat-milestones-"); - db = createDb(external ?? temporary!.connectionString, { maxConnections: 6, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); + else await applyPendingMigrations(external); + db = createDb(external ?? temporary!.connectionString, { maxConnections: 4, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); fixtures = createChatMilestoneFixtures(db); }, 60_000); const fixture = (companyId?: string) => fixtures.create(companyId); @@ -765,6 +771,151 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu expect((await publications()).map(row => row.idempotencyKey)).toContain(`run:${targetId}:queued:${f.endpointId}`); await quiet(); }); + it("parks the first managed form through seven waits and worker restart, then commits one send and its exact grant", async () => { + const identity = { instanceId: "publication-readiness-instance", environment: "test" as const }; + const [originalEndpoint] = await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)); + const binding = parseSlackFormBinding({ version: 1, instanceId: identity.instanceId, companyId: f.companyId, + endpointId: f.endpointId, connectionId: originalEndpoint.connectionId, publicId: "p".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); + const { credentialFingerprint: _fingerprint, eventBindingId, ...base } = binding; + const enrollment = { ...base, bindingId: eventBindingId }; + // Initial fixture configuration has a real pending enrollment, not an + // enrolled owner or fabricated grant/transport receipt. + await db.update(chatEndpoints).set({ publicId: binding.publicId, capabilities: CAPABILITIES.slack, setup: sql`${JSON.stringify({ runtimeGeneration: 1, + slackCloudInstanceId: identity.instanceId, slackCloudSubject: f.userId, + slackCloudEvents: { runtimeGeneration: 1, bindingId: eventBindingId, teamId: binding.teamId, appId: binding.appId } })}::jsonb` }).where(eq(chatEndpoints.id, f.endpointId)); + await db.update(chatConversations).set({ externalConversationId: "slack:CTEST", externalThreadId: "slack:CTEST:1770000000.000001" }).where(eq(chatConversations.id, f.conversationId)); + const ownerId = await db.transaction(tx => stageSlackFormEnrollment(tx, { enrollment, subject: f.userId, environment: identity.environment })); + const source = await run(f, { status: "failed", errorCode: "test_run_failed" }); + const interaction = await issueThreadInteractionService(db).create({ id: f.issueId, companyId: f.companyId }, { + kind: "ask_user_questions", continuationPolicy: "wake_assignee", sourceRunId: source.id, + title: "Choose", payload: { version: 1, title: "Choose", questions: ["first", "second"].map(id => ({ + id, prompt: `Choose ${id}`, selectionMode: "single" as const, required: true, options: [{ id: "yes", label: "Yes" }], + })) }, + }, { agentId: f.agentId, runId: source.id }); + const prompt = (await publications()).find(row => row.idempotencyKey === `interaction:${interaction.id}:${f.endpointId}`)!; + const issued = (await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))) + .filter(row => ["question_form_open", "question_form_submit"].includes(row.kind) && row.payload.publicationId === prompt.id); + expect(issued.map(row => row.kind).sort()).toEqual(["question_form_open", "question_form_submit"]); + const read = async () => (await db.select().from(chatPublications).where(eq(chatPublications.id, prompt.id)))[0]!; + const post = vi.fn(async () => ({ id: "1770000001.000001" })); + const failures = createChatPublicationFailures(db, { + FailedChatRetryPublicationReadError: class extends Error {}, MAX_ERROR_TEXT: 1000, + getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null, + }); + // These are the same controlled authorization/SDK ports as the adjacent + // receipt-race control. Attempt claims, credential leases, enrollment, + // publication receipt and grant transactions are real PostgreSQL owners. + const makeWorker = () => createChatPublicationWorker(db, ({ + CAPABILITIES, settleGitHubAutomaticPublication: async () => false, + ensureDurablePublicationTransport: async publication => publication, + tryProcessTeamsFilePublication: async () => false, + endpointRecord: async () => ({ endpoint: (await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!, + assignedAgentName: "Maya", connectionName: "Chat", applicationId: "test", credentialSecretRefs: [] }), + runtimeContextForRecord: () => ({ generation: 1, localEpoch: 0, credentialFingerprint: "test", version: "test" }), + hasCommittedTaskControlCompletion: async () => false, nonDirectDestinationAllowed: () => true, + runOwnershipMilestoneSupersessionReason: async () => null, progressSupersededByFastResponse: async () => false, + runProgressSupersededByPublishedInteraction: async () => false, currentTaskControlPayload: async publication => publication.payload, + withCredentialMutationLease: chatCredentialMutationLease(db), + closeProgressToReplace: async () => null, interactionResolutionPublicationToReplace: async () => null, + interactionPromptPublicationToReplace: async () => null, runPublicationToReplace: async () => null, + inboundWakePublicationToReplace: async () => null, taskStatusPublicationToReplace: async () => null, + claimPublicationTransportAuthorization: async () => ({ endpoint: (await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!, + conversation: (await db.select().from(chatConversations).where(eq(chatConversations.id, f.conversationId)))[0]!, authorizationActionId: null, preparedText: null }), + prepareSlackFormPublication: createSlackFormPublicationStage(db), postSafePublication: post, + commitTaskControlCompletion: async () => {}, receiptReactionCompletionRunId: async () => null, + finalizePublicationFailure: failures.finalizePublicationFailure, redactError: String, + } satisfies Partial[1]>) as unknown as Parameters[1]); + for (let wait = 0; wait < 7; wait++) { + const before = Date.now(); + // A new worker reconstructs from the same durable publication each time. + await makeWorker().processSelectedPublication(await read()); + expect(post).not.toHaveBeenCalled(); + const parked = await read(); + expect(parked).toMatchObject({ state: "retry", attempts: 0, providerMessageId: null, + redactedError: "form_publication_enrollment_pending", payload: prompt.payload }); + expect(parked.nextAttemptAt!.getTime()).toBeGreaterThanOrEqual(before + 2000); + expect(parked.nextAttemptAt!.getTime()).toBeLessThanOrEqual(Date.now() + 2000); + await vi.advanceTimersByTimeAsync(2000); + } + let logicalEnrollments = 0; + const protocol = vi.fn(async () => { if (++logicalEnrollments === 1) throw new Error("Controlled enrollment reply lost after external acceptance"); return { binding }; }); + const reconcile = () => reconcileSlackFormEnrollment(db, { actionId: ownerId, identity, protocol, + resolveBotToken: async () => "xoxb-controlled-enrollment" }); + await expect(reconcile()).rejects.toThrow("Controlled enrollment reply lost"); + expect(post).not.toHaveBeenCalled(); + expect((await db.select().from(chatActions).where(eq(chatActions.id, ownerId)))[0]!.result?.phase).toBe("staged"); + await reconcile(); + expect((await db.select().from(chatActions).where(eq(chatActions.id, ownerId)))[0]!.result?.phase).toBe("enrolled"); + const selected = await read(); + await makeWorker().processSelectedPublication(selected); + expect(post).toHaveBeenCalledTimes(1); + expect(await read()).toMatchObject({ state: "published", attempts: 1, providerMessageId: "1770000001.000001" }); + const grants = (await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))).filter(row => row.kind === "slack_form_grant"); + expect(grants).toHaveLength(1); + expect(grants[0]!.payload).toMatchObject({ binding, subject: f.userId, environment: identity.environment, + grant: { publicationId: prompt.id, interactionId: interaction.id, providerMessageId: "1770000001.000001" } }); + // Lost local receipt response/reconstruction cannot re-claim the old exact attempt. + await makeWorker().processSelectedPublication(selected); + expect(post).toHaveBeenCalledTimes(1); + expect((await db.select().from(chatActions).where(eq(chatActions.endpointId, f.endpointId))).filter(row => row.kind === "slack_form_grant")).toEqual(grants); + }); + + it("an untyped pre-send error with the same diagnostic still exhausts its ordinary five-attempt budget", async () => { + const failures = createChatPublicationFailures(db, { FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null }); + const [row] = await db.insert(chatPublications).values({ companyId: f.companyId, endpointId: f.endpointId, + conversationId: f.conversationId, issueId: f.issueId, idempotencyKey: `untyped-preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 5 }).returning(); + await expect(failures.finalizePublicationFailure({ ...row, attempts: 4 }, new Error("form_publication_enrollment_pending"), + false, null, undefined, "not_started")).resolves.toBe(true); + expect((await db.select().from(chatPublications).where(eq(chatPublications.id, row.id)))[0]).toMatchObject({ state: "failed", attempts: 5, nextAttemptAt: null }); + }); + + it.each([ + ["not_started", false, "retry", 4], + ["possible", false, "delivery_unknown", 5], + ["not_started", true, "delivery_unknown", 5], + ] as const)("typed readiness at attempt five respects invocation=%s accepted=%s", async (providerInvocation, providerAccepted, state, attempts) => { + const failures = createChatPublicationFailures(db, { FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null }); + const [row] = await db.insert(chatPublications).values({ companyId: f.companyId, endpointId: f.endpointId, + conversationId: f.conversationId, issueId: f.issueId, idempotencyKey: `typed-preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 5 }).returning(); + await expect(failures.finalizePublicationFailure({ ...row, attempts: 4 }, new Error("form_publication_enrollment_pending"), + providerAccepted, null, undefined, providerInvocation, + { kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2000 })).resolves.toBe(true); + const [settled] = await db.select().from(chatPublications).where(eq(chatPublications.id, row.id)); + expect(settled).toMatchObject({ state, attempts, providerMessageId: null, payload: row.payload }); + if (state !== "retry") expect(settled.nextAttemptAt).toBeNull(); + }); + + it.each([ + ["not_started", false, "retry"], + ["possible", false, "delivery_unknown"], + ["not_started", true, "delivery_unknown"], + ] as const)("settles local preparation context %s with accepted=%s as %s", async (providerInvocation, providerAccepted, state) => { + const [claimed] = await db.insert(chatPublications).values({ + companyId: f.companyId, endpointId: f.endpointId, conversationId: f.conversationId, + issueId: f.issueId, idempotencyKey: `preparation:${randomUUID()}`, + payload: { text: "Original publication" }, state: "streaming", attempts: 1, + }).returning(); + const failures = createChatPublicationFailures(db, { + FailedChatRetryPublicationReadError: class extends Error {}, + MAX_ERROR_TEXT: 1000, getTaskBaseUrl: () => null, + invalidateRuntime: async () => false, runtimeCallbackEndpoint: async () => null, + }); + const before = Date.now(); + await expect(failures.finalizePublicationFailure({ ...claimed, attempts: 0 }, + new Error("form_publication_enrollment_pending"), providerAccepted, null, undefined, providerInvocation)).resolves.toBe(true); + const [settled] = await db.select().from(chatPublications).where(eq(chatPublications.id, claimed.id)); + expect(settled).toMatchObject({ state, attempts: 1, providerMessageId: null, payload: claimed.payload }); + if (state === "retry") { + expect(settled.nextAttemptAt?.getTime()).toBeGreaterThanOrEqual(before + 2000); + expect(settled.nextAttemptAt?.getTime()).toBeLessThanOrEqual(Date.now() + 2000); + expect(settled.redactedError).toBe("form_publication_enrollment_pending"); + } else expect(settled.nextAttemptAt).toBeNull(); + }); it.each([ ["retry", null], ["failed", null], ["delivery_unknown", "cancel"], ["delivery_unknown", "mark_delivered"], ] as const)("rechecks resolved prompt ownership after delivery %s (%s)", async (state, resolution) => { diff --git a/server/src/__tests__/cloud-control-task-drain.test.ts b/server/src/__tests__/cloud-control-task-drain.test.ts index 46c2962edc5..985c7188325 100644 --- a/server/src/__tests__/cloud-control-task-drain.test.ts +++ b/server/src/__tests__/cloud-control-task-drain.test.ts @@ -1,6 +1,6 @@ import { connectionEventWakeRoutes } from "../routes/connection-event-wake.js"; import { registerConnectionEventWake } from "../services/connection-event-wake.js"; -import { generateKeyPairSync, sign } from "node:crypto"; +import { createHash, generateKeyPairSync, sign } from "node:crypto"; import express from "express"; import request from "supertest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; @@ -382,3 +382,65 @@ describe("cloudControlMiddleware", () => { expect(res.status).toBe(200); }); }); + + +describe("body-bound Cloud idle status assertion", () => { + const body = Buffer.from(JSON.stringify({ idleCapabilities: { deadlineCatalog: "domain_deadlines_v1" } })); + const digest = (value: Uint8Array) => createHash("sha256").update(value).digest("hex"); + const token = (raw: Uint8Array, extra: Record = {}) => controlAssertion({ action: "task-drain:read-body", claims: { bodySha256: digest(raw), ...extra } }); + const read = (compactJws: string, rawBody?: Uint8Array) => verifyCloudControlAssertion({ compactJws, rawBody, + expectedAction: "task-drain:read-body", env: ENV, now: NOW }); + beforeEach(() => resetCloudControlReplayFenceForTests()); + it("installs the complete closed body only after its exact-byte binding and rejects replay", () => { + const jws = token(body); + const claims = read(jws, body); + expect(claims.idleCapabilities).toEqual({ deadlineCatalog: "domain_deadlines_v1" }); + expect(Object.isFrozen(claims.idleCapabilities)).toBe(true); + expect(() => read(jws, body)).toThrow(/already been used/); + }); + it("rejects changed or missing bytes without burning the valid exact request", () => { + const jws = token(body); + expect(() => read(jws)).toThrow(/body binding/); + expect(() => read(jws, Buffer.concat([body, Buffer.from(" ")]))).toThrow(/digest changed/); + expect(read(jws, body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("rejects malformed UTF8 before consuming the same nonce", () => { + const requestId = "invalid-utf8-status-read"; + const malformed = Buffer.concat([Buffer.from('{"idleCapabilities":{"deadlineCatalog":"'), Buffer.from([0xff]), Buffer.from('"}}')]); + expect(() => read(token(malformed, { requestId }), malformed)).toThrow(TypeError); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it.each([ + ["unknown catalog", '{"idleCapabilities":{"deadlineCatalog":"unowned"}}'], + ["extra field", '{"idleCapabilities":{"deadlineCatalog":"domain_deadlines_v1"},"extra":true}'], + ["array body", '[{"idleCapabilities":{"deadlineCatalog":"domain_deadlines_v1"}}]'], + ["malformed JSON", "{"], + ])("rejects %s before consuming the same nonce", (_name, text) => { + const requestId = "rejected-status-body"; + const raw = Buffer.from(text); + expect(() => read(token(raw, { requestId }), raw)).toThrow(); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("rejects mixed embedded capabilities and malformed digest without consuming a nonce", () => { + for (const extra of [{ idleCapabilities: { deadlineCatalog: "domain_deadlines_v1" } }, { bodySha256: "invalid" }]) { + resetCloudControlReplayFenceForTests(); + const requestId = "mixed-status-body"; + expect(() => read(token(body, { requestId, ...extra }), body)).toThrow(/body binding/); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + } + }); + it("enforces the unchanged ten MiB JSON limit before consuming the same nonce", () => { + const raw = Buffer.alloc(10 * 1024 * 1024 + 1, 32), requestId = "oversized-status-body"; + expect(() => read(token(raw, { requestId }), raw)).toThrow(/body binding/); + expect(read(token(body, { requestId }), body).idleCapabilities?.deadlineCatalog).toBe("domain_deadlines_v1"); + }); + it("does not turn body-read authority into start or stop authority", () => { + for (const action of ["task-drain:start", "task-drain:stop"] as const) { + const jws = token(body); + expect(() => verifyCloudControlAssertion({ compactJws: jws, expectedAction: action, rawBody: body, env: ENV, now: NOW })).toThrow(/authorize/); + expect(read(jws, body).action).toBe("task-drain:read-body"); + expect(() => verifyCloudControlAssertion({ compactJws: controlAssertion({ action, claims: { bodySha256: digest(body) } }), + expectedAction: action, rawBody: body, env: ENV, now: NOW })).toThrow(/Body|body/); + } + }); +}); diff --git a/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs b/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs new file mode 100755 index 00000000000..87c115216ac --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/controlled-codex-cli.cjs @@ -0,0 +1,19 @@ +#!/usr/bin/env node +// Controlled provider executable, NOT the real Codex model or service. +const assert = require("node:assert/strict"), fs = require("node:fs"); +require("./scoped-cli-prewire.cjs"); +assert.equal(process.versions.node.split(".")[0], "24", "The qualification Node24 executable is required"); +const version = process.argv.length === 3 && process.argv[2] === "--version"; +fs.appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify({phase:"controlled-cli-entry",pid:process.pid,ppid:process.ppid, + execPath:process.execPath,argv:process.argv,execArgv:process.execArgv,versionProbe:version,realModelProvider:false})+"\n"); +if (version) { console.log("paperclip-controlled-codex-jsonl-fixture (no real model)"); } +else { + assert.ok(process.argv.slice(2).includes("exec"), "Only actual normal Codex exec invocation accepted"); + assert.ok(process.env.PAPERCLIP_RUN_ID && process.env.PAPERCLIP_API_KEY, "Normal run token required"); + // Consume normal prompt stdin without executing it or leaking it to logs. + (async () => { + let size=0; for await (const chunk of process.stdin) { size+=chunk.length; assert.ok(size<=2*1024*1024,"Controlled prompt bound"); } + console.log(JSON.stringify({type:"thread.started",thread_id:`controlled-${process.env.PAPERCLIP_RUN_ID}`})); + await import("./normal-scoped-useful-final.mjs"); + })().catch(error=>{console.error(error);process.exitCode=1;}); +} diff --git a/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs b/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs new file mode 100644 index 00000000000..5dca71db88a --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/enabled-quiet-host.mjs @@ -0,0 +1,400 @@ +import assert from "node:assert/strict"; +import { createHash, generateKeyPairSync, randomBytes, randomUUID } from "node:crypto"; +import { once } from "node:events"; +import { writeFile } from "node:fs/promises"; +import { createServer } from "node:http"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import path from "node:path"; +import { realpath, readFile } from "node:fs/promises"; + +const cloudRoot = await realpath(process.env.PAPERCLIP_CLOUD_REPO_ROOT ?? ""); +const cloudPackage = JSON.parse(await readFile(path.join(cloudRoot, "package.json"), "utf8")); +assert.equal(cloudPackage.name, "@paperclip/cloud-harness"); +const { createIdleIncomingWakeCapability } = await import(pathToFileURL(path.join(cloudRoot, "dist/sleep/incoming-wake-capability.js")).href); +const { default: pg } = await import(pathToFileURL(path.join(cloudRoot, "node_modules/pg/lib/index.js")).href); +const { PostgresCloudHarnessRegistry } = await import(pathToFileURL(path.join(cloudRoot, "dist/registry/postgres.js")).href); +const { PostgresConnectorStore } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/postgres.js")).href); +const { ConnectorBroker } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/broker.js")).href); +const { SLACK_FORM_OPERATION_BYTES } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-forms.js")).href); +const { ConnectorEventDeliveryWorker, createConnectorEventNotifier } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/event-delivery.js")).href); +const { providerRegistryFromEnvironment } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/providers.js")).href); +const { handleTenantSlackForm } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-form-http.js")).href); +const { handleTenantSlackEvent } = await import(pathToFileURL(path.join(cloudRoot, "dist/connections/slack-event-http.js")).href); +const { InMemoryProviderSecretStore } = await import(pathToFileURL(path.join(cloudRoot, "dist/providers/neon.js")).href); +const { FakeCloudStackProvider } = await import(pathToFileURL(path.join(cloudRoot, "dist/providers/fake.js")).href); +const { CloudStackProvisioner } = await import(pathToFileURL(path.join(cloudRoot, "dist/provisioner/provisioner.js")).href); +const { createRuntimeIdentitySignerFromEnv } = await import(pathToFileURL(path.join(cloudRoot, "dist/runtime/runtime-identity-assertion.js")).href); +assert.equal(process.versions.node.split(".")[0], "22"); +assert.equal(process.argv.length, 3); +assert.ok(process.env.SLACK_FORM_FIXTURE_SCOPES?.length); +const stateFile = process.argv[2]; +assert.ok(stateFile && path.isAbsolute(stateFile)); +const databaseUrl = process.env.SLACK_EVENT_TEST_DATABASE_URL; +const database = new URL(databaseUrl ?? ""); +assert.equal(database.protocol, "postgres:"); +assert.equal(database.hostname, "127.0.0.1"); +assert.match(database.port, /^[0-9]+$/, "Explicit database port required before Pool or migrations"); +const databasePort = Number(database.port); +assert.ok(Number.isInteger(databasePort) && databasePort >= 1 && databasePort <= 65535); +assert.match(database.pathname, /^\/sleep2_slack_form_[a-z0-9_]+_cloud_test$/); +const pool = new pg.Pool({ connectionString: databaseUrl, max: 2 }); +const registry = new PostgresCloudHarnessRegistry(pool); +const store = new PostgresConnectorStore(pool); +const secrets = new InMemoryProviderSecretStore(); +const signing = generateKeyPairSync("ed25519"), sealing = generateKeyPairSync("x25519"), control = generateKeyPairSync("ed25519"); +const controlPublic = { ...control.publicKey.export({ format: "jwk" }), kid: "slack-form-journey", use: "sig", alg: "EdDSA" }; +const signer = createRuntimeIdentitySignerFromEnv({ + CLOUD_HARNESS_RUNTIME_IDENTITY_SIGNING_JWK: JSON.stringify({ ...control.privateKey.export({ format: "jwk" }), kid: controlPublic.kid, use: "sig", alg: "EdDSA" }), + CLOUD_HARNESS_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [controlPublic] }), +}, { required: true }); +assert.ok(signer?.signControlAssertion); +const fixture = { + companyId: randomUUID(), endpointId: randomUUID(), connectionId: randomUUID(), applicationId: randomUUID(), agentId: randomUUID(), + actorId: `form-owner-${randomUUID()}`, instanceId: `form-stack-${randomUUID()}`, publicId: randomBytes(32).toString("base64url"), + teamId: "TTEST", appId: "ATEST", botId: "UBOT", userId: "UOPERATOR", channelId: "CTEST", threadTs: "1788.100000", + botToken: "xoxb-controlled-journey-only", signingSecret: randomBytes(16).toString("hex"), controlNonce: randomBytes(32).toString("base64url"), + signPrivate: signing.privateKey.export({ format: "jwk" }).d, sealPrivate: sealing.privateKey.export({ format: "jwk" }).d, + controlJwks: { keys: [controlPublic] }, +}; +const failures = [], calls = [], views = new Map(), triggers = new Map(), postedMessages = new Map(); +const faults = { dropImmediateHint: false, loseTerminalReply: false, loseAckReply: false }; +let tenantOrigin = process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN, broker, worker, stopping = false, wakerStarts = 0, droppedHints = 0, lostTerminalReplies = 0, lostAckReplies = 0; +const json = (res, status, body) => { res.writeHead(status, { "content-type": "application/json", connection: "close" }); res.end(JSON.stringify(body)); }; +const read = async req => { + const chunks = []; let size = 0; + for await (const chunk of req) { size += chunk.length; assert.ok(size <= 2_100_000); chunks.push(Buffer.from(chunk)); } + return Buffer.concat(chunks); +}; +const loopback = value => { const url = new URL(value); assert.equal(url.protocol, "http:"); assert.equal(url.hostname, "127.0.0.1"); assert.equal(url.pathname, "/"); return url.origin; }; +tenantOrigin = loopback(tenantOrigin); +fixture.tenantOrigin = tenantOrigin; +const notification = createConnectorEventNotifier({ + getStack: async id => { + assert.equal(id, fixture.instanceId); + const stack = await registry.getStack(id); + if (!stack || !tenantOrigin) return undefined; + return { id: stack.id, lifecycleState: stack.lifecycleState, providerRefs: [{ provider: "railway", resourceType: "service_domain:web", resourceId: "fixture-tenant", name: "tenant.example", metadata: { upstreamOrigin: tenantOrigin } }] }; + }, + wake: async id => { assert.equal(id, fixture.instanceId); return { status: "awake" }; }, + sign: input => signer.signControlAssertion(input), +}); +const hint = async () => { const instance = await store.getInstance(fixture.instanceId); assert.ok(instance); await notification(instance); }; +const restartWaker = async () => { + await worker?.stop(); + worker = new ConnectorEventDeliveryWorker({ store, deliver: notification, onError: error => failures.push(String(error?.stack ?? error)) }); + wakerStarts++; worker.start(); +}; +// PRIVATE controlled-provider wire witnesses: never derive from App rows/ACKs. +const wireRequests = new WeakMap(); +const receivedSessionStatuses = new Map(); +const wireMethods = new Set(["auth.test", "users.info", "conversations.info", "conversations.list", "conversations.open", "conversations.history", "conversations.replies", + "reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral", "chat.postMessage", "chat.scheduleMessage", "chat.deleteScheduledMessage", "chat.delete", + "chat.startStream", "chat.appendStream", "chat.stopStream", "agents.sessions.setStatus", "assistant.threads.setStatus", "assistant.threads.setTitle", "assistant.threads.setSuggestedPrompts", "views.open", "views.update"]); +function usefulReplyWitness(message) { + if (message.channel !== fixture.channelId || message.user !== fixture.botId || !postedMessages.has(message.ts) + || ![fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(message.thread_ts)) return null; + // The pinned SDK sends markdown_text exclusively for {markdown}; raw/string + // messages use text. Do not scan unrelated JSON/metadata for an apparent run. + let text; + if (typeof message.markdown_text === "string") { + if (message.text !== undefined || message.blocks !== undefined) return null; + text = message.markdown_text; + } else if (typeof message.text === "string" && message.blocks === undefined) text = message.text; + else return null; + const match = /^Scoped Slack useful reply for run ([a-f0-9-]{36})$/.exec(text); + if (!match || match[0] !== text) return null; + return { id: message.ts, threadTs: message.thread_ts, runId: match[1] }; +} +function wireSourceFrames(error) { + return String(error?.stack ?? "").split("\n").slice(1, 9).flatMap(frame => { + const match = /slack-form-journey-host\.mjs:(\d+):(\d+)/.exec(frame); + return match ? [`controlled-provider:${match[1]}:${match[2]}`] : []; + }); +} +const platform = async (req, res, pathname) => { + const raw = await read(req), encoding = req.headers["content-type"] ?? ""; + const body = encoding.includes("application/json") ? JSON.parse(raw.toString()) : Object.fromEntries(new URLSearchParams(raw.toString())); + if (typeof body.view === "string") body.view = JSON.parse(body.view); + const method = pathname.slice("/fixture/slack/".length); + assert.ok(req.method === "POST" || req.method === "GET" && method === "conversations.list", "fixture_method_contract"); + const token = req.headers.authorization?.replace(/^Bearer /, "") ?? body.token; + assert.equal(token, fixture.botToken); + wireRequests.set(req, { method: wireMethods.has(method) ? method : "unsupported-slack-method", methodDigest: createHash("sha256").update(method).digest("hex"), + ownedProviderId: typeof body.ts === "string" && postedMessages.has(body.ts) ? body.ts + : typeof body.oldest === "string" && postedMessages.has(body.oldest) ? body.oldest : null }); + calls.push({ method, at: Date.now(), ...(method.startsWith("views.") ? { body } : {}) }); + if (method === "auth.test") { + res.setHeader("x-oauth-scopes", process.env.SLACK_FORM_FIXTURE_SCOPES ?? ""); + return json(res, 200, { ok: true, team_id: fixture.teamId, user_id: fixture.botId, user: "fixture" }); + } + if (method === "users.info") return json(res, 200, { ok: true, user: { id: fixture.userId, name: "operator", is_bot: false, profile: { display_name: "Operator" } } }); + if (method === "conversations.info") return json(res, 200, { ok: true, channel: { id: fixture.channelId, name: "fixture", is_member: true } }); + if (method === "conversations.list") return json(res, 200, { ok: true, channels: [{ id: fixture.channelId, name: "fixture", is_member: true }], response_metadata: { next_cursor: "" } }); + if (method === "agents.sessions.setStatus") { + // Exact production setSlackSessionStatus wire contract; record the real + // idempotent effect before acknowledging it. Never settle App rows here. + assert.equal(req.method, "POST"); assert.ok(encoding.includes("application/json")); + assert.deepEqual(Object.keys(body).sort(), ["channel_id", "status", "thread_ts"]); + assert.equal(body.channel_id, fixture.channelId); + assert.ok([fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(body.thread_ts)); + assert.ok(["processing", "active", "suspended", "closed"].includes(body.status)); + const previous = receivedSessionStatuses.get(body.thread_ts) ?? null; + receivedSessionStatuses.set(body.thread_ts, body.status); + console.info(JSON.stringify({ event: "controlled-slack-session-status-received", method, + methodDigest: createHash("sha256").update(method).digest("hex"), status: 200, + ownedChannel: true, ownedThreadTs: body.thread_ts, previousStatus: previous, sessionStatus: body.status, + effectRecordedBeforeAcknowledgement: receivedSessionStatuses.get(body.thread_ts) === body.status })); + return json(res, 200, { ok: true }); + } + if (["reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral"].includes(method)) return json(res, 200, { ok: true, ts: fixture.threadTs, channel: fixture.channelId }); + if (method === "chat.postMessage") { + assert.equal(body.channel, fixture.channelId); + const ts = `1788.${String(200000 + calls.length).padStart(6, "0")}`; + const message = { ...body, ts, user: fixture.botId }; + postedMessages.set(ts, message); + const witness = usefulReplyWitness(message); + console.info(JSON.stringify({ event: "controlled-slack-message-received", method: "chat.postMessage", status: 200, + ownedProviderId: ts, ownedChannel: message.channel === fixture.channelId, + ownedThread: [fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(message.thread_ts), + contentField: typeof message.markdown_text === "string" ? "markdown_text" : typeof message.text === "string" ? "text" : "other", + exactUsefulText: witness !== null, usefulRunId: witness?.runId ?? null, + contentDigest: typeof message.markdown_text === "string" || typeof message.text === "string" + ? createHash("sha256").update(message.markdown_text ?? message.text).digest("hex") : null })); + return json(res, 200, { ok: true, ts, channel: fixture.channelId, message }); + } + if (method === "conversations.replies") { + assert.equal(body.channel, fixture.channelId); + assert.ok([fixture.threadTs, (Number(fixture.threadTs) + 1).toFixed(6)].includes(body.ts)); + assert.equal(body.limit, "1"); + assert.equal(body.inclusive, "true"); + assert.equal(body.cursor, undefined); + assert.equal(body.latest, undefined); + assert.equal(typeof body.oldest, "string"); + const message = postedMessages.get(body.oldest); + assert.ok(message, "only a message created by this fixture's real chat.postMessage may be fetched"); + assert.equal(message.thread_ts, body.ts); + if (Object.hasOwn(body, "token")) assert.equal(body.token, fixture.botToken); + assert.deepEqual(Object.keys(body).filter(key => key !== "token").sort(), ["channel", "inclusive", "limit", "oldest", "ts"]); + return json(res, 200, { ok: true, messages: [message], has_more: false, response_metadata: { next_cursor: "" } }); + } + if (method === "views.open") { + const deadline = triggers.get(body.trigger_id); + assert.ok(deadline, "only the original issued trigger may open a view"); + triggers.delete(body.trigger_id); + if (Date.now() >= deadline) return json(res, 200, { ok: false, error: "expired_trigger_id" }); + const targets = await pool.query("SELECT delivery_id FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged'", [fixture.instanceId]); + assert.equal(targets.rows.length, 1, "the original target must be durable before trigger spend"); + assert.equal(body.view.notify_on_close, true); + assert.ok(!body.view.blocks.some(block => block.type === "input")); + assert.equal(body.view.callback_id, "paperclip_slack_form_loading_v1"); + const view = { ...body.view, id: "VJOURNEY", hash: "loading-hash", app_id: fixture.appId, state: { values: {} } }; + views.set(view.id, view); return json(res, 200, { ok: true, view }); + } + if (method === "views.update") { + const old = views.get(body.view_id); assert.ok(old); + assert.equal(body.hash, old.hash); assert.equal(body.view.external_id, old.external_id); + assert.equal(body.view.notify_on_close, true); assert.ok(body.view.blocks.some(block => block.type === "input")); + assert.ok(tenantOrigin); + const response = await fetch(`${tenantOrigin}/__fixture/committed-hold`, { headers: { "x-fixture-control": fixture.controlNonce } }); + assert.equal(response.status, 200); + const hold = await response.json(); + assert.equal(hold.status, "processing"); assert.equal(hold.phase, "update_attempted"); + assert.deepEqual(hold.view, { id: old.id, hash: old.hash, externalId: old.external_id }); + assert.equal(hold.operationId, old.external_id.slice("pc-form-".length)); + const view = { ...old, ...body.view, hash: "useful-hash" }; + views.set(view.id, view); return json(res, 200, { ok: true, view }); + } + throw new Error(`Unexpected actual SDK request ${method}`); +}; +const snapshot = async () => ({ + sessionStatuses: [...receivedSessionStatuses].map(([threadTs, status]) => ({ threadTs, status })), + usefulReplies: [...postedMessages.values()].flatMap(message => { + const witness = usefulReplyWitness(message); + return witness ? [witness] : []; + }), + calls, views: [...views.values()], failures, wakerStarts, droppedHints, lostTerminalReplies, lostAckReplies, + instance: await store.getInstance(fixture.instanceId), + bindings: (await store.listWebhookBindings(fixture.instanceId)).map(row => ({ id: row.id, profile: row.profile, metadata: row.providerMetadata })), + targets: (await pool.query("SELECT delivery_id,lease_id,lease_expires_at FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged' ORDER BY delivery_id", [fixture.instanceId])).rows, +}); +const server = createServer((req, res) => { void (async () => { + const url = new URL(req.url, fixture.origin); + if (url.pathname.startsWith("/__fixture/")) { + assert.equal(req.headers["x-fixture-control"], fixture.controlNonce); + if (req.method === "GET" && url.pathname === "/__fixture/state") return json(res, 200, await snapshot()); + if (req.method === "POST" && url.pathname === "/__fixture/max-header") { + assert.ok(tenantOrigin); + const instance = await store.getInstance(fixture.instanceId); assert.ok(instance); + const initial = await store.listWebhookBindings(fixture.instanceId); + assert.equal(initial.length, 2); + assert.ok(broker.slackEvents && broker.slackForms); + for (let index = 1; index < 500; index++) { + const event = await broker.slackEvents.enroll(instance, fixture.actorId, { + companyId: fixture.companyId, endpointId: randomUUID(), connectionId: randomUUID(), publicId: randomBytes(32).toString("base64url"), + runtimeGeneration: 1, teamId: fixture.teamId, appId: `AHEADER${index}`, signingSecret: randomBytes(16).toString("hex"), + }); + await broker.slackForms.enroll(instance, fixture.actorId, event, `xoxb-controlled-header-${index}`); + } + const rows = await store.listWebhookBindings(fixture.instanceId); assert.equal(rows.length, 1000); + const start = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:start", requestId: randomUUID() }); + const started = await fetch(`${tenantOrigin}/api/instance/task-drain`, { method: "POST", redirect: "error", + headers: { "content-type": "application/json", "x-paperclip-cloud-control": start }, body: JSON.stringify({ purpose: "idle", ttlMs: 30000 }), signal: AbortSignal.timeout(10000) }); + assert.equal(started.status, 200); const drain = await started.json(); assert.equal(typeof drain.ownerId, "string"); + let primary, result; + try { + const incomingWake = await createIdleIncomingWakeCapability({ stackId: fixture.instanceId, ownerId: drain.ownerId, + apiEnabled: false, brokerEnabled: true, store, readSlackCoverage: id => broker.readSlackCoverage(id) }); + assert.ok(incomingWake?.slackBroker); assert.equal(incomingWake.slackBroker.entries.length, 500); + const token = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:read", requestId: randomUUID(), idleCapabilities: { incomingWake } }); + const url = `${tenantOrigin}/api/instance/task-drain?idleSleepSafety=1&idleSleepDiagnostics=1&ownerId=${encodeURIComponent(drain.ownerId)}`; + const baseline = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:read", requestId: randomUUID() }); + const tampered = baseline.slice(0, -1) + (baseline.endsWith("A") ? "B" : "A"); + const denied = await fetch(url, { headers: { "x-paperclip-cloud-control": tampered }, redirect: "error", signal: AbortSignal.timeout(10000) }); + const tamperedStatus = denied.status; await denied.arrayBuffer(); assert.equal(tamperedStatus, 401); + const response = await fetch(url, { headers: { "x-paperclip-cloud-control": token }, redirect: "error", signal: AbortSignal.timeout(10000) }); + const raw = await response.text(); let report; try { report = JSON.parse(raw); } catch {} + result = { bindingCount: rows.length, coverageEntries: incomingWake.slackBroker.entries.length, + headerBytes: Buffer.byteLength(token), responseStatus: response.status, normalRouteReached: !!report?.idleSleepSafety, tamperedStatus }; + console.info(JSON.stringify({ diagnostic: "slack_form_header_wire", ...result })); + } catch (error) { primary = error; throw error; } + finally { + const token = signer.signControlAssertion({ stackId: fixture.instanceId, action: "task-drain:stop", requestId: randomUUID() }); + try { const response = await fetch(`${tenantOrigin}/api/instance/task-drain?ownerId=${encodeURIComponent(drain.ownerId)}`, { method: "DELETE", + headers: { "x-paperclip-cloud-control": token }, redirect: "error", signal: AbortSignal.timeout(10000) }); + assert.equal(response.status, 200); await response.arrayBuffer(); + } catch (error) { throw primary ? new AggregateError([primary, error], "Header preflight and normal hold cleanup failed") : error; } + } + return json(res, 200, result); + } + if (req.method === "POST" && url.pathname === "/__fixture/idle-projection") { + assert.ok(tenantOrigin); + const request = async (method, action, suffix, body, idleCapabilities) => { + const response = await fetch(`${tenantOrigin}/api/instance/task-drain${suffix}`, { method, redirect: "error", + headers: { "content-type": "application/json", "x-paperclip-cloud-control": signer.signControlAssertion({ stackId: fixture.instanceId, + action, requestId: randomUUID(), ...(idleCapabilities ? { idleCapabilities } : {}) }) }, + ...(body ? { body: JSON.stringify(body) } : {}), signal: AbortSignal.timeout(10_000) }); + assert.equal(response.status, 200); return await response.json(); + }; + const drain = await request("POST", "task-drain:start", "", { purpose: "idle", ttlMs: 30_000 }); + assert.equal(typeof drain.ownerId, "string"); + let primary, projection; + try { + const incomingWake = await createIdleIncomingWakeCapability({ stackId: fixture.instanceId, ownerId: drain.ownerId, + apiEnabled: false, brokerEnabled: true, store, readSlackCoverage: id => broker.readSlackCoverage(id) }); + const report = await request("GET", "task-drain:read", `?idleSleepSafety=1&idleSleepDiagnostics=1&ownerId=${encodeURIComponent(drain.ownerId)}`, + undefined, incomingWake ? { incomingWake } : undefined); + const safety = report.idleSleepSafety; assert.ok(safety?.diagnostics); + assert.ok(["none", "present", "unknown"].includes(safety.backgroundWork)); + assert.ok(safety.diagnostics.checks.length <= 200); + projection = { backgroundWork: safety.backgroundWork, capabilityPresent: Boolean(incomingWake), + checks: safety.diagnostics.checks.map(value => ({ reason: value.reason, state: value.state })) }; + } catch (error) { primary = error; throw error; } + finally { + try { await request("DELETE", "task-drain:stop", `?ownerId=${encodeURIComponent(drain.ownerId)}`); } + catch (error) { throw primary ? new AggregateError([primary, error], "Idle report and hold cleanup failed") : error; } + } + return json(res, 200, { idleProjection: projection }); + } + const body = JSON.parse((await read(req)).toString() || "{}"); + if (req.method === "POST" && url.pathname === "/__fixture/tenant") { assert.equal(loopback(body.origin), tenantOrigin); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/trigger") { assert.equal(typeof body.id, "string"); assert.equal(typeof body.actionTs, "string"); assert.ok(!triggers.has(body.id)); triggers.set(body.id, Number(body.actionTs) * 1000 + 3000); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/faults") { for (const key of Object.keys(body)) { assert.ok(Object.hasOwn(faults, key)); assert.equal(body[key], true); faults[key] = true; } return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/restart-waker") { await restartWaker(); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/stop-waker") { await worker?.stop(); return json(res, 200, {}); } + if (req.method === "POST" && url.pathname === "/__fixture/shutdown") { json(res, 200, await snapshot()); queueMicrotask(() => { void shutdown(); }); return; } + throw new Error("Unknown closed fixture operation"); + } + if (url.pathname.startsWith("/fixture/slack/")) return platform(req, res, url.pathname); + if (url.pathname.startsWith("/v1/connector/")) { + const loss = url.pathname === "/v1/connector/events/ack" && faults.loseAckReply; + if (url.pathname === "/v1/connector/events/ack") { + const original = await pool.query("SELECT delivery_id FROM cloud_harness.connector_webhook_delivery_targets WHERE connector_instance_id=$1 AND status<>'acknowledged'", [fixture.instanceId]); + if (original.rows.some(row => row.delivery_id.startsWith("slack_form_"))) { + assert.ok(tenantOrigin); + const proof = await fetch(`${tenantOrigin}/__fixture/proof-accepted`, { headers: { "x-fixture-control": fixture.controlNonce } }); + assert.equal(proof.status, 200); + const accepted = await proof.json(); assert.equal(accepted.deliveryId, original.rows.find(row => row.delivery_id.startsWith("slack_form_")).delivery_id); + assert.equal(accepted.proof.operationId, accepted.deliveryId.slice("slack_form_".length)); + assert.ok(accepted.proof.appReceiptId); + } + } + if (url.pathname === "/v1/connector/slack-forms" && faults.loseTerminalReply) { + const end = res.end.bind(res); + res.end = function(chunk, ...args) { + if (typeof chunk === "string" && chunk.includes('"slackForm"') && chunk.includes('"sealed"')) { + let requestKind; + try { requestKind = JSON.parse(JSON.parse(Buffer.concat(captured).toString()).operation).kind; } catch {} + if (requestKind === "terminal-ack") { faults.loseTerminalReply = false; lostTerminalReplies++; res.destroy(); return res; } + } + return end(chunk, ...args); + }; + const captured = []; let capturedBytes = 0; + const originalIterator = req[Symbol.asyncIterator].bind(req); + req[Symbol.asyncIterator] = async function* () { + for await (const chunk of originalIterator()) { + const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk); capturedBytes += buffer.byteLength; + if (capturedBytes <= SLACK_FORM_OPERATION_BYTES + 16_000) captured.push(Buffer.from(buffer)); + yield chunk; + } + }; + } + if (loss) { faults.loseAckReply = false; const end = res.end.bind(res); res.end = function(chunk, ...args) { if (res.statusCode === 200) { lostAckReplies++; res.destroy(); return res; } return end(chunk, ...args); }; } + const handled = await broker.handle(req, res, url); assert.equal(handled, true); return; + } + assert.equal(url.pathname, `/api/chat-webhooks/${fixture.publicId}/slack`); + const handled = await handleTenantSlackForm(req, res, { instanceId: fixture.instanceId, publicId: fixture.publicId, owner: broker.slackForms, ingress: broker.slackFormIngress }); + if (handled.handled) return; + if (!handled.consumedBody) { + const event = await handleTenantSlackEvent(req, res, { instanceId: fixture.instanceId, publicId: fixture.publicId, ingress: broker.slackEvents }); + if (event.handled) return; + handled.consumedBody = event.consumedBody; + } + assert.ok(tenantOrigin); + const upstream = await fetch(`${tenantOrigin}${url.pathname}`, { method: "POST", headers: { "content-type": req.headers["content-type"] ?? "", "x-slack-request-timestamp": req.headers["x-slack-request-timestamp"] ?? "", "x-slack-signature": req.headers["x-slack-signature"] ?? "" }, body: handled.consumedBody ?? await read(req), redirect: "error" }); + res.writeHead(upstream.status, { "content-type": upstream.headers.get("content-type") ?? "text/plain", connection: "close" }); + res.end(Buffer.from(await upstream.arrayBuffer())); +})().catch(error => { + const pathname = new URL(req.url ?? "/", fixture.origin ?? "http://127.0.0.1").pathname; + const knownPaths = ["auth.test", "users.info", "conversations.info", "conversations.list", "reactions.add", "reactions.remove", "chat.update", "chat.postEphemeral", "chat.postMessage", "views.open", "views.update"].map(method => `/fixture/slack/${method}`); + const path = knownPaths.includes(pathname) ? pathname : pathname.startsWith("/v1/connector/") ? "/v1/connector/operation" : "/tenant-webhook"; + const category = error?.message === "fixture_method_contract" ? "fixture_method_contract" : error?.code === "ERR_ASSERTION" ? "fixture_assertion" : "fixture_boundary_error"; + const wire = wireRequests.get(req); + console.error(JSON.stringify({ event: "slack-form-fixture-boundary-failed", category, + method: req.method === "GET" || req.method === "POST" ? req.method : "OTHER", path, + providerMethod: wire?.method ?? "not-classified", providerMethodDigest: wire?.methodDigest ?? null, status: res.headersSent ? res.statusCode : 500, + responseAlreadyStarted: res.headersSent, ownedProviderId: wire?.ownedProviderId ?? null, + sourceFrames: wireSourceFrames(error) })); + failures.push(String(error?.stack ?? error)); if (!res.headersSent) json(res, 500, { error: "fixture_boundary_failed" }); else res.destroy(); +}); }); +async function shutdown() { + if (stopping) return; stopping = true; + const errors = []; + try { await worker?.stop(); } catch (error) { errors.push(error); } + try { if (server.listening) { server.close(); server.closeAllConnections(); await once(server, "close"); } } catch (error) { errors.push(error); } + try { await registry.close(); } catch (error) { errors.push(error); } + if (errors.length) { console.error(new AggregateError(errors, "Cloud journey cleanup failed")); process.exitCode = 1; } +} +try { + await registry.applyMigrations(path.join(cloudRoot, "migrations"), 30_000); await registry.hydrate(); + await new CloudStackProvisioner({ provider: new FakeCloudStackProvider(), registry }).provisionDevelopmentStack({ + idempotencyKey: fixture.instanceId, accountGroup: { id: `account-${fixture.instanceId}`, slug: fixture.instanceId, displayName: "Form journey" }, owner: { id: fixture.actorId, email: `${fixture.actorId}@example.test` }, + stack: { stackId: fixture.instanceId, accountGroupId: `account-${fixture.instanceId}`, stackSlug: fixture.instanceId, displayName: "Form journey", kind: "demo", primaryHost: "tenant.example", region: "us-east-1" }, + versionPlan: { appVersion: "fixture", schemaVersion: "fixture", rolloutTrack: "stable" }, sleepPolicy: { desiredState: "awake" }, primaryCompany: { paperclipCompanyId: fixture.companyId, displayName: "Form journey", defaultBudgetCents: 0 }, + }); + await store.putCloudStackInstance({ id: fixture.instanceId, kind: "cloud_stack", companyStackId: fixture.instanceId, environment: "test", ownerUserId: fixture.actorId, + signPublicKey: signing.publicKey.export({ format: "jwk" }).x, sealPublicKey: sealing.publicKey.export({ format: "jwk" }).x, + origins: ["https://tenant.example"], activeOrigin: "https://tenant.example", label: "Form journey", status: "active", createdAt: new Date(), updatedAt: new Date() }); + server.listen(0, "127.0.0.1"); await once(server, "listening"); + const address = server.address(); assert.ok(address && typeof address !== "string"); fixture.origin = `http://127.0.0.1:${address.port}`; + broker = new ConnectorBroker({ publicOrigin: fixture.origin, environment: "test", store, providers: providerRegistryFromEnvironment({}, "test"), secrets: { resolve: async () => undefined }, + formReceiptAuthority: signer, + authenticate: async () => ({ globalUserId: fixture.actorId }), slackEvents: { secrets, secretRefPrefix: "secret:///test/form-journey" }, eventWakeEnrollmentEnabled: true, eventWakeDeliveryEnabled: true, + wakeCloudStack: async () => { if (faults.dropImmediateHint) { faults.dropImmediateHint = false; droppedHints++; return; } await hint(); }, + fetchImplementation: async (input, init) => { const url = new URL(input instanceof Request ? input.url : String(input)); assert.equal(url.origin, "https://slack.com"); assert.equal(url.pathname, "/api/views.open"); return fetch(`${fixture.origin}/fixture/slack/views.open`, init); }, + }); + await restartWaker(); + await writeFile(stateFile, JSON.stringify(fixture), { mode: 0o600, flag: "wx" }); + console.log(JSON.stringify({ event: "slack-form-host-ready", pid: process.pid, node: process.version, origin: fixture.origin, poolMax: 2 })); + process.once("SIGTERM", () => { void shutdown(); }); + process.once("SIGINT", () => { void shutdown(); }); +} catch (error) { console.error(error); process.exitCode = 1; await shutdown(); } diff --git a/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs b/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs new file mode 100644 index 00000000000..6d29e48deff --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/normal-scoped-useful-final.mjs @@ -0,0 +1,24 @@ +// Controlled ordinary run-token acceptance helper; normal run token only. +const peer = new URL(process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN ?? "invalid:"); +const base = new URL(process.env.PAPERCLIP_API_URL ?? "invalid:"); +if (peer.protocol !== "http:" || peer.hostname !== "127.0.0.1" || peer.origin !== base.origin || peer.pathname !== "/") throw Error("unexpected scoped agent API peer"); +if (process.env.SLACK_GATE_GUARD_PID !== String(process.pid)) throw Error("missing actual process guard"); +const runId=process.env.PAPERCLIP_RUN_ID, agentId=process.env.PAPERCLIP_AGENT_ID, companyId=process.env.PAPERCLIP_COMPANY_ID, token=process.env.PAPERCLIP_API_KEY; +if (!runId || !agentId || !companyId || !token) throw Error("missing normal run identity"); +const headers={authorization:`Bearer ${token}`,"x-paperclip-run-id":runId,"content-type":"application/json"}; +async function request(path, method="GET", body) { const response=await fetch(new URL(path,base),{method,headers,...(body===undefined?{}:{body:JSON.stringify(body)})});if(!response.ok)throw Error(`normal agent API refused ${method} ${response.status}`);return response.json(); } +const run=await request(`/api/heartbeat-runs/${encodeURIComponent(runId)}`); +if(run.id!==runId || run.companyId!==companyId || run.agentId!==agentId || typeof run.contextSnapshot?.issueId!=="string")throw Error("run scope changed"); +const issueId=run.contextSnapshot.issueId, route=`/api/issues/${encodeURIComponent(issueId)}`; +const issue=await request(route);if(issue.companyId!==companyId || issue.assigneeAgentId!==agentId || !["todo","in_progress"].includes(issue.status))throw Error("task authority changed"); +await request(`${route}/checkout`,"POST",{agentId,expectedStatuses:["todo","in_progress"]}); +const body=`Scoped Slack useful reply for run ${runId}`; +const comment=await request(`${route}/comments`,"POST",{body}); +if(comment.issueId!==issueId || comment.createdByRunId!==runId || comment.authorAgentId!==agentId || comment.body!==body)throw Error("useful result attribution changed"); +const completed=await request(route,"PATCH",{status:"done"}); +if(completed.id!==issueId || completed.companyId!==companyId || completed.status!=="done")throw Error("normal completion missing"); +console.log(JSON.stringify({event:"scoped-useful-turn-completed",runId,issueId,commentId:comment.id})); +// Actual Codex CLI wire format, parsed by the normal adapter. The text is +// exactly the already-authorized run comment; normal presentation can reuse it. +console.log(JSON.stringify({type:"item.completed",item:{id:`controlled-final-${runId}`,type:"agent_message",text:body}})); +console.log(JSON.stringify({type:"turn.completed",usage:{input_tokens:0,cached_input_tokens:0,output_tokens:0}})); diff --git a/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs b/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs new file mode 100644 index 00000000000..d8bd56d3363 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/quiet-prewire.cjs @@ -0,0 +1,153 @@ +const key = Symbol.for('paperclip.slack.fixture.prewire'); +if (!process[key]) { + const net = require('node:net'); + const tls = require('node:tls'); + const http = require('node:http'); + const https = require('node:https'); + const fs = require('node:fs'); + const loopback = new Set(['127.0.0.1', 'localhost', '::1', '[::1]']); + const listeners = new Set(); + const pgTargets = new Set(); + const pgPorts = new Set([5432, 55448]); + let peer; + if (process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN) { + const value = new URL(process.env.SLACK_FORM_FIXTURE_PEER_ORIGIN); + if (value.protocol !== 'http:' || value.hostname !== '127.0.0.1' || !value.port + || value.pathname !== '/' || value.search || value.hash || value.username || value.password + || pgPorts.has(Number(value.port))) throw new Error('Invalid exact Slack form fixture peer'); + peer = `127.0.0.1:${Number(value.port)}`; + } + for (const name of ['PAPERCLIP_SLACK_TEST_DATABASE_URL', 'SLACK_EVENT_TEST_DATABASE_URL']) { + if (!process.env[name]) continue; + let url; + try { url = new URL(process.env[name]); } catch { throw new Error('Invalid Slack fixture database target'); } + if (!['postgres:', 'postgresql:'].includes(url.protocol) || !loopback.has(url.hostname) + || !/^[a-zA-Z0-9_]+_test$/.test(decodeURIComponent(url.pathname.slice(1))) + || ['options', 'database', 'dbname'].some(k => url.searchParams.has(k))) throw new Error('Invalid Slack fixture database target'); + const port = Number(url.port || 5432); + pgPorts.add(port); + pgTargets.add(`${url.hostname}:${port}`); + } + const sockets = new Set(); + const observed = new WeakSet(); + const attempts = []; + let peakPgSockets = 0; + // Passive scalar counters only: no packet contents or authority changes. + let pgConnectAttempts = 0, pgConnects = 0, pgCloses = 0, pgWriteCalls = 0; + let outboundRequests = 0, nonPgConnectAttempts = 0; + let activeWindow = null; + const closedWaiters = new Set(); + const record = value => { + if (process.env.SLACK_GATE_PROCESS_LOG) fs.appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify(value) + '\n'); + }; + const fail = (host, port) => { + attempts.push({ host: String(host), port: Number(port) || null }); + process.exitCode = 98; + throw new Error('Slack fixture blocked an unowned network target before transport'); + }; + const check = (host = 'localhost', port, protocol) => { + host = String(host).toLowerCase(); + if (!loopback.has(host)) fail(host, port); + if (protocol && !['http:', 'https:'].includes(protocol)) fail(host, port); + if (port && pgPorts.has(Number(port)) && !pgTargets.has(`${host}:${Number(port)}`)) fail(host, port); + if (port && !pgTargets.has(`${host}:${Number(port)}`) && !listeners.has(Number(port)) && `${host}:${Number(port)}` !== peer) fail(host, port); + }; + const checkArgs = args => { + const first = args[0]; + if (first instanceof URL || typeof first === 'string' && /^https?:/.test(first)) { + const url = first instanceof URL ? first : new URL(first); + check(url.hostname, url.port || (url.protocol === 'https:' ? 443 : 80), url.protocol); + } else if (first && typeof first === 'object') { + if (first.path && !first.port && !first.host && !first.hostname) fail('unix-socket', 0); + check(first.hostname || first.host, first.port, first.protocol); + } else if (typeof first === 'number') check(typeof args[1] === 'string' ? args[1] : undefined, first); + else fail('unknown-target', 0); + }; + const listen = net.Server.prototype.listen; + net.Server.prototype.listen = function (...args) { + this.once('listening', () => { + const address = this.address(); + if (address && typeof address === 'object' && loopback.has(address.address)) { + listeners.add(address.port); + this.once('close', () => listeners.delete(address.port)); + } + }); + return Reflect.apply(listen, this, args); + }; + for (const module of [http, https]) for (const name of ['request', 'get']) { + const original = module[name]; + module[name] = function (...args) { checkArgs(args); outboundRequests += 1; return Reflect.apply(original, this, args); }; + } + const connect = net.Socket.prototype.connect; + net.Socket.prototype.connect = function (...args) { + const normalized = Array.isArray(args[0]) ? args[0] : args; + checkArgs(normalized); + const first = normalized[0]; + const port = typeof first === 'object' ? first.port : first; + if (pgPorts.has(Number(port))) pgConnectAttempts += 1; + if (pgPorts.has(Number(port)) && !observed.has(this)) { + observed.add(this); + this.once('connect', () => { + sockets.add(this); pgConnects += 1; + peakPgSockets = Math.max(peakPgSockets, sockets.size); + if (activeWindow) activeWindow.peakPgSockets = Math.max(activeWindow.peakPgSockets, sockets.size); + }); + this.once('close', () => { + if (sockets.delete(this)) pgCloses += 1; + if (!sockets.size) for (const settled of [...closedWaiters]) settled(); + }); + } + if (!pgPorts.has(Number(port))) nonPgConnectAttempts += 1; + return Reflect.apply(connect, this, args); + }; + const write = net.Socket.prototype.write; + net.Socket.prototype.write = function (...args) { + if (observed.has(this)) pgWriteCalls += 1; + return Reflect.apply(write, this, args); + }; + const tlsConnect = tls.connect; + tls.connect = function (...args) { + const first = args[0]; + const host = first && typeof first === 'object' ? first.hostname || first.host || 'localhost' : typeof args[1] === 'string' ? args[1] : 'localhost'; + const port = first && typeof first === 'object' ? first.port : first; + if (`${host}:${Number(port)}` === peer) fail(host, port); + checkArgs(args); return Reflect.apply(tlsConnect, this, args); + }; + const fetch = globalThis.fetch; + if (fetch) globalThis.fetch = function (input, init) { + checkArgs([input instanceof Request ? input.url : input]); + outboundRequests += 1; + return fetch(input, init); + }; + const wireSnapshot = () => ({ pid: process.pid, observedAt: Date.now(), pgConnectAttempts, pgConnects, pgCloses, + pgWriteCalls, outboundRequests, nonPgConnectAttempts, livePgSockets: sockets.size }); + const waitForPgSocketsClosed = timeoutMs => new Promise((resolve, reject) => { + if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) { reject(new Error('Original settlement budget exhausted')); return; } + if (!sockets.size) { resolve(); return; } + let timer; + const settled = () => { if (sockets.size) return; clearTimeout(timer); closedWaiters.delete(settled); resolve(); }; + closedWaiters.add(settled); + timer = setTimeout(() => { closedWaiters.delete(settled); reject(new Error('Used PG sockets did not close within original settlement budget')); }, timeoutMs); + }); + const beginWireWindow = () => { + if (activeWindow || sockets.size) throw new Error('Quiet wire window requires actual PG closure'); + activeWindow = { before: wireSnapshot(), peakPgSockets: sockets.size }; + return activeWindow.before; + }; + const endWireWindow = () => { + if (!activeWindow) throw new Error('Missing quiet wire window'); + const after = wireSnapshot(), before = activeWindow.before; + const result = { before, after, peakPgSockets: activeWindow.peakPgSockets, + pgWriteCalls: after.pgWriteCalls - before.pgWriteCalls, pgConnectAttempts: after.pgConnectAttempts - before.pgConnectAttempts, + pgConnects: after.pgConnects - before.pgConnects, pgCloses: after.pgCloses - before.pgCloses, + outboundRequests: after.outboundRequests - before.outboundRequests, nonPgConnectAttempts: after.nonPgConnectAttempts - before.nonPgConnectAttempts }; + activeWindow = null; + return result; + }; + const stats = () => ({ pid: process.pid, peakPgSockets, remainingPgSockets: sockets.size, unexpectedAttempts: attempts.slice(), wire: wireSnapshot() }); + process[key] = Object.freeze({ pid: process.pid, stats, wireSnapshot, waitForPgSocketsClosed, beginWireWindow, endWireWindow }); + globalThis[key] = process[key]; + process.env.SLACK_GATE_GUARD_PID = String(process.pid); + record({ phase: 'repository-preload', pid: process.pid }); + process.on('exit', () => record({ phase: 'exit', ...stats(), exitCode: process.exitCode || 0 })); +} diff --git a/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs b/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs new file mode 100644 index 00000000000..b58345a11fd --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/read-owned-scopes.mjs @@ -0,0 +1,8 @@ +import assert from "node:assert/strict"; +import { pathToFileURL } from "node:url"; +assert.equal(process.versions.node.split(".")[0], "24"); +assert.equal(process.argv.length, 3); +const { SLACK_CHAT_BOT_SCOPES } = await import(pathToFileURL(process.argv[2])); +assert.ok(Array.isArray(SLACK_CHAT_BOT_SCOPES) && SLACK_CHAT_BOT_SCOPES.length > 0); +assert.ok(SLACK_CHAT_BOT_SCOPES.every(value => typeof value === "string" && (value === "commands" || /^[a-z_]+:[a-z_]+$/.test(value)))); +console.log(JSON.stringify({ scopes: SLACK_CHAT_BOT_SCOPES, pid: process.pid, node: process.version })); diff --git a/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs new file mode 100644 index 00000000000..3fdb65b17c5 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet-app.mjs @@ -0,0 +1,44 @@ +import { spawn } from "node:child_process"; +import { createRequire } from "node:module"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +if (!process.version.startsWith("v24.")) throw new Error("Slack form journey requires the current Node24 executable"); +if (process.argv.length !== 2) throw new Error("Slack form journey has one closed complete case and accepts no selectors"); +const value = process.env.PAPERCLIP_SLACK_TEST_DATABASE_URL; +const cloudValue = process.env.SLACK_EVENT_TEST_DATABASE_URL; +const fixture = new URL(value ?? "invalid:"), cloud = new URL(cloudValue ?? "invalid:"), admin = new URL(process.env.PAPERCLIP_TEST_POSTGRES_ADMIN_URL ?? "invalid:"); +for (const url of [fixture, cloud, admin]) if (!["postgres:", "postgresql:"].includes(url.protocol) || url.hostname !== "127.0.0.1" || !url.port || url.search || url.hash) throw new Error("Explicit owned PG17 fixture/admin URLs required"); +if (fixture.protocol !== cloud.protocol || fixture.protocol !== admin.protocol || admin.pathname !== "/postgres" + || !/^\/sleep2_slack_form_[a-z0-9_]+_app_test$/.test(fixture.pathname) || cloud.pathname !== fixture.pathname.replace(/_app_test$/, "_cloud_test")) throw new Error("Distinct same-grant form fixture pair required"); +const stateFile = process.env.SLACK_FORM_JOURNEY_STATE_FILE; +if (!stateFile || !path.isAbsolute(stateFile)) throw new Error("Owned private host state file required"); +const state = JSON.parse(readFileSync(stateFile, "utf8")); +const peer = new URL(state.origin), tenant = new URL(state.tenantOrigin); +for (const url of [peer, tenant]) if (url.protocol !== "http:" || url.hostname !== "127.0.0.1" || !url.port || ["55448", "5432"].includes(url.port) || url.pathname !== "/" || url.search || url.hash || url.username || url.password) throw new Error("Closed loopback HTTP peer origin required"); +if (peer.origin === tenant.origin) throw new Error("Cloud and tenant must have distinct owned listeners"); +const log = process.env.SLACK_GATE_PROCESS_LOG; +if (!log || !path.isAbsolute(log) || existsSync(log)) throw new Error("Fresh absolute guard log required"); +const packet = path.dirname(fileURLToPath(import.meta.url)); +const server = path.resolve(packet, "../../../.."); +const mode = process.env.SLACK_FORM_SHORT_GATE; +if (mode !== "callback") throw new Error("One existing closed HTTP mode required"); +const entry = path.join(server, "src/__tests__/slack-enabled-quiet.postgres.test.ts"); +const require = createRequire(path.join(server,"package.json")); +const vitest = path.join(path.dirname(require.resolve("vitest/package.json")), "vitest.mjs"); +const guard = path.join(packet, "quiet-prewire.cjs"); +const home = mkdtempSync(path.join(tmpdir(), "slack-form-journey-")); +const env = { PATH: process.env.PATH, LANG: process.env.LANG, HOME: home, TMPDIR: home, PAPERCLIP_HOME: home, + NODE_ENV: "test", NODE_OPTIONS: "", SLACK_GATE_EXPECT_PRELOAD: "1", PAPERCLIP_SLACK_TEST_DATABASE_URL: value, + SLACK_EVENT_TEST_DATABASE_URL: cloudValue, PAPERCLIP_TEST_POSTGRES_ADMIN_URL: process.env.PAPERCLIP_TEST_POSTGRES_ADMIN_URL, + SLACK_GATE_PROCESS_LOG: log, SLACK_FORM_JOURNEY_STATE_FILE: stateFile, SLACK_FORM_FIXTURE_PEER_ORIGIN: peer.origin, SLACK_FORM_SHORT_GATE: process.env.SLACK_FORM_SHORT_GATE, SLACK_PUBLICATION_REPORT: process.env.SLACK_PUBLICATION_REPORT }; +if (env.SLACK_FORM_SHORT_GATE !== "callback") throw new Error("One closed short gate required"); +const config = path.join(home,"short-gate.config.ts"); +writeFileSync(config, `import base from ${JSON.stringify(path.join(server,"vitest.config.ts"))};\nexport default {...base,root:${JSON.stringify(server)},test:{...base.test,include:[${JSON.stringify(entry)}],exclude:[],maxWorkers:1,fileParallelism:false}};\n`); +const child = spawn(process.execPath, ["--require", guard, vitest, "run", "--config", config, + "--maxWorkers=1", "--no-file-parallelism", "--reporter=default", "--reporter=json", "--outputFile=" + process.env.SLACK_PUBLICATION_REPORT, `--execArgv=--require=${guard}`], { cwd: server, env, stdio: "inherit" }); +for (const signal of ["SIGINT", "SIGTERM"]) process.on(signal, () => child.kill(signal)); +child.on("error", error => { console.error(error); process.exitCode = 1; }); +child.on("close", (code, signal) => { process.exitCode = signal ? 1 : code ?? 1; if (!process.exitCode) rmSync(home, { recursive: true, force: true }); }); diff --git a/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py new file mode 100644 index 00000000000..6447f4477c9 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/run-enabled-quiet.py @@ -0,0 +1,216 @@ +import pathlib, json, hashlib, subprocess, os, sys, time, signal, shutil, tempfile, secrets, socket, re, urllib.request, urllib.parse, traceback +O=pathlib.Path(__file__).resolve().parent +A=O.parents[4] +D=pathlib.Path(os.environ['PAPERCLIP_CLOUD_REPO_ROOT']).resolve() +assert D.is_dir() and (D/'package.json').is_file() +MODE='callback' +assert len(sys.argv)==2, 'One exact whole qualification; pass a fresh output directory' +OUT=pathlib.Path(sys.argv[1]).resolve() +assert not OUT.exists(), 'Fresh output directory required' +OUT.mkdir(parents=True) +sha=lambda p:hashlib.sha256(pathlib.Path(p).read_bytes()).hexdigest() +F={'node24':str(pathlib.Path(os.environ['PAPERCLIP_SLACK_NODE24']).resolve()),'node22':str(pathlib.Path(os.environ['PAPERCLIP_SLACK_NODE22']).resolve())} +for key in ['node24','node22']:assert pathlib.Path(F[key]).is_file();F[key+'SHA']=sha(F[key]) +# Fail closed before git metadata, Node children, or any database owner. +app_url=os.environ['PAPERCLIP_SLACK_TEST_DATABASE_URL'];cloud_url=os.environ['SLACK_EVENT_TEST_DATABASE_URL'];admin_url=os.environ['PAPERCLIP_TEST_POSTGRES_ADMIN_URL'] +ap=urllib.parse.urlparse(app_url);cp=urllib.parse.urlparse(cloud_url);ad=urllib.parse.urlparse(admin_url) +assert ap.port is not None and 1 <= ap.port <= 65535, 'Explicit valid fixture port required before child entry' +for u in [ap,cp,ad]:assert u.scheme in ('postgres','postgresql') and u.hostname=='127.0.0.1' and u.port==ap.port and not u.query and not u.fragment +assert ap.scheme==cp.scheme==ad.scheme and ad.path=='/postgres' +assert re.fullmatch('/sleep2_slack_form_[a-z0-9_]+_app_test',ap.path) and cp.path==ap.path.replace('_app_test','_cloud_test') +database_names=[ap.path[1:],cp.path[1:]] +# Snapshot only current local sources/helpers, no committed run inventory. +inputFiles=[p for p in O.iterdir()if p.is_file()] +inputFiles.append(A/'server/src/__tests__/slack-enabled-quiet.postgres.test.ts') +sourcePins={str(p):sha(p)for p in inputFiles} +sourceHeads={} +for root in [A,D]: + sourceHeads[str(root)]=subprocess.check_output(['git','rev-parse','HEAD'],cwd=root,text=True).strip() + names=subprocess.check_output(['git','ls-files','-z'],cwd=root).decode().split('\0') + for name in names: + p=root/name + if name and p.is_file():sourcePins[str(p)]=sha(p) +# Exact installed workspace/tool setup is supplied by normal producer commands; +# no foreign outputs are copied or generated by this runtime-only helper. + +def authority(): + return {'scope':'Explicit local *_test fixture URLs/selected own checkouts; no runtime control-plane privileges','sourceHeads':sourceHeads} +def pins(): + for p,h in sourcePins.items():assert sha(p)==h, 'Qualification source changed' + for key in ['node24','node22']:assert sha(F[key])==F[key+'SHA'] +children=[]; outcomes=[]; cleanup_errors=[]; pid_samples=[] +PRIVATE=None +helper_pids=set(); helper_groups=set(); closed_groups=set(); metadata_probes=[] + +def write(name,value): + with (OUT/name).open('x') as target:json.dump(value,target,indent=2);target.write('\n') +def events(p): + if not pathlib.Path(p).exists():return [] + return [json.loads(line) for line in pathlib.Path(p).read_text().splitlines() if line.strip()] +def closed_env(label,peer=None,databases=False): + home=PRIVATE/label;home.mkdir();(home/'tmp').mkdir();(home/'cache').mkdir() + env={'PATH':str(pathlib.Path(F['node24']).parent)+':/usr/bin:/bin:/usr/sbin:/sbin','LANG':'en_US.UTF-8','HOME':str(home),'TMPDIR':str(home/'tmp'),'XDG_CACHE_HOME':str(home/'cache'),'PAPERCLIP_HOME':str(home),'NODE_OPTIONS':'','NODE_ENV':'test','CI':'true','FORCE_COLOR':'0','SLACK_GATE_PROCESS_LOG':str(OUT/(label+'-guard.jsonl'))} + if peer:env['SLACK_FORM_FIXTURE_PEER_ORIGIN']=peer + if databases:env.update({'PAPERCLIP_SLACK_TEST_DATABASE_URL':app_url,'SLACK_EVENT_TEST_DATABASE_URL':cloud_url,'PAPERCLIP_TEST_POSTGRES_ADMIN_URL':admin_url}) + return env +def spawn(label,argv,env): + auth=authority() + for node in [F['node24'],F['node22']]:assert sha(node)==F['node24SHA' if node==F['node24'] else 'node22SHA'] + log=(OUT/(label+'.log')).open('xb') + process=subprocess.Popen(argv,cwd=A/'server',env=env,stdout=log,stderr=subprocess.STDOUT,start_new_session=True) + row={'label':label,'pid':process.pid,'pgid':process.pid,'argv':argv,'envNames':sorted(env),'authority':auth,'started':time.time()} + write(label+'-started.json',row);children.append((process,log,row));print(json.dumps({'started':label,'pid':process.pid,'pgid':process.pid}),flush=True) + return process + +def group_exists(pgid): + try:os.killpg(pgid,0);return True + except ProcessLookupError:return False + +def sample_owned_groups(): + groups={row['pgid'] for _,_,row in children if row['pgid'] not in closed_groups}|(helper_groups-closed_groups) + p=subprocess.Popen(['/bin/ps','-axo','pid=,ppid=,pgid='],text=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True) + probe={'pid':p.pid,'pgid':p.pid,'started':time.time()};metadata_probes.append((p,probe)) + stdout,stderr=p.communicate(timeout=5);probe.update(exit=p.returncode,closed=time.time()) + assert p.returncode==0,stderr + assert not group_exists(p.pid),'Metadata probe group remains' + closed_groups.add(p.pid) + rows=[] + for line in stdout.splitlines(): + fields=line.split() + if len(fields)==3 and all(v.isdecimal() for v in fields) and int(fields[2]) in groups:rows.append(dict(zip(['pid','ppid','pgid'],map(int,fields)))) + pid_samples.append({'at':time.time(),'ownedGroupsOnly':rows}) + +def wait(process,label,timeout,expected=0): + start=time.monotonic();seen=set() + while process.poll() is None: + if time.monotonic()-start>=timeout:raise TimeoutError(f'{label} exceeded outer process bound {timeout}s') + if MODE in ('header','callback') and label=='app': + markers=events(OUT/'app-guard.jsonl');helpers={v['pid'] for v in markers if v.get('phase') in ('scoped-useful-process-preload','scoped-useful-version-preload')} + helper_pids.update(helpers);helper_groups.update(helpers) # Normal process adapter runChild(detached:true) creates pgid=pid; sampled metadata below corroborates when live. + current={v['pid'] for v in markers if v.get('phase')=='repository-preload'} + if current-seen:sample_owned_groups();seen|=current + time.sleep(.05) + code=process.wait();outcomes.append({'label':label,'pid':process.pid,'pgid':process.pid,'exit':code,'expectedExit':expected,'seconds':time.monotonic()-start});print(json.dumps(outcomes[-1]),flush=True) + assert code==expected,f'{label}: expected exit {expected}, actual {code}' + return code + +def ready(process,file,timeout=20): + start=time.monotonic() + while not file.exists(): + assert process.poll() is None,'Owner exited before ready; preserve setup log' + if time.monotonic()-start>timeout:raise TimeoutError('Owner ready-file bound exceeded') + time.sleep(.05) + return json.loads(file.read_text()) + + +def stop(process,label,normal_signal=True): + if process.poll() is None: + if normal_signal:process.send_signal(signal.SIGTERM) + process.wait(timeout=20) + assert process.returncode==0,f'{label} normal owner shutdown failed' + row=next(row for child,_,row in children if child is process);outcomes.append({'label':label,'pid':process.pid,'pgid':process.pid,'exit':process.returncode,'expectedExit':0,'seconds':time.time()-row['started'],'normalOwnerStop':True}) + + +def http_control(origin,nonce,path): + assert path=='shutdown' + url=urllib.parse.urlparse(origin);assert url.scheme=='http' and url.hostname=='127.0.0.1' + request=urllib.request.Request(origin+'/__fixture/'+path,data=b'{}',method='POST',headers={'x-fixture-control':nonce,'content-type':'application/json'}) + class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self,*args,**kwargs):return None + opener=urllib.request.build_opener(urllib.request.ProxyHandler({}),NoRedirect()) + with opener.open(request,timeout=15) as response: + body=response.read(2_100_000);assert response.status==200 + return {'status':response.status,'bodySHA256':hashlib.sha256(body).hexdigest(),'bodyBytes':len(body)} + +def journey(): + guard=str(O/'quiet-prewire.cjs') + source=spawn('owned-scopes',[F['node24'],'--require',guard,str(O/'read-owned-scopes.mjs'),str(A/'packages/shared/dist/index.js')],closed_env('scopes')) + wait(source,'owned-scopes',20) + scopes=json.loads((OUT/'owned-scopes.log').read_text().splitlines()[-1])['scopes'];assert scopes and all(s=='commands' or re.fullmatch('[a-z_]+:[a-z_]+',s) for s in scopes) + sock=socket.socket();sock.bind(('127.0.0.1',0));port=sock.getsockname()[1];sock.close();assert port not in (5432,55448) + tenant='http://127.0.0.1:'+str(port) + env=closed_env('cloud',tenant,True);env.update({'PAPERCLIP_CLOUD_REPO_ROOT':str(D),'SLACK_FORM_FIXTURE_SCOPES':','.join(scopes),'SLACK_GATE_PROCESS_LOG':str(OUT/'cloud-guard.jsonl'),'SLACK_PEER_CONTROL_LOG':str(OUT/'cloud-transport.jsonl')}) + host=spawn('cloud',[F['node22'],'--require',str(O/'transport-observer.cjs'),'--require',guard,str(O/'enabled-quiet-host.mjs'),str(PRIVATE/'state.json')],env) + state=ready(host,PRIVATE/'state.json',90) + origin=urllib.parse.urlparse(state['origin']);assert origin.scheme=='http' and origin.hostname=='127.0.0.1' and origin.port not in (port,5432,55448) + assert state['tenantOrigin']==tenant and (PRIVATE/'state.json').stat().st_mode & 0o777==0o600 + write('host-ready.json',{'pid':host.pid,'origin':state['origin'],'tenantOrigin':tenant,'stateSHA256':sha(PRIVATE/'state.json'),'stateMode':'0600','syntheticSecretStateNotArchived':True,'scopes':scopes}) + app_env=closed_env('app',state['origin'],True);app_env.update({'SLACK_FORM_JOURNEY_STATE_FILE':str(PRIVATE/'state.json'),'SLACK_GATE_PROCESS_LOG':str(OUT/'app-guard.jsonl'),'SLACK_FORM_SHORT_GATE':MODE,'SLACK_PUBLICATION_REPORT':str(OUT/'app.report.json')}) + app=spawn('app',[F['node24'],str(O/'run-enabled-quiet-app.mjs')],app_env) + try: + wait(app,'app',600) + report=json.loads((OUT/'app.report.json').read_text());assert report['numTotalTests']==1 and report['numPassedTests']==1 and report['numFailedTests']==0 and report['numPendingTests']==0 and report['numTodoTests']==0,report + assertions=[x for suite in report['testResults']for x in suite['assertionResults']];assert len(assertions)==1 and assertions[0]['status']=='passed',assertions + write('whole-enabled-quiet-GREEN.json',{'total':1,'pass':1,'fail':0,'scope':'Two actual composed enabled-quiet windows around ordinary useful tasks; no physical suspension or whole-area claim'}) + + finally: + if app.poll() is None:raise RuntimeError('App still active; outer cleanup must stop its exact owned group before observer') + write('cloud-shutdown-response.json',http_control(state['origin'],state['controlNonce'],'shutdown')) + stop(host,'cloud',False) + assert not group_exists(app.pid) and not group_exists(host.pid),'All actual owner groups must close before final observer' + app_marks=events(OUT/'app-guard.jsonl') + helpers=[v for v in app_marks if v.get('phase')=='scoped-useful-process-preload'] + assert len(helpers)==2 and len({v['pid'] for v in helpers})==2,'Exactly two actual useful-process guard entries required' + version_helpers=[v for v in app_marks if v.get('phase')=='scoped-useful-version-preload'] + for helper in helpers+version_helpers: + pid=helper['pid'];helper_pids.add(pid);helper_groups.add(pid) + exits=[v for v in app_marks if v.get('phase')=='exit' and v.get('pid')==pid] + assert len(exits)==1 and exits[0]['exitCode']==0 and exits[0]['remainingPgSockets']==0 and exits[0]['unexpectedAttempts']==[],'Actual detached helper EXIT required' + assert not group_exists(pid),'Detached useful-process group remains after normal owner drain' + closed_groups.add(pid) + write('useful-process-closure.json',{'helpers':helpers,'versionProbes':version_helpers,'actualExits':[v for v in app_marks if v.get('phase')=='exit' and v.get('pid') in helper_pids],'expectedDetachedGroups':sorted(helper_groups),'sampledMetadata':pid_samples,'allExpectedGroupsAbsent':True,'scopeLimit':'Normal adapter detached-session pgid=pid plus actual guard entry/EXIT and group-absence checks; short lived descendants may escape OS sampling, not a full OS census.'}) + guard_receipts={} + for name in ['app','cloud']: + marks=events(OUT/(name+'-guard.jsonl'));entries=[v for v in marks if v.get('phase')=='repository-preload'];assert entries,f'{name}: actual before-import marker required' + assert all(not v.get('unexpectedAttempts') for v in marks) + exits=[v for v in marks if v.get('phase')=='exit'];assert all(v.get('remainingPgSockets')==0 for v in exits) + assert any(v['pid']==entries[0]['pid'] for v in exits),f'{name}: actual main guard exit required' + missing=sorted({v['pid'] for v in entries}-{v['pid'] for v in exits}) + if name!='app':assert not missing + guard_receipts[name]={'entries':entries,'exits':exits,'missingWorkerExits':missing} + actual_worker_stats=[] + for line in (OUT/'app.log').read_text().splitlines(): + if '"networkStats"' not in line:continue + try:value=json.loads(line[line.index('{'):]) + except (ValueError,json.JSONDecodeError):continue + if 'networkStats' in value:actual_worker_stats.append(value['networkStats']) + known={v['pid'] for v in guard_receipts['app']['entries']} + assert actual_worker_stats and all(v['pid'] in known and v['remainingPgSockets']==0 and v['unexpectedAttempts']==[] for v in actual_worker_stats) + write('actual-guard-receipts.json',{'owners':guard_receipts,'actualWorkerPostCleanupStats':actual_worker_stats}) + write('journey-result.json',{'status':'one enabled quiet/reactivation control GREEN; no managed JWKS rotation, physical suspension or whole-area claim','outcomes':outcomes,'caseBodyCapSeconds':360,'outerAppSeconds':600,'configuredAppPotential':10,'configuredCloudPool':2,'duringCaseObserver':0,'finalObserver':0,'externalFinalObserverOnlyAfterControllerClosed':True,'actualGuardLogs':['app-guard.jsonl','cloud-guard.jsonl','observer-guard.jsonl'],'missingWorkerExitLimit':'An actual worker preload marker is mandatory. A missing worker exit marker is reported, never replaced with a parent marker. Per-process guard peaks do not establish an exact concurrent aggregate SQL peak.','physicalAndRestartLimit':'Fixture controls wake/routing and restarts delivery owners, not physical platform suspension or whole service processes.'}) + +primary=None +try: + auth=authority();pins() + PRIVATE=pathlib.Path(tempfile.mkdtemp(prefix='paperclip-slack-quiet-'));os.chmod(PRIVATE,0o700) + write('controller-started.json',{'pid':os.getpid(),'pgid':os.getpgrp(),'mode':MODE,'authority':auth,'privateRoot':str(PRIVATE),'sourcePinsSHA':hashlib.sha256(json.dumps(sourcePins,sort_keys=True).encode()).hexdigest(),'started':time.time()}) + journey() +except BaseException as error: + primary={'type':type(error).__name__,'message':str(error),'traceback':traceback.format_exc()};write('primary-failure.json',primary) +finally: + for process,log,row in reversed(children): + try: + if process.poll() is None: + os.killpg(process.pid,signal.SIGTERM) + try:process.wait(timeout=20) + except subprocess.TimeoutExpired:raise RuntimeError('Owned group remains after SIGTERM; retain live cohort and forensics, no hard kill') + log.close() + if group_exists(process.pid):raise RuntimeError('Owned group remains after parent exit; retain for caller identity audit, no historical-group signal') + closed_groups.add(process.pid) + except BaseException as error:cleanup_errors.append({'label':row['label'],'type':type(error).__name__,'message':str(error)}) + for probe,row in metadata_probes: + if probe.poll() is None or group_exists(row['pgid']):cleanup_errors.append({'metadataProbe':row,'error':'Metadata probe remains; retain for caller audit'}) + for pid in sorted(helper_groups): + if group_exists(pid):cleanup_errors.append({'helperPID':pid,'pgid':pid,'error':'Detached helper group remains; retain for identity audit, no blind signal'}) + if PRIVATE: + if any(group_exists(process.pid) for process,_,row in children) or any(group_exists(pid) for pid in helper_groups) or any(probe.poll() is None or group_exists(row['pgid']) for probe,row in metadata_probes): + cleanup_errors.append({'privateRoot':str(PRIVATE),'error':'Retained owned root while an owned process group remains'}) + elif primary or cleanup_errors: + cleanup_errors.append({'privateRoot':str(PRIVATE),'forensicsRetained':True}) + else: + try:shutil.rmtree(PRIVATE) + except BaseException as error:cleanup_errors.append({'privateRoot':str(PRIVATE),'error':str(error)}) + try:pins() + except BaseException as error:cleanup_errors.append({'sourcePinError':str(error)}) + write('controller-cleanup.json',{'controller':os.getpid(),'children':[row for _,_,row in children],'outcomes':outcomes,'pidSamples':pid_samples,'metadataProbes':[row for _,row in metadata_probes],'detachedHelperPIDs':sorted(helper_pids),'detachedHelperGroups':sorted(helper_groups),'closedGroups':sorted(closed_groups),'remainingGroups':[row['pgid'] for process,_,row in children if group_exists(process.pid)],'privateRoot':str(PRIVATE),'privateRemoved':not PRIVATE or not PRIVATE.exists(),'primary':primary,'secondaryCleanup':cleanup_errors,'databaseDisposition':'No DROP, no broad cleanup. Caller retains/audits exact explicit fixtures.'}) +if primary or cleanup_errors:sys.exit(1) diff --git a/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs b/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs new file mode 100644 index 00000000000..31443499919 --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/scoped-cli-prewire.cjs @@ -0,0 +1,17 @@ +// Controlled exact useful-process fixture: host instrumentation, current run token only. +const permitted = new Set([ + "PATH", "HOME", "TMPDIR", "XDG_CACHE_HOME", "LANG", "LC_ALL", "CI", "NODE_ENV", "NODE_OPTIONS", + "PAPERCLIP_API_URL", "PAPERCLIP_RUN_ID", "PAPERCLIP_AGENT_ID", "PAPERCLIP_COMPANY_ID", "PAPERCLIP_API_KEY", + "SLACK_FORM_FIXTURE_PEER_ORIGIN", "SLACK_GATE_PROCESS_LOG", +]); +// The approved controller passes empty NODE_OPTIONS and the exact preload as +// explicit argv. Refuse an earlier unreviewed preload instead of keeping its +// possibly already-installed DB/origin authority after environment reduction. +if (process.env.NODE_OPTIONS) throw Error("Unexpected useful-process preload before authority reduction"); +for (const key of Object.keys(process.env)) if (!permitted.has(key)) delete process.env[key]; +require("./quiet-prewire.cjs"); +if (process.env.SLACK_GATE_GUARD_PID !== String(process.pid)) throw Error("Missing actual useful-process guard"); +require("node:fs").appendFileSync(process.env.SLACK_GATE_PROCESS_LOG, JSON.stringify({ + phase: process.argv.length === 3 && process.argv[2] === "--version" ? "scoped-useful-version-preload" : "scoped-useful-process-preload", pid: process.pid, ppid: process.ppid, + argv: process.argv, execArgv: process.execArgv, envNames: Object.keys(process.env).sort(), +}) + "\n"); diff --git a/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs b/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs new file mode 100644 index 00000000000..35240ded21b --- /dev/null +++ b/server/src/__tests__/helpers/slack-cloud/transport-observer.cjs @@ -0,0 +1,27 @@ +const net = require('node:net'); +const tls = require('node:tls'); +const http = require('node:http'); +const dns = require('node:dns'); +const fs = require('node:fs'); +const key = Symbol.for('paperclip.slack.peer.control.transport'); +if (process[key]) throw new Error('Duplicate passive peer observer'); +let connects = 0, tlsCalls = 0, httpCalls = 0, dnsCalls = 0; +const sockets = new Set(); +const connect = net.Socket.prototype.connect; +net.Socket.prototype.connect = function (...args) { + connects++; + this.once('connect', () => sockets.add(this)); + this.once('close', () => sockets.delete(this)); + return Reflect.apply(connect, this, args); +}; +const secure = tls.connect; +tls.connect = function (...args) { tlsCalls++; return Reflect.apply(secure, this, args); }; +const request = http.request; +http.request = function (...args) { httpCalls++; return Reflect.apply(request, this, args); }; +const lookup = dns.lookup; +dns.lookup = function (...args) { dnsCalls++; return Reflect.apply(lookup, this, args); }; +const stats = () => ({ pid: process.pid, connects, tlsCalls, httpCalls, dnsCalls, sockets: sockets.size }); +process[key] = Object.freeze({ pid: process.pid, stats }); +const record = value => fs.appendFileSync(process.env.SLACK_PEER_CONTROL_LOG, JSON.stringify(value) + '\n'); +record({ phase: 'passive-observer-before-guard', pid: process.pid, node: process.version, argv: process.argv, execArgv: process.execArgv }); +process.on('exit', () => record({ phase: 'passive-observer-exit', ...stats(), exitCode: process.exitCode || 0 })); diff --git a/server/src/__tests__/idle-admission.test.ts b/server/src/__tests__/idle-admission.test.ts index 6ab60153332..55412da846b 100644 --- a/server/src/__tests__/idle-admission.test.ts +++ b/server/src/__tests__/idle-admission.test.ts @@ -34,6 +34,19 @@ describe("idle admission", () => { expect(idleWorkSnapshot().active).toBe(0); }); + it("exempts only the exact body status read from self-accounting and keeps drain mutations tracked", async () => { + const server = app(); + server.post("/api/instance/task-drain/status", (_req, res) => res.json({ active: idleWorkSnapshot().active })); + server.post("/api/instance/task-drain", (_req, res) => res.json({ active: idleWorkSnapshot().active })); + trackIdleRequestHandlers(server); + startTaskDrain({ purpose: "idle", ttlMs: 60_000 }); + expect((await request(server).post("/api/instance/task-drain/status").expect(200)).body.active).toBe(0); + expect((await request(server).post("/api/instance/task-drain").expect(200)).body.active).toBe(1); + await request(server).post("/api/instance/task-drain/status/mutate").expect(503); + await request(server).get("/api/instance/task-drain/status").expect(503); + expect(idleWorkSnapshot().active).toBe(0); + }); + it("counts a nested async handler after it sends a response", async () => { const server = app(); const router = express.Router(); diff --git a/server/src/__tests__/instance-settings-routes.test.ts b/server/src/__tests__/instance-settings-routes.test.ts index cf4803eda84..56b4b211ee1 100644 --- a/server/src/__tests__/instance-settings-routes.test.ts +++ b/server/src/__tests__/instance-settings-routes.test.ts @@ -1,4 +1,5 @@ -import { generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { DEFAULT_JSON_BODY_LIMIT } from "../http/body-limits.js"; +import { createHash, generateKeyPairSync, randomUUID, sign } from "node:crypto"; import express from "express"; import request from "supertest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; @@ -100,7 +101,7 @@ describe("instance settings routes", () => { function createApp(actor: any, pluginWorkers?: PluginWorkerManager, prepareBackup?: () => Promise, verifyControl = false) { const { errorHandler, instanceSettingsRoutes, cloudControlMiddleware } = routeModules.value; const app = express(); - app.use(express.json()); + app.use(express.json({ limit: DEFAULT_JSON_BODY_LIMIT, verify: (req, _res, raw) => { Reflect.set(req, "rawBody", raw); } })); app.use((req, _res, next) => { req.actor = actor; next(); @@ -1190,6 +1191,46 @@ describe("instance settings routes", () => { }); afterEach(() => vi.unstubAllEnvs()); + it("reads the same owner and deadline scope through the real body-bound middleware and production handler", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + const url = `/api/instance/task-drain/status?idleSleepSafety=1&scheduledWake=domain-deadlines-v1&ownerId=${ownerId}`; + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + expect(mockReadIdleSleepSafety).toHaveBeenCalledExactlyOnceWith(mockDb, expect.any(Function), Date.now, + ownerId, expect.any(Function), undefined, undefined, true, true, + { incomingWake: catalog, deadlineCapability: catalog, prepareLocalWork: expect.any(Function) }); + expect(Object.isFrozen(mockReadIdleSleepSafety.mock.calls[0][9].incomingWake)).toBe(true); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(401); + }); + it("rejects body tampering before installing capabilities or consuming the valid request", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + const url = `/api/instance/task-drain/status?idleSleepSafety=1&ownerId=${ownerId}`; + await request(createControlApp(adminActor)).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw + " ").expect(401); + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + }); + it("rejects unsigned and actor-only body authority and cross-route method replay", async () => { + const raw = JSON.stringify({ idleCapabilities: catalog }), url = `/api/instance/task-drain/status?idleSleepSafety=1&ownerId=${ownerId}`; + for (const source of ["session", "cloud_tenant", "local_implicit", "cloud_control"]) { + await request(createControlApp({ ...adminActor, source, idleCapabilities: catalog })).post(url) + .set("content-type", "application/json").send(raw).expect(403); + } + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + const token = assertion({ action: "task-drain:read-body", idleCapabilities: undefined, bodySha256: createHash("sha256").update(raw).digest("hex") }); + await request(createControlApp(adminActor)).get(url).set("x-paperclip-cloud-control", token).expect(400); + await request(createControlApp(adminActor)).post("/api/instance/task-drain").set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(401); + await request(createControlApp(adminActor)).delete("/api/instance/task-drain").set("x-paperclip-cloud-control", token).expect(401); + expect(mockReadIdleSleepSafety).not.toHaveBeenCalled(); + await request(createControlApp({ type: "none" })).post(url).set("content-type", "application/json") + .set("x-paperclip-cloud-control", token).send(raw).expect(200); + }); + it("passes only the immutable signed catalog through real middleware and the production route", async () => { const token = assertion(); const response = await request(createControlApp({ type: "none" })) diff --git a/server/src/__tests__/slack-cloud-forms.test.ts b/server/src/__tests__/slack-cloud-forms.test.ts new file mode 100644 index 00000000000..aafbee6190b --- /dev/null +++ b/server/src/__tests__/slack-cloud-forms.test.ts @@ -0,0 +1,330 @@ +import { isRetryableSlackFormUpdate } from "../services/chat-channels/slack/form-open-callback.js"; +import { slackFormConnectorIdentity, selectSlackFormRetainedConsumer } from "../services/chat-channels/slack/cloud-consumer.js"; +import { createHash, createHmac } from "node:crypto"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { slackFormSubmissionEvidence, withSlackFormSubmissionEvidence, assertSlackFormTerminalAck, lookupSlackFormDelivery, parseSlackFormOperation, type SlackFormBinding, type SlackFormView, type SlackFormOperation } from "../services/chat-channels/slack/cloud-forms.js"; + +import { parseSlackFormProofAcceptance, prepareSlackFormProofAcceptance, projectSlackFormRetainedOwner, type SlackFormTerminalIntent, type SlackFormRetainedRow, openSlackFormSubmissionIntent, prepareSlackFormSubmissionIntent } from "../services/chat-channels/slack/form-session.js"; +import { createLocalEncryptedCipher } from "../secrets/local-encrypted-provider.js"; + +afterEach(() => { vi.unstubAllEnvs(); }); +const hash = (s: string) => createHash("sha256").update(s).digest("hex"); +function fixture() { + const acceptedAt = "2099-01-01T00:00:00.000Z", secret = "0123456789abcdef0123456789abcdef"; + const binding: SlackFormBinding = { version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "a".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: `slack-event:${hash("event")}`, credentialFingerprint: hash("credential") }; + const raw = Buffer.from(new URLSearchParams({ payload: JSON.stringify({ type: "block_actions", trigger_id: "trigger", user: { id: "UTEST" }, team: { id: "TTEST" }, api_app_id: "ATEST" }) }).toString()); + const rawBody = raw.toString("base64"), timestamp = String(Date.parse(acceptedAt) / 1000); + const signature = `v0=${createHmac("sha256", secret).update(`v0:${timestamp}:`).update(raw).digest("hex")}`; + const id = hash("operation"); + const operation: SlackFormOperation = { version: 1, binding, grantId: hash("grant"), id, externalId: `pc-form-${id}`, deliveryId: `slack_form_${id}`, + requestDigest: hash(JSON.stringify([rawBody, timestamp, signature])), triggerDigest: hash("trigger"), userId: "UTEST", acceptedAt, + triggerDeadline: "2099-01-01T00:00:03.000Z", phase: "reserved", view: null, closed: null, acknowledgement: null, rejection: null }; + const packet = { version: 1, instanceId: "instance", environment: "test", operation, acceptedAt, rawBody, timestamp, signature }; + const event = { id: operation.deliveryId, provider: "slack", event: "form_open", installationId: `slack-form-open:${id}`, bindingIds: [`slack-form-open:${id}`], createdAt: acceptedAt, payload: { slackFormOpen: packet } }; + const current = { kind: "current" as const, binding, instanceId: "instance", environment: "test", signingSecret: secret }; + const lookup = vi.fn(async () => ({ version: 1, instanceId: "instance", environment: "test", operation })); + function close(view: SlackFormView): NonNullable { + const body = Buffer.from(new URLSearchParams({ payload: JSON.stringify({ type: "view_closed", api_app_id: "ATEST", team: { id: "TTEST" }, + user: { id: "UTEST" }, view: { id: view.id, hash: view.hash, external_id: view.externalId, app_id: "ATEST" } }) }).toString()); + const rawBody = body.toString("base64"), signature = `v0=${createHmac("sha256", secret).update(`v0:${timestamp}:`).update(body).digest("hex")}`; + return { kind: "closed", view, acceptedAt, requestDigest: hash(JSON.stringify([rawBody, timestamp, signature])), evidence: { + version: 1, kind: "close", submitReceiptId: null, instanceId: "instance", environment: "test", operationId: id, binding, acceptedAt, rawBody, timestamp, signature } }; + } + return { operation, packet, event, current, lookup, close, now: Date.parse(acceptedAt) + 3500 }; +} + +describe("Slack form original evidence and on-demand receipt", () => { + it("pending receipt makes one lookup and retains retry responsibility without SDK or ACK", async () => { + const f = fixture(); + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_receipt_pending", disposition: "retry" }); + expect(f.lookup).toHaveBeenCalledExactlyOnceWith({ binding: f.current.binding, operationId: f.operation.id, deliveryId: f.event.id }); + expect(f.packet.operation.phase).toBe("reserved"); + }); + it("delayed confirmed receipt preserves original acceptance and current generation", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view } }); + const result = await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now); + expect(result.operation.view).toEqual(view); expect(result.operation.acceptedAt).toBe(f.event.createdAt); + expect(result.originalPayload.trigger_id).toBe("trigger"); + }); + it.each(["companyId", "endpointId", "connectionId", "credentialFingerprint", "instanceId"] as const)("foreign %s cannot reach lookup", async key => { + const f = fixture(); + await expect(lookupSlackFormDelivery(f.event, { ...f.current, binding: { ...f.current.binding, [key]: "foreign" } }, f.lookup, f.now)).rejects.toMatchObject({ disposition: "attention" }); + expect(f.lookup).not.toHaveBeenCalled(); + }); + it("original signature failure cannot reset freshness or reach lookup", async () => { + const f = fixture(); f.packet.signature = `v0=${"0".repeat(64)}`; + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_original_authentication_failed" }); + expect(f.lookup).not.toHaveBeenCalled(); + }); + it("lookup successor identity cannot overwrite the original queue operation", async () => { + const f = fixture(); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, requestDigest: hash("successor") } }); + await expect(lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).rejects.toMatchObject({ code: "form_lookup_identity_changed", disposition: "attention" }); + expect(f.packet.operation.requestDigest).not.toBe(hash("successor")); + }); + it("confirmed but already closed is surfaced as closed custody, never an update instruction", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + const closed = f.close(view); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view, closed } }); + expect((await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).operation.closed).toEqual(closed); + }); + it("ambiguous provider outcome remains explicit, never becomes fabricated confirmation", async () => { + const f = fixture(); + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "ambiguous" } }); + const result = await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now); + expect(result.operation.phase).toBe("ambiguous"); expect(result.operation.view).toBeNull(); + }); + it("active or merely closed views cannot ACK before durable owner transfer", () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "confirmed", view }, "11111111-1111-1111-1111-111111111111")).toThrow("form_view_active"); + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "confirmed", view, closed: f.close(view) }, "11111111-1111-1111-1111-111111111111")).toThrow(); + expect(() => assertSlackFormTerminalAck({ ...f.operation, phase: "acknowledged", view, closed: f.close(view), + acknowledgement: { appReceiptId: "11111111-1111-1111-1111-111111111111", acceptedAt: f.operation.acceptedAt } }, "11111111-1111-1111-1111-111111111111")).not.toThrow(); + }); + it("retained custody accepts only sealed verified close settlement after local credentials removed", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view, closed: f.close(view) } }); + const result = await lookupSlackFormDelivery(f.event, { kind: "retained", binding: f.current.binding, instanceId: "instance", environment: "test" }, f.lookup, f.now); + expect(result.authority).toBe("retained"); expect(result.operation.closed?.view.id).toBe("VTEST"); + }); + it("retained unresolved operation cannot authorize a new useful form", async () => { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "confirmed", view } }); + await expect(lookupSlackFormDelivery(f.event, { kind: "retained", binding: f.current.binding, instanceId: "instance", environment: "test" }, f.lookup, f.now)) + .rejects.toMatchObject({ code: "form_retained_resource_unresolved", disposition: "attention" }); + }); + it("definite no-effect is identity-bound and exchanged trigger is never a no-effect proof", async () => { + const f = fixture(), observation = { source: "provider_response", status: 200, body: { ok: false, error: "expired_trigger_id" } }; + const responseDigest = hash(JSON.stringify([f.operation.id, "expired_trigger_id", observation])); + const rejection: NonNullable = { code: "expired_trigger_id", observedAt: f.operation.acceptedAt, responseDigest, + evidence: { version: 1, kind: "rejection", instanceId: "instance", environment: "test", operationId: f.operation.id, binding: f.current.binding, + observedAt: f.operation.acceptedAt, code: "expired_trigger_id", responseDigest, observation } }; + f.lookup.mockResolvedValueOnce({ version: 1, instanceId: "instance", environment: "test", operation: { ...f.operation, phase: "rejected", rejection } }); + expect((await lookupSlackFormDelivery(f.event, f.current, f.lookup, f.now)).operation.rejection?.responseDigest).toBe(responseDigest); + expect(() => parseSlackFormOperation({ ...f.operation, phase: "rejected", rejection: { ...rejection, code: "exchanged_trigger_id" } })).toThrow(); + }); + it("closed parser refuses unknown fields and conflicting view identity", () => { + const f = fixture(); + expect(() => parseSlackFormOperation({ ...f.operation, token: "forbidden" })).toThrow(); + expect(() => parseSlackFormOperation({ ...f.operation, phase: "confirmed", view: { id: "VTEST", hash: "h", externalId: "foreign" } })).toThrow(); + }); +}); + +function submissionRequest(endpointId = "endpoint") { + const f = fixture(), payload = { type: "view_submission", api_app_id: "ATEST", team: { id: "TTEST" }, user: { id: "UTEST" }, + view: { id: "VTEST", hash: "useful-hash", external_id: f.operation.externalId, app_id: "ATEST", callback_id: "opaque-submit", state: { values: { secret: "private-answer-for-terminal-cipher-proof" } } } }; + const raw = new URLSearchParams({ payload: JSON.stringify(payload) }).toString(), timestamp = "4070908800"; + const signature = `v0=${createHmac("sha256", f.current.signingSecret).update(`v0:${timestamp}:`).update(raw).digest("hex")}`; + const request = new Request("https://paperclip.invalid/slack", { method: "POST", headers: { "content-type": "application/x-www-form-urlencoded", + "x-slack-request-timestamp": timestamp, "x-slack-signature": signature }, body: raw }); + return { f, request, raw, endpointId }; +} + +describe("Slack form submission leaf evidence, without SDK/domain authorization claims", () => { + it("preserves original SDK request and scopes concurrent immutable evidence", async () => { + const a = submissionRequest("a"), b = submissionRequest("b"); + const result = await Promise.all([a, b].map(async f => withSlackFormSubmissionEvidence(f.request, f.endpointId, true, async () => { + await Promise.resolve(); const original = slackFormSubmissionEvidence()!; + expect(Object.isFrozen(original)).toBe(true); expect(original.endpointId).toBe(f.endpointId); + expect(Buffer.from(original.rawBody, "base64").toString()).toBe(f.raw); + expect(await f.request.text()).toBe(f.raw); return original.endpointId; + }))); + expect(result).toEqual(["a", "b"]); expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("absent feature does not read request; failed normal callback discards context", async () => { + const f = submissionRequest(), error = new Error("controlled normal authentication refusal"); + await withSlackFormSubmissionEvidence(f.request, "endpoint", false, async () => { expect(slackFormSubmissionEvidence()).toBeUndefined(); }); + expect(f.request.bodyUsed).toBe(false); + await expect(withSlackFormSubmissionEvidence(f.request, "endpoint", true, async () => { throw error; })).rejects.toBe(error); + expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("bounded capture cannot run the callback after excessive input", async () => { + const request = new Request("https://paperclip.invalid/slack", { method: "POST", body: "x".repeat(1048577) }), work = vi.fn(async () => {}); + await expect(withSlackFormSubmissionEvidence(request, "endpoint", true, work)).rejects.toMatchObject({ code: "form_submission_too_large" }); + expect(work).not.toHaveBeenCalled(); expect(slackFormSubmissionEvidence()).toBeUndefined(); + }); + it("existing local cipher protects original evidence and refuses wrong key or foreign custody", async () => { + const f = submissionRequest(); vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "ab".repeat(32)); + const cipher = createLocalEncryptedCipher("read"); + await withSlackFormSubmissionEvidence(f.request, "endpoint", true, async () => { + const original = slackFormSubmissionEvidence()!; + const intent = prepareSlackFormSubmissionIntent({ version: 1, binding: f.f.current.binding, operationId: f.f.operation.id, deliveryId: f.f.event.id, + userId: "UTEST", grantId: f.f.operation.grantId, publicationId: "publication", interactionId: "interaction", appReceiptId: "11111111-1111-4111-8111-111111111111" }, + { id: "VTEST", hash: "useful-hash", externalId: f.f.operation.externalId }, "22222222-2222-4222-8222-222222222222", original, cipher); + expect(JSON.stringify(intent)).not.toContain("private-answer-for-terminal-cipher-proof"); + expect(JSON.stringify(intent)).not.toContain(original.rawBody); + expect(openSlackFormSubmissionIntent(intent, cipher).rawBody).toBe(original.rawBody); + expect(() => openSlackFormSubmissionIntent({ ...intent, binding: { ...intent.binding, companyId: "foreign" } }, cipher)).toThrow("form_submission_cipher_binding_changed"); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "cd".repeat(32)); + const wrong = createLocalEncryptedCipher("read"); + expect(() => openSlackFormSubmissionIntent(intent, wrong)).toThrow("form_submission_cipher_unavailable"); + }); + }); +}); + +function closedProofFixture() { + const f = fixture(), view = { id: "VTEST", hash: "hash1", externalId: f.operation.externalId }, appReceiptId = "11111111-1111-1111-1111-111111111111"; + const intent: SlackFormTerminalIntent = { version: 1, binding: f.current.binding, operationId: f.operation.id, deliveryId: f.operation.deliveryId, + userId: f.operation.userId, grantId: f.operation.grantId, publicationId: "publication", interactionId: "interaction", appReceiptId, view, + outcome: "closed", submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + const operation: SlackFormOperation = { ...f.operation, phase: "terminal_proven", view, closed: f.close(view), + acknowledgement: { appReceiptId, acceptedAt: f.operation.acceptedAt } }; + const row: SlackFormRetainedRow = { id: appReceiptId, companyId: intent.binding.companyId, endpointId: intent.binding.endpointId, kind: "slack_form_session", + providerActionId: `slack-form-session:${intent.operationId}`, status: "cancelled", payload: { binding: intent.binding, operationId: intent.operationId, + userId: intent.userId, grantId: intent.grantId, publicationId: intent.publicationId, interactionId: intent.interactionId }, result: { phase: "closed", terminalIntent: intent } }; + return { f, intent, operation, row }; +} +describe("Slack form two phase terminal acceptance, provider boundaries only", () => { + it("accepts only actual terminal proof for the same durable App receipt without finalizing Cloud ACK", () => { + const f = closedProofFixture(), before = structuredClone(f.operation); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(parseSlackFormProofAcceptance(accepted)).toEqual(accepted); expect(accepted.appReceiptId).toBe(f.row.id); + expect(f.operation).toEqual(before); expect(f.operation.phase).toBe("terminal_proven"); + }); + it("pending, foreign instance, changed receipt and changed terminal intent cannot become proof accepted", () => { + const f = closedProofFixture(); + expect(() => prepareSlackFormProofAcceptance(f.intent, { ...f.operation, phase: "confirmed", acknowledgement: null }, "instance")).toThrow(); + expect(() => prepareSlackFormProofAcceptance(f.intent, f.operation, "foreign")).toThrow(); + expect(() => prepareSlackFormProofAcceptance({ ...f.intent, appReceiptId: "22222222-2222-2222-2222-222222222222" }, f.operation, "instance")).toThrow(); + expect(() => prepareSlackFormProofAcceptance({ ...f.intent, view: { ...f.intent.view!, hash: "foreign" } }, f.operation, "instance")).toThrow(); + }); + it("phase parser refuses a proof without exact terminal evidence or acceptance", () => { + const f = closedProofFixture(); + expect(() => parseSlackFormOperation({ ...f.operation, acknowledgement: null })).toThrow(); + expect(() => parseSlackFormOperation({ ...f.operation, closed: null })).toThrow(); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(() => parseSlackFormProofAcceptance({ ...accepted, outcome: "invented" })).toThrow(); + expect(() => parseSlackFormProofAcceptance({ ...accepted, providerToken: "forbidden" })).toThrow(); + }); + it("retained accepted history supplies a stable event identity without making completed history polling work", () => { + const f = closedProofFixture(); + const pending = projectSlackFormRetainedOwner(f.row, "instance"); expect(pending?.requiresDelivery).toBe(true); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const retained = projectSlackFormRetainedOwner({ ...f.row, result: { ...f.row.result, cloudProofAcceptance: accepted } }, "instance"); + expect(retained).toMatchObject({ subject: "github-bot-events", companyId: "company", requiresDelivery: false }); + expect(retained?.binding).toEqual(f.intent.binding); + }); + it("foreign malformed history is isolated before parsing and changed current receipt is attention", () => { + const f = closedProofFixture(); + expect(projectSlackFormRetainedOwner({ ...f.row, payload: { binding: { instanceId: "foreign", malformed: true } } }, "instance")).toBeNull(); + const accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + expect(() => projectSlackFormRetainedOwner({ ...f.row, companyId: "foreign" }, "instance")).toThrow(); + expect(() => projectSlackFormRetainedOwner({ ...f.row, result: { ...f.row.result, cloudProofAcceptance: { ...accepted, appReceiptId: "22222222-2222-2222-2222-222222222222" } } }, "instance")).toThrow(); + }); + it("definite no effect proof requires the recorded digest and no view or closed input", () => { + const f = closedProofFixture(), observation = { ok: false, error: "expired_trigger_id" }, responseDigest = hash(JSON.stringify([f.operation.id, "expired_trigger_id", observation])); + const operation: SlackFormOperation = { ...f.operation, view: null, closed: null, rejection: { code: "expired_trigger_id", observedAt: f.operation.acceptedAt, + responseDigest, evidence: { version: 1, kind: "rejection", instanceId: "instance", environment: "test", operationId: f.operation.id, binding: f.operation.binding, + observedAt: f.operation.acceptedAt, code: "expired_trigger_id", responseDigest, observation } } }; + const intent: SlackFormTerminalIntent = { ...f.intent, outcome: "no_effect", view: null, rejectionDigest: responseDigest }; + expect(prepareSlackFormProofAcceptance(intent, operation, "instance").outcome).toBe("no_effect"); + expect(() => prepareSlackFormProofAcceptance({ ...intent, rejectionDigest: hash("foreign") }, operation, "instance")).toThrow(); + }); +}); + +it("submitted proof accepts only the encrypted same submission and is idempotent without another encryption", () => { + const f = submissionRequest(); vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "ab".repeat(32)); + const cipher = createLocalEncryptedCipher("read"), encrypt = vi.spyOn(cipher, "encrypt"), base = closedProofFixture(); + const original = { endpointId: "endpoint", acceptedAt: f.f.operation.acceptedAt, rawBody: Buffer.from(f.raw).toString("base64"), + timestamp: f.request.headers.get("x-slack-request-timestamp")!, signature: f.request.headers.get("x-slack-signature")! }; + const view = { id: "VTEST", hash: "useful-hash", externalId: f.f.operation.externalId }, submitReceiptId = "22222222-2222-4222-8222-222222222222"; + const intent = prepareSlackFormSubmissionIntent(base.intent, view, submitReceiptId, original, cipher); + const operation: SlackFormOperation = { ...base.operation, view, closed: { kind: "submitted", view, acceptedAt: original.acceptedAt, + requestDigest: hash(JSON.stringify([original.rawBody, original.timestamp, original.signature])), evidence: { version: 1, kind: "submission", submitReceiptId, + instanceId: "instance", environment: "test", operationId: base.operation.id, binding: base.operation.binding, acceptedAt: original.acceptedAt, + rawBody: original.rawBody, timestamp: original.timestamp, signature: original.signature } } }; + const accepted = prepareSlackFormProofAcceptance(intent, operation, "instance", cipher); + expect(accepted.outcome).toBe("submitted"); expect(prepareSlackFormProofAcceptance(intent, operation, "instance", cipher)).toEqual(accepted); + expect(encrypt).toHaveBeenCalledTimes(1); + expect(() => prepareSlackFormProofAcceptance(intent, { ...operation, closed: { ...operation.closed!, requestDigest: hash("foreign") } }, "instance", cipher)).toThrow(); +}); + + +describe("retained form consumer identity", () => { + it("old connector mocks and absent Cloud do not acquire form authority", () => { + expect(slackFormConnectorIdentity(null)).toBeNull(); + expect(slackFormConnectorIdentity({ leaseEvents: vi.fn() })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => ({ instanceId: "instance", environment: "foreign" }) })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => { throw new Error("unsupported"); } })).toBeNull(); + expect(slackFormConnectorIdentity({ instanceIdentity: () => ({ instanceId: "instance", environment: "test" }) })).toEqual({ instanceId: "instance", environment: "test" }); + }); + it("malformed owned history cannot starve a valid proof-accepted ACK consumer", () => { + const f = closedProofFixture(), accepted = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const good = { ...f.row, payload: { ...f.row.payload, environment: "test" }, result: { ...f.row.result, cloudProofAcceptance: accepted } }; + const malformed = { ...f.row, id: "malformed", payload: { binding: { instanceId: "instance", malformed: true }, environment: "test" } }; + const result = selectSlackFormRetainedConsumer([malformed, good], { instanceId: "instance", environment: "test" }); + expect(result.attention).toEqual([{ actionId: "malformed", code: "form_retained_owner_invalid" }]); + expect(result.owner).toMatchObject({ subject: "github-bot-events", companyId: "company", requiresDelivery: false }); + expect(result.owner?.binding).toEqual(f.intent.binding); + }); + it("a foreign or wrong-environment row cannot supply terminal lease authority", () => { + const f = closedProofFixture(); + const foreign = { ...f.row, payload: { binding: { instanceId: "foreign", malformed: true } } }; + expect(selectSlackFormRetainedConsumer([foreign], { instanceId: "instance", environment: "test" })).toEqual({ owner: null, attention: [] }); + const result = selectSlackFormRetainedConsumer([{ ...f.row, payload: { ...f.row.payload, environment: "production" } }], { instanceId: "instance", environment: "test" }); + expect(result.owner).toBeNull(); expect(result.attention).toHaveLength(1); + }); + it("lost enrollment response retains public custody without inventing a credential binding", () => { + const f = closedProofFixture(), { eventBindingId, credentialFingerprint: _, ...identity } = f.intent.binding; + const row: SlackFormRetainedRow = { ...f.row, kind: "slack_form_owner", payload: { enrollment: { ...identity, bindingId: eventBindingId }, subject: "actor", environment: "test" }, result: { phase: "staged" } }; + const result = selectSlackFormRetainedConsumer([row], { instanceId: "instance", environment: "test" }); + expect(result.attention).toEqual([]); + expect(result.owner).toEqual({ subject: "github-bot-events", companyId: "company", binding: null, requiresDelivery: true }); + }); +}); + + +it("retained session selection keeps unfinished GC custody but completed exact receipt history supplies no consumer", () => { + const f = closedProofFixture(), proof = prepareSlackFormProofAcceptance(f.intent, f.operation, "instance"); + const receipt = { version: 1 as const, kind: "slack_form_finalized" as const, binding: f.intent.binding, environment: "test" as const, + operationId: f.intent.operationId, deliveryId: f.intent.deliveryId, appReceiptId: f.row.id, grantId: f.intent.grantId, + publicationId: f.intent.publicationId, interactionId: f.intent.interactionId, userId: f.intent.userId, + viewDigest: hash(JSON.stringify([f.intent.view!.id, f.intent.view!.hash, f.intent.view!.externalId])), + terminalDigest: proof.terminalDigest, outcome: "closed" as const, finalizedAt: f.operation.acceptedAt }; + const base = { ...f.row, payload: { ...f.row.payload, environment: "test" }, result: { ...f.row.result, cloudProofAcceptance: proof, + formAck: { version: 1, phase: "transferred", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }, nextAttemptAt: null } }; + const identity = { instanceId: "instance", environment: "test" as const }; + expect(selectSlackFormRetainedConsumer([base], identity)).toEqual({ owner: null, attention: [] }); + for (const phase of ["accepted", "gc_pending"]) { + const pending = { ...base, result: { ...base.result, formAck: { ...base.result.formAck, phase }, nextAttemptAt: f.operation.acceptedAt } }; + expect(selectSlackFormRetainedConsumer([pending], identity).owner).toMatchObject({ companyId: f.row.companyId, subject: "github-bot-events", requiresDelivery: false }); + } + const malformed = { ...base, result: { ...base.result, formAck: { ...base.result.formAck, receipt: { ...receipt, binding: { ...receipt.binding, companyId: "foreign" } } } } }; + const result = selectSlackFormRetainedConsumer([malformed], identity); + expect(result.owner !== null || result.attention.length > 0).toBe(true); + const stillDue = { ...base, result: { ...base.result, nextAttemptAt: f.operation.acceptedAt } }; + expect(selectSlackFormRetainedConsumer([stillDue], identity).owner).not.toBeNull(); +}); + +describe("Slack held-view retry boundary", () => { + const limited = () => Object.assign(new Error("controlled definite Slack rejection"), { code: "slack_webapi_platform_error", data: { error: "ratelimited" } }); + const held = () => { + const f = fixture(), view = { id: "VRETRY", hash: "loading-hash", externalId: f.operation.externalId }; + return { operation: { ...f.operation, phase: "confirmed" as const, view }, currentBinding: f.current.binding, + environment: "test", sessionActionId: "held-session", updateAttempted: true }; + }; + it("retains the exact held attempt on a definite retryable update refusal", () => { + const context = held(), before = structuredClone(context); + expect(isRetryableSlackFormUpdate(context, limited())).toBe(true); + expect(context).toEqual(before); + }); + it("does not replay an ambiguous update or an accepted update whose local commit failed", () => { + const context = held(); + expect(isRetryableSlackFormUpdate(context, new Error("response lost after possible update"))).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, updatedView: { ...context.operation.view, hash: "accepted-hash" } }, limited())).toBe(false); + }); + it("does not reopen a native trigger, unprepared session, stale binding, or closed view", () => { + const context = held(), f = fixture(); + expect(isRetryableSlackFormUpdate(undefined, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, sessionActionId: undefined }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, updateAttempted: false }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, currentBinding: { ...context.currentBinding, runtimeGeneration: 2 } }, limited())).toBe(false); + expect(isRetryableSlackFormUpdate({ ...context, operation: { ...context.operation, closed: f.close(context.operation.view) } }, limited())).toBe(false); + }); + it("preserves definite invalid-view and stale-hash refusals as nonretryable", () => { + for (const error of ["invalid_arguments", "hash_conflict"]) expect(isRetryableSlackFormUpdate(held(), + Object.assign(new Error("controlled definite view refusal"), { code: "slack_webapi_platform_error", data: { error } }))).toBe(false); + }); +}); diff --git a/server/src/__tests__/slack-cloud-owner-work.test.ts b/server/src/__tests__/slack-cloud-owner-work.test.ts index 587b2ea822a..7343cce4b98 100644 --- a/server/src/__tests__/slack-cloud-owner-work.test.ts +++ b/server/src/__tests__/slack-cloud-owner-work.test.ts @@ -6,7 +6,7 @@ import { and, eq, sql } from "drizzle-orm"; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { SLACK_CHAT_BOT_SCOPES } from "@paperclipai/shared"; import { activityLog, agentRuntimeState, agentTaskSessions, agentWakeupRequests, agents, applyPendingMigrations, authUsers, - chatDeliveries, chatEndpoints, chatExternalPrincipals, companySecretBindings, chatIdentityLinks, chatSlackRegistrations, companies, companyMemberships, + chatActions, chatDeliveries, chatEndpoints, chatExternalPrincipals, companySecretBindings, chatIdentityLinks, chatSlackRegistrations, companies, companyMemberships, companySecrets, companySkills, costEvents, financeEvents, heartbeatRunEvents, heartbeatRuns, issueComments, issues, toolApplications, toolConnections, closeCheckpointDb, closeFileStorageDb, closeLivenessDb, createDb } from "@paperclipai/db"; import { databaseWorkPending, reconcileDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db/work-signals"; @@ -229,7 +229,16 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu expect(Date.parse(retained.nextAttemptAt!)).toBeGreaterThan(Date.now()); expect(cloud.calls.slice(start).filter(call => call.enrollment?.endpointId === a)).toEqual([]); const retryAt = Date.parse(retained.nextAttemptAt!); - expect(owner.nextPollAt()).toBe(retryAt); + const formRetries = await db.select({ result: chatActions.result }).from(chatActions).where(and( + eq(chatActions.companyId, companyId), + sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`, + sql`${chatActions.payload}->>'environment' = ${cloud.config.environment}`, + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${cloud.config.instanceId}`, + sql`${chatActions.result}->>'nextAttemptAt' is not null`)); + expect(formRetries.some(value => value.result?.failureCode === "form_owner_broker_unavailable")).toBe(true); + const formDeadlines = formRetries.map(value => Date.parse(String(value.result?.nextAttemptAt))); + expect(formDeadlines.every(Number.isFinite)).toBe(true); + expect(owner.nextPollAt()).toBe(Math.min(retryAt, ...formDeadlines)); releases.splice(0).forEach(release => release()); await Promise.all(heldLeases); pollTime = retryAt - 1; await owner.pollOnce(); diff --git a/server/src/__tests__/slack-enabled-quiet.postgres.test.ts b/server/src/__tests__/slack-enabled-quiet.postgres.test.ts new file mode 100644 index 00000000000..eb9c26267c6 --- /dev/null +++ b/server/src/__tests__/slack-enabled-quiet.postgres.test.ts @@ -0,0 +1,361 @@ +import { registerBudgetEnforcementWork } from "../services/budget-enforcement-worker.js"; +import { deliverBudgetResumeContinuations } from "../services/budget-resume-continuation.js"; +import { registerChatDeliveryWork } from "../services/chat-delivery-work.js"; +import { registerChatActionWork } from "../services/chat-action-work.js"; +import { isIdleTaskDrainActive, idleWorkSnapshot } from "../services/task-admission.js"; +import { actorMiddleware } from "../middleware/auth.js"; +import { issueRoutes } from "../routes/issues.js"; +import { agentRoutes } from "../routes/agents.js"; +import { createStorageService } from "../storage/service.js"; +import { createLocalDiskStorageProvider } from "../storage/local-disk-provider.js"; +import { parseSlackFormAckWork, assertSlackFormReceiptHistory } from "../services/chat-channels/slack/form-ack-work.js"; +import { verifySlackFormFinalizedReceipt } from "../services/chat-channels/slack/receipt-authority.js"; +import { createHmac, randomUUID } from "node:crypto"; +import { once } from "node:events"; +import { readFile, mkdtemp, rm } from "node:fs/promises"; +import { createServer } from "node:http"; +import os from "node:os"; +import { fileURLToPath } from "node:url"; +import path from "node:path"; +import express from "express"; +import { and, eq } from "drizzle-orm"; +import { expect, it, vi } from "vitest"; +import { activityLog, agentRuntimeState, agentTaskSessions, agentWakeupRequests, agents, applyPendingMigrations, authUsers, + chatActions, chatConversations, chatEndpoints, chatExternalPrincipals, chatPublications, chatSlackRegistrations, + closeCheckpointDb, closeFileStorageDb, closeLivenessDb, companies, companyMemberships, companySecrets, companySkills, + costEvents, createDb, financeEvents, heartbeatRunEvents, heartbeatRuns, issueComments, issues, toolApplications, toolConnections } from "@paperclipai/db"; +import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; +import { guardSlackFixtureNetwork } from "./helpers/slack-cloud/transport.js"; +import { ChatRuntime } from "../services/chat-runtime.js"; +import { CAPABILITIES } from "../services/chat-channels/provider-catalog.js"; +import { chatChannelService } from "../services/chat-channels.js"; +import { reconcilePersistedRuntimeServicesOnStartup } from "../services/workspace-runtime.js"; +import { markIdleStartupComplete } from "../services/idle-local-work.js"; +import { idleAdmissionMiddleware, trackIdleRequestHandlers } from "../middleware/idle-admission.js"; +import { heartbeatService } from "../services/heartbeat.js"; +import { githubConnectionEventService } from "../services/github-connection-events.js"; +import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js"; +import { registerGitHubConnectionEventWork } from "../services/github-connection-work.js"; +import { issueThreadInteractionService } from "../services/issue-thread-interactions.js"; +import { cloudControlMiddleware } from "../middleware/cloud-control.js"; +import { instanceSettingsRoutes } from "../routes/instance-settings.js"; +import { connectionEventWakeRoutes } from "../routes/connection-event-wake.js"; +import { currentSlackFormOwnerBinding, slackFormSetupRecord } from "../services/chat-channels/slack/cloud-forms.js"; + +type PlatformView = { id: string; hash: string; external_id: string; app_id: string; callback_id: string; private_metadata: string; + blocks: { type: string; block_id?: string; element?: { action_id: string; type: string } }[]; state: { values: Record> } }; +type HostState = { + usefulReplies?: { id: string; threadTs: string; runId: string }[]; + bindingCount?: number; coverageEntries?: number; responseStatus?: number; normalRouteReached?: boolean; tamperedStatus?: number; headerBytes?: number; + calls: { method: string; at: number; body?: unknown }[]; views: PlatformView[]; failures: string[]; + wakerStarts: number; droppedHints: number; lostTerminalReplies: number; lostAckReplies: number; + idleProjection?: { backgroundWork: "none" | "present" | "unknown"; checks: { reason: string; state: string }[]; capabilityPresent: boolean }; + instance: { eventWakeVersion?: number }; targets: { delivery_id: string; lease_id: string | null; lease_expires_at: string | null }[]; + bindings: { id: string; profile?: string; metadata?: Record }[]; +}; +type Fixture = { + companyId: string; endpointId: string; connectionId: string; applicationId: string; agentId: string; actorId: string; + instanceId: string; publicId: string; teamId: string; appId: string; botId: string; userId: string; channelId: string; threadTs: string; + botToken: string; signingSecret: string; controlNonce: string; signPrivate: string; sealPrivate: string; controlJwks: unknown; origin: string; tenantOrigin: string; +}; +const appUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_SLACK_TEST_DATABASE_URL"); +const stateFile = process.env.SLACK_FORM_JOURNEY_STATE_FILE; +const shortGate = process.env.SLACK_FORM_SHORT_GATE; +if (appUrl && stateFile && shortGate !== "callback") throw new Error("One closed enabled-quiet callback qualification required"); + +it.skipIf(!appUrl || !stateFile)("enabled Slack finishes ordinary useful tasks and becomes composed quiet before and after fresh signed work", async () => { + const network = guardSlackFixtureNetwork({ requirePreload: true }); + let db: ReturnType | undefined; + let heartbeat: ReturnType | undefined; + let service: ReturnType | undefined; + let coordinator: ReturnType | undefined; + let worker: ReturnType | undefined; + let budgetWorker: ReturnType | undefined; + let fixture: Fixture | undefined, directory: string | undefined, primary: unknown; + const errors: unknown[] = [], ownerErrors: unknown[] = [], observations: unknown[] = []; + const app = express(); + const server = createServer(app); + const sleep = (milliseconds: number) => new Promise(resolve => setTimeout(resolve, milliseconds)); + try { + fixture = JSON.parse(await readFile(stateFile!, "utf8")) as Fixture; + const f = fixture; + expect(new URL(f.origin).hostname).toBe("127.0.0.1"); + expect(new URL(appUrl!).pathname).toMatch(/^\/sleep2_slack_form_[a-z0-9_]+_app_test$/); + const control = async (name: string, body?: unknown): Promise => { + const response = await fetch(`${f.origin}/__fixture/${name}`, { method: body === undefined ? "GET" : "POST", + headers: { "x-fixture-control": f.controlNonce, "content-type": "application/json" }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) }); + expect(response.status).toBe(200); + return await response.json() as HostState; + }; + const signed = (value: unknown, form = false) => { + const body = form ? new URLSearchParams({ payload: JSON.stringify(value) }).toString() : JSON.stringify(value); + const timestamp = String(Math.floor(Date.now() / 1000)); + return { body, headers: { "content-type": form ? "application/x-www-form-urlencoded" : "application/json", "x-slack-request-timestamp": timestamp, + "x-slack-signature": `v0=${createHmac("sha256", f.signingSecret).update(`v0:${timestamp}:${body}`).digest("hex")}` } }; + }; + const post = async (request: ReturnType, origin = f.origin) => { + const started = Date.now(); + const response = await fetch(`${origin}/api/chat-webhooks/${f.publicId}/slack`, { method: "POST", ...request }); + const text = await response.text(); + return { status: response.status, text, elapsed: Date.now() - started }; + }; + await applyPendingMigrations(appUrl!); + db = createDb(appUrl!, { maxConnections: 2, maxLifetimeSeconds: 0 }); + const database = db; + const reconstruction = await reconcilePersistedRuntimeServicesOnStartup(database); + expect(reconstruction).toEqual({ reconciled: 0, adopted: 0, stopped: 0, backfilled: 0, restarted: 0, restartFailed: 0, exposureReservationDrift: [] }); + directory = await mkdtemp(path.join(os.tmpdir(), "form-journey-")); + for (const [key, value] of Object.entries({ + PAPERCLIP_CLOUD_CONNECTOR_BASE_URL: f.origin, PAPERCLIP_CLOUD_CONNECTOR_ENVIRONMENT: "test", PAPERCLIP_CLOUD_CONNECTOR_INSTANCE_ID: f.instanceId, + PAPERCLIP_CLOUD_CONNECTOR_SIGN_PRIVATE_KEY: f.signPrivate, PAPERCLIP_CLOUD_CONNECTOR_SEAL_PRIVATE_KEY: f.sealPrivate, + PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify(f.controlJwks), PAPERCLIP_CLOUD_STACK_ID: f.instanceId, + PAPERCLIP_SECRETS_MASTER_KEY_FILE: path.join(directory, "master.key"), + PAPERCLIP_AGENT_JWT_SECRET: `${randomUUID()}${randomUUID()}`, PAPERCLIP_API_URL: f.tenantOrigin, + })) vi.stubEnv(key, value); + const now = new Date(); + await db.insert(companies).values({ id: f.companyId, name: "Form boundary journey", issuePrefix: `F${f.companyId.slice(0, 7)}` }); + await db.insert(authUsers).values({ id: f.actorId, name: "Form operator", email: `${f.actorId}@example.test`, emailVerified: true, createdAt: now, updatedAt: now }); + await db.insert(companyMemberships).values({ companyId: f.companyId, principalType: "user", principalId: f.actorId, status: "active", membershipRole: "operator" }); + await db.insert(agents).values({ id: f.agentId, companyId: f.companyId, name: "Scoped useful process", adapterType: "codex_local", status: "idle", + adapterConfig: { engine: "cli", command: fileURLToPath(new URL("./helpers/slack-cloud/controlled-codex-cli.cjs", import.meta.url)), cwd: directory, + env: { SLACK_FORM_FIXTURE_PEER_ORIGIN: f.tenantOrigin, CODEX_HOME: path.join(directory, "owned-controlled-cli-home"), + OPENAI_API_KEY: "controlled-command-no-real-provider-authority" }, timeoutSec: 10, graceSec: 1 }, + runtimeConfig: { heartbeat: { wakeOnDemand: true, intervalSec: 0 } } }); + await db.insert(companyMemberships).values({ companyId: f.companyId, principalType: "agent", principalId: f.agentId, status: "active", membershipRole: "member" }); + await db.insert(toolApplications).values({ id: f.applicationId, companyId: f.companyId, name: "Slack", type: "chat" }); + await db.insert(toolConnections).values({ id: f.connectionId, companyId: f.companyId, applicationId: f.applicationId, name: "Slack form", uid: f.endpointId, + transport: "chat_sdk", connectionPurpose: "channel", enabled: true, status: "active" }); + await db.insert(chatEndpoints).values({ id: f.endpointId, companyId: f.companyId, connectionId: f.connectionId, assignedAgentId: f.agentId, publicId: f.publicId, + provider: "slack", status: "draft", sponsorUserId: f.actorId, allowGroupChats: true, allowUnlinkedPeople: false, + capabilities: CAPABILITIES.slack, setup: { step: "provider_setup" } }); + await db.insert(chatSlackRegistrations).values({ companyId: f.companyId, endpointId: f.endpointId, requestId: randomUUID(), status: "configured", appId: f.appId, + manifest: {}, manifestHash: "fixture", callbackUri: `https://tenant.example/api/chat-webhooks/${f.publicId}/slack`, createdByUserId: f.actorId }); + const runtime = new ChatRuntime(), replace = runtime.replaceEndpoint.bind(runtime); + vi.spyOn(runtime, "replaceEndpoint").mockImplementation(input => { + if (input.providerConfig.provider !== "slack") throw new Error("Unexpected form fixture provider"); + return replace({ ...input, providerConfig: { ...input.providerConfig, credentials: { ...input.providerConfig.credentials, apiUrl: `${f.origin}/fixture/slack/` } } }); + }); + const providerFetch: typeof fetch = (input, init) => { + const url = new URL(input instanceof Request ? input.url : String(input)); + if (url.origin !== "https://slack.com" || !url.pathname.startsWith("/api/")) throw new Error("Unexpected provider leaf request"); + return fetch(`${f.origin}/fixture/slack/${url.pathname.slice(5)}`, init); + }; + heartbeat = heartbeatService(db); + service = chatChannelService(db, { runtime, heartbeat, fetch: providerFetch, publicBaseUrl: "https://tenant.example" }); + const channels = service; + app.use(idleAdmissionMiddleware); + app.use(actorMiddleware(database, { deploymentMode: "authenticated" })); + app.use(cloudControlMiddleware()); app.use(connectionEventWakeRoutes(db)); + app.use("/api/instance/task-drain", express.json()); + app.use("/api", instanceSettingsRoutes(database)); + app.use("/api/issues", express.json()); + app.use("/api", issueRoutes(database, createStorageService(createLocalDiskStorageProvider(path.join(directory, "storage"))))); + app.use("/api", agentRoutes(database, { deploymentMode: "authenticated" })); + app.post(`/api/chat-webhooks/${f.publicId}/slack`, express.raw({ type: () => true, limit: "1mb" }), async (req, res, next) => { + try { + const headers = new Headers(); + for (const [name, value] of Object.entries(req.headers)) if (typeof value === "string" && name !== "host") headers.set(name, value); + const response = await channels.handleWebhook(f.publicId, "slack", new Request(`https://tenant.example${req.path}`, { method: "POST", headers, body: new Uint8Array(req.body) })); + res.status(response.status); response.headers.forEach((value, name) => res.setHeader(name, value)); res.end(Buffer.from(await response.arrayBuffer())); + } catch (error) { next(error); } + }); + app.use((error: unknown, _req: express.Request, res: express.Response, _next: express.NextFunction) => { ownerErrors.push(error); res.status(500).end("fixture handler failed"); }); + expect(new URL(f.tenantOrigin).hostname).toBe("127.0.0.1"); + trackIdleRequestHandlers(app); + server.listen(Number(new URL(f.tenantOrigin).port), "127.0.0.1"); await once(server, "listening"); + const address = server.address(); if (!address || typeof address === "string") throw new Error("Missing tenant fixture address"); + const tenantOrigin = `http://127.0.0.1:${address.port}`; + expect(tenantOrigin).toBe(f.tenantOrigin); + await control("tenant", { origin: tenantOrigin }); + const startOwner = async () => { + // Same queue owners and idle-drain gate as normal app.ts composition. + // This fixture has no optional CloudWarmStandby owner (normal default). + const canDispatch = () => !isIdleTaskDrainActive(); + coordinator = createDeliveryWorkCoordinator({ owner: database, canRun: canDispatch, canReconcile: () => true, + onError: error => ownerErrors.push(error) }); + const deliveries = registerChatDeliveryWork(coordinator, channels, canDispatch); + const actions = registerChatActionWork(coordinator, channels, canDispatch); + const ownedHeartbeat = heartbeat; + if (!ownedHeartbeat) throw new Error("Missing normal heartbeat budget authority"); + budgetWorker = registerBudgetEnforcementWork(database, coordinator, { + cancelWorkForScope: ownedHeartbeat.cancelBudgetScopeWork, + deliverResumeContinuations: () => deliverBudgetResumeContinuations(database, { + prepareChatRetry: channels.prepareFailedChatRunRetry, + resumeTaskRetry: (companyId, requestId) => ownedHeartbeat.resumeTaskRetry(companyId, requestId), + }), + }); + worker = registerGitHubConnectionEventWork(database, coordinator, githubConnectionEventService(database), canDispatch); + await Promise.all([deliveries.ready, actions.ready, worker.ready, budgetWorker.ready]); + }; + const stopOwner = async () => { await worker?.stop(); await budgetWorker?.stop(); await coordinator?.stop(); worker = undefined; budgetWorker = undefined; coordinator = undefined; }; + await startOwner(); + markIdleStartupComplete({ scheduledBackups: false }); + await service.configure(f.endpointId, { action: "configure", credentials: { botToken: f.botToken, signingSecret: f.signingSecret } }, f.actorId); + expect((await post(signed({ type: "url_verification", team_id: f.teamId, api_app_id: f.appId, challenge: "form-journey-challenge" }), tenantOrigin)).status).toBe(200); + await service.configure(f.endpointId, { action: "verify" }, f.actorId); + const [principal] = await db.insert(chatExternalPrincipals).values({ companyId: f.companyId, provider: "slack", providerAccountId: f.teamId, externalId: f.userId, kind: "user", displayName: "Operator" }).returning(); + expect(principal).toBeDefined(); + const link = await service.createLinkIntent(f.endpointId, principal!.id, 1800); + const token = new URL(link.confirmationUrl).searchParams.get("token"); if (!token) throw new Error("Missing normal identity link token"); + await service.confirmIdentityLink(token, f.actorId); + expect((await service.test(f.endpointId, { optionalSlackTestForUser: f.actorId })).status).toBe("active"); + const endpoint = async () => (await database.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.endpointId)))[0]!; + await vi.waitFor(async () => { + expect(currentSlackFormOwnerBinding(slackFormSetupRecord((await endpoint()).setup))).not.toBeNull(); + expect((await control("state")).instance.eventWakeVersion).toBe(1); + }, { timeout: 15_000 }); + const startupProjection = (await control("idle-projection", {})).idleProjection; + expect(startupProjection).toBeDefined(); + console.info(JSON.stringify({ diagnostic: "slack_form_idle_startup", ...startupProjection })); + expect(startupProjection?.checks.filter(value => value.reason.startsWith("local.") && value.state === "unknown")).toEqual([]); + const bootstrap = signed({ type: "event_callback", event_id: `EvForm${randomUUID().replaceAll("-", "")}`, team_id: f.teamId, api_app_id: f.appId, + event: { type: "app_mention", text: `<@${f.botId}> help me choose`, user: f.userId, channel: f.channelId, ts: f.threadTs, event_ts: f.threadTs } }); + expect((await post(bootstrap)).status).toBe(200); + const conversation = async () => (await database.select().from(chatConversations).where(eq(chatConversations.endpointId, f.endpointId)))[0]; + await vi.waitFor(async () => expect(await conversation()).toBeDefined(), { timeout: 15_000 }); + const current = (await conversation())!; + const enabledIdentity = async () => { + const [endpoint] = await database.select().from(chatEndpoints).where(and(eq(chatEndpoints.companyId, f.companyId), eq(chatEndpoints.id, f.endpointId))); + const [connection] = await database.select().from(toolConnections).where(and(eq(toolConnections.companyId, f.companyId), eq(toolConnections.id, f.connectionId))); + expect(endpoint?.status).toBe("active"); expect(connection?.enabled).toBe(true); + return { endpointId: endpoint!.id, connectionId: connection!.id, capabilities: endpoint!.capabilities, setup: endpoint!.setup }; + }; + const configured = await enabledIdentity(); + const usefulRuns = async (count: number) => { + const runs = await database.select().from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, f.companyId), eq(heartbeatRuns.agentId, f.agentId))); + expect(runs).toHaveLength(count); + const receipts = []; + for (const run of runs) { + expect(run.status).toBe("succeeded"); expect(run.exitCode).toBe(0); + const issueId = run.contextSnapshot?.issueId; + if (typeof issueId !== "string") throw new Error("Missing admitted actual task identity"); + const [task] = await database.select().from(issues).where(and(eq(issues.companyId, f.companyId), eq(issues.id, issueId))); + expect(task?.status).toBe("done"); expect(task?.checkoutRunId).toBeNull(); expect(task?.executionRunId).toBeNull(); + const replies = await database.select().from(issueComments).where(and(eq(issueComments.issueId, issueId), eq(issueComments.createdByRunId, run.id), eq(issueComments.authorAgentId, f.agentId))); + expect(replies).toHaveLength(1); expect(replies[0]?.body).toBe(`Scoped Slack useful reply for run ${run.id}`); + const publications = await database.select().from(chatPublications).where(and(eq(chatPublications.companyId, f.companyId), eq(chatPublications.endpointId, f.endpointId), eq(chatPublications.commentId, replies[0]!.id))); + expect(publications).toHaveLength(1); expect(publications[0]?.state).toBe("published"); expect(publications[0]?.providerMessageId).toBeTruthy(); + const platform = await control("state"); + expect(platform.usefulReplies?.filter(reply => reply.runId === run.id && reply.id === publications[0]!.providerMessageId)).toHaveLength(1); + receipts.push({ runId: run.id, issueId, commentId: replies[0]!.id, publicationId: publications[0]!.id, providerMessageId: publications[0]!.providerMessageId }); + } + expect(new Set(receipts.map(value => value.issueId)).size).toBe(count); + return receipts.sort((left, right) => left.runId.localeCompare(right.runId)); + }; + type WireSnapshot = { pid: number; observedAt: number; pgConnectAttempts: number; pgConnects: number; pgCloses: number; + pgWriteCalls: number; outboundRequests: number; nonPgConnectAttempts: number; livePgSockets: number }; + const wire = Reflect.get(process, Symbol.for("paperclip.slack.fixture.prewire")) as { + pid: number; wireSnapshot(): WireSnapshot; waitForPgSocketsClosed(timeoutMs: number): Promise; + beginWireWindow(): WireSnapshot; endWireWindow(): { before: WireSnapshot; after: WireSnapshot; peakPgSockets: number; + pgWriteCalls: number; pgConnectAttempts: number; pgConnects: number; pgCloses: number; outboundRequests: number; nonPgConnectAttempts: number }; + }; + expect(wire?.pid).toBe(process.pid); + const observeQuiet = async (label: string, count: number) => { + const settlementDeadline = Date.now() + 95_000; + let ordinaryQueryWitness: { before: WireSnapshot; after: WireSnapshot } | undefined; + let lastInspection: { observedAt: number; projection: HostState["idleProjection"]; coordinatorNextWakeAt: number | null; localWork: ReturnType; ownerErrorCount: number } | undefined; + let reportedSignature: string | undefined, diagnosticTransitions = 0; + try { + await vi.waitFor(async () => { + const queryBefore = wire.wireSnapshot(); + await usefulRuns(count); // Existing ordinary reads are the counter liveness control; no extra query. + const queryAfter = wire.wireSnapshot(); + expect(queryAfter.pgWriteCalls).toBeGreaterThan(queryBefore.pgWriteCalls); + ordinaryQueryWitness ??= { before: queryBefore, after: queryAfter }; + const projection = (await control("idle-projection", {})).idleProjection; + // Retain only values already returned by this required normal read. + // Passive snapshots below do not query, drain, schedule or change authority. + lastInspection = { observedAt: Date.now(), projection, coordinatorNextWakeAt: coordinator?.nextWakeAt() ?? null, + localWork: idleWorkSnapshot(), ownerErrorCount: ownerErrors.length }; + const signature = JSON.stringify(projection); + if (signature !== reportedSignature && diagnosticTransitions < 16) { + reportedSignature = signature; diagnosticTransitions += 1; + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_owner_observation", label, + sequence: diagnosticTransitions, ...lastInspection, noAdditionalRead: true })); } catch { /* Observation cannot change assertion. */ } + } + expect(projection).toBeDefined(); expect(projection?.backgroundWork).toBe("none"); + }, { timeout: Math.max(1, settlementDeadline - Date.now()), interval: 250 }); + } catch (error) { + // Log before finally deletes fixture rows, without extra SQL/HTTP. + // The original error remains primary even if evidence output fails. + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_wait_failure", label, + lastInspection, observedAt: Date.now(), noAdditionalRead: true })); } catch { /* Preserve original failure. */ } + throw error; + } + const before = { observedAt: Date.now(), receipts: await usefulRuns(count), config: await enabledIdentity(), state: await control("state"), projection: (await control("idle-projection", {})).idleProjection }; + expect(before.state.targets).toHaveLength(0); expect(before.state.failures).toEqual([]); expect(before.config).toEqual(configured); + expect(before.projection?.backgroundWork).toBe("none"); + // All existing before-snapshot reads above are complete. Wait only on + // actual close events, inside the SAME original 95s settlement budget. + console.info(JSON.stringify({ diagnostic: "slack_enabled_used_pg_close_before", label, + observedAt: Date.now(), settlementDeadline, ordinaryQueryWitness, wire: wire.wireSnapshot(), noAdditionalRead: true })); + try { await wire.waitForPgSocketsClosed(settlementDeadline - Date.now()); } + catch (error) { + try { console.info(JSON.stringify({ diagnostic: "slack_enabled_used_pg_close_failure", label, + observedAt: Date.now(), settlementDeadline, wire: wire.wireSnapshot(), noAdditionalRead: true })); } catch { /* Preserve original failure. */ } + throw error; + } + expect(wire.wireSnapshot().livePgSockets).toBe(0); + const wireBefore = wire.beginWireWindow(); + await sleep(30_000); // Original observation duration; no SQL/control reads in this window. + const wireWindow = wire.endWireWindow(); // Immediately after sleep, BEFORE any after-snapshot query. + console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_wire_window", label, ordinaryQueryWitness, wireBefore, wireWindow })); + expect(wireWindow.pgWriteCalls).toBe(0); expect(wireWindow.pgConnectAttempts).toBe(0); + expect(wireWindow.pgConnects).toBe(0); expect(wireWindow.pgCloses).toBe(0); + expect(wireWindow.peakPgSockets).toBe(0); expect(wireWindow.after.livePgSockets).toBe(0); + expect(wireWindow.outboundRequests).toBe(0); expect(wireWindow.nonPgConnectAttempts).toBe(0); + const after = { observedAt: Date.now(), receipts: await usefulRuns(count), config: await enabledIdentity(), state: await control("state"), projection: (await control("idle-projection", {})).idleProjection }; + expect(after.projection?.backgroundWork).toBe("none"); expect(after.receipts).toEqual(before.receipts); expect(after.config).toEqual(configured); + expect(after.state.targets).toHaveLength(0); expect(after.state.failures).toEqual([]); + expect(after.state.usefulReplies).toEqual(before.state.usefulReplies); + expect(after.state.calls).toHaveLength(before.state.calls.length); + console.info(JSON.stringify({ diagnostic: "slack_enabled_quiet_window", label, startedAt: before.observedAt, endedAt: after.observedAt, + runCount: count, usefulReplies: after.state.usefulReplies?.length, targets: after.state.targets.length, + configurationRetained: true, before: before.projection, after: after.projection, noPhysicalOrSqlZeroClaim: true })); + }; + await observeQuiet("initial-useful-task", 1); + const secondThread = (Number(f.threadTs) + 1).toFixed(6); + const fresh = signed({ type: "event_callback", event_id: `EvFresh${randomUUID().replaceAll("-", "")}`, team_id: f.teamId, api_app_id: f.appId, + event: { type: "app_mention", text: `<@${f.botId}> complete a fresh useful task`, user: f.userId, channel: f.channelId, ts: secondThread, event_ts: secondThread } }); + const admitted = await post(fresh); expect(admitted.status).toBe(200); expect(admitted.elapsed).toBeLessThan(3000); + const duplicate = await post(fresh); expect(duplicate.status).toBe(200); expect(duplicate.elapsed).toBeLessThan(3000); + await observeQuiet("fresh-useful-reactivation-and-duplicate", 2); + await usefulRuns(2); expect(await enabledIdentity()).toEqual(configured); + expect(ownerErrors).toEqual([]); + } catch (error) { primary = error; } + finally { + for (const close of [async () => worker?.stop(), async () => budgetWorker?.stop(), async () => coordinator?.stop(), async () => service?.shutdown(), async () => heartbeat?.drainActiveRunExecutions(), + async () => { if (server.listening) { server.close(); server.closeAllConnections(); await once(server, "close"); } }]) { + try { await close(); } catch (error) { errors.push(error); } + } + if (db && fixture) { + const f = fixture; + try { + await db.delete(chatEndpoints).where(eq(chatEndpoints.companyId, f.companyId)); + await db.delete(financeEvents).where(eq(financeEvents.companyId, f.companyId)); + await db.delete(costEvents).where(eq(costEvents.companyId, f.companyId)); + await db.delete(activityLog).where(eq(activityLog.companyId, f.companyId)); + await db.delete(agentTaskSessions).where(eq(agentTaskSessions.companyId, f.companyId)); + await db.delete(agentRuntimeState).where(eq(agentRuntimeState.companyId, f.companyId)); + await db.delete(heartbeatRunEvents).where(eq(heartbeatRunEvents.companyId, f.companyId)); + await db.delete(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId)); + await db.delete(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId)); + await db.delete(issueComments).where(eq(issueComments.companyId, f.companyId)); + await db.delete(issues).where(eq(issues.companyId, f.companyId)); + await db.delete(agents).where(eq(agents.companyId, f.companyId)); + await db.delete(companyMemberships).where(eq(companyMemberships.companyId, f.companyId)); + await db.delete(companySecrets).where(eq(companySecrets.companyId, f.companyId)); + await db.delete(companySkills).where(eq(companySkills.companyId, f.companyId)); + await db.delete(companies).where(eq(companies.id, f.companyId)); await db.delete(authUsers).where(eq(authUsers.id, f.actorId)); + } catch (error) { errors.push(error); } + for (const close of [closeCheckpointDb, closeFileStorageDb, closeLivenessDb]) { try { await close(db, 1); } catch (error) { errors.push(error); } } + try { await db.$client.end({ timeout: 1 }); } catch (error) { errors.push(error); } + } + try { network.assertQuiet(); } catch (error) { errors.push(error); } + network.restore(); vi.restoreAllMocks(); vi.unstubAllEnvs(); + if (directory) { try { await rm(directory, { recursive: true, force: true }); } catch (error) { errors.push(error); } } + } + if (primary && errors.length) throw new AggregateError([primary, ...errors], "Slack form journey body and cleanup failed"); + if (primary) throw primary; + if (errors.length) throw new AggregateError(errors, "Slack form journey cleanup failed"); +}, 360_000); diff --git a/server/src/__tests__/slack-form-publication-readiness.test.ts b/server/src/__tests__/slack-form-publication-readiness.test.ts new file mode 100644 index 00000000000..dcf0fd3bc2e --- /dev/null +++ b/server/src/__tests__/slack-form-publication-readiness.test.ts @@ -0,0 +1,116 @@ +import { describe, expect, it, vi } from "vitest"; +import { chatActions, chatEndpoints, chatPublications, toolConnections, type Db } from "@paperclipai/db"; +import { createSlackFormPublicationStage } from "../services/chat-channels/slack/form-publication.js"; + +const fixture = vi.hoisted(() => ({ interactions: [] as unknown[] })); +vi.mock("../services/issue-thread-interactions.js", () => ({ + issueThreadInteractionService: () => ({ listForIssue: async () => fixture.interactions }), +})); +const binding = { version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", connectionId: "connection", + publicId: "p".repeat(43), runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", + eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }; +const { credentialFingerprint: _fingerprint, eventBindingId, ...identity } = binding; +const enrollment = { ...identity, bindingId: eventBindingId }; +const scheduledIntent = { version: 1, id: "intent", active: true, subject: "actor", instanceId: "instance", + companyId: "company", endpointId: "endpoint", connectionId: "connection", publicId: binding.publicId, + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", attempts: 0, nextAttemptAt: "2030-01-01T00:00:00Z" }; +const confirmedSetup = { runtimeGeneration: 1, slackCloudInstanceId: "instance", slackCloudSubject: "actor", + slackCloudEvents: { version: 1, runtimeGeneration: 1, bindingId: eventBindingId, teamId: "TTEST", appId: "ATEST" } }; +function currentOwner() { + return { id: "owner", status: "issued", payload: { enrollment, binding, subject: "actor", environment: "test" }, + result: { phase: "enrolled", nextAttemptAt: null as string | null, desiredActive: true } }; +} +function inputs(setup: Record, owners: unknown[] = []) { + const endpoint: typeof chatEndpoints.$inferSelect = { + id: "endpoint", companyId: "company", connectionId: "connection", provider: "slack", + setup: { step: "provider_setup", ...setup }, status: "active", publicId: binding.publicId, providerAccountId: "TTEST", + assignedAgentId: "agent", sponsorUserId: "actor", publicationMode: "automatic", externalExecutionPolicy: "restricted", + deploymentMode: "direct", providerAccountLabel: null, botExternalId: null, botUsername: null, botDisplayName: null, + botAvatarUrl: null, communicationInstructions: "", executionDefaults: null, allowDirectMessages: true, + requireAtMention: false, allowGroupChats: false, allowUnlinkedPeople: false, concurrencyPolicy: "queue", + capabilities: { threads: true, directMessages: true, nativeStreaming: true, messageEdits: true, messageDeletes: true, + reactions: true, files: true, cards: true, actions: true, modals: true, slashCommands: true, + ephemeralMessages: true, proactiveDirectMessages: false }, + healthMessage: null, lastEventAt: null, lastPublicationAt: null, lastError: null, activatedAt: null, archivedAt: null, + createdAt: new Date(0), updatedAt: new Date(0), + }; + const publication = { id: "publication", conversationId: "conversation", companyId: endpoint.companyId, + endpointId: endpoint.id, issueId: "issue", payload: { interactionId: "interaction" } } as typeof chatPublications.$inferSelect; + const connection = { id: "connection", companyId: "company", status: "active", enabled: true }; + fixture.interactions = [{ id: "interaction", companyId: "company", issueId: "issue", status: "pending", kind: "ask_user_questions", + payload: { version: 1, title: "Choose", questions: ["first", "second"].map(id => ({ id, prompt: "Choose", + selectionMode: "single", options: [{ id: "a", label: "A" }] })) } }]; + const select = vi.fn((columns?: unknown) => ({ from: (table: unknown) => ({ where: () => ({ limit: async () => + table === chatEndpoints ? [endpoint] : table === toolConnections ? [connection] + : table === chatActions && columns ? [{ id: "open" }] : owners, + }) }) })); + const db = { select } as unknown as Db; + return { endpoint, publication, connection, select, prepare: createSlackFormPublicationStage(db) }; +} +describe("Slack form publication enrollment readiness", () => { + it("waits for the first managed enrollment before a forms binding exists", async () => { + const f = inputs({ slackCloudIntent: scheduledIntent, runtimeGeneration: 1 }); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toEqual({ kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2000 }); + expect(f.select).toHaveBeenCalledTimes(3); + }); + it("preserves local Slack publication when no managed Cloud enrollment is configured", async () => { + const f = inputs({ step: "ready" }); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toBeNull(); + expect(f.select).not.toHaveBeenCalled(); + }); + it("does not wait for unsupported or no longer pending interactions", async () => { + const f = inputs({ slackCloudIntent: scheduledIntent, runtimeGeneration: 1 }); + fixture.interactions = []; + await expect(f.prepare(f.endpoint, f.publication)).resolves.toBeNull(); + expect(f.select).not.toHaveBeenCalled(); + }); + it("retains fail-closed parsing for a present malformed binding", async () => { + const f = inputs({ ...confirmedSetup, slackCloudForms: { version: 1 } }); + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "invalid_form_receipt" }); + }); + it("waits for an exact current scheduled forms owner, including a retained older binding", async () => { + const owner = { ...currentOwner(), payload: { enrollment, subject: "actor", environment: "test" }, + result: { phase: "staged", nextAttemptAt: "2030-01-01T00:00:00Z", desiredActive: true } }; + for (const retained of [undefined, { ...binding, runtimeGeneration: 2 }]) { + const f = inputs({ ...confirmedSetup, ...(retained ? { slackCloudForms: retained } : {}) }, [owner]); + await expect(f.prepare(f.endpoint, f.publication)).resolves.toMatchObject({ kind: "wait" }); + } + }); + it("returns the commit hook only for the exact current enrolled owner", async () => { + const f = inputs({ ...confirmedSetup, slackCloudForms: binding }, [currentOwner()]); + expect(typeof await f.prepare(f.endpoint, f.publication)).toBe("function"); + }); + it("preserves an attention endpoint's exact usable enrolled owner without allowing attention waits", async () => { + const ready = inputs({ ...confirmedSetup, slackCloudForms: binding }, [currentOwner()]); + ready.endpoint.status = "attention"; + expect(typeof await ready.prepare(ready.endpoint, ready.publication)).toBe("function"); + const pending = inputs({ ...confirmedSetup }, [{ ...currentOwner(), payload: { enrollment, subject: "actor", environment: "test" }, + result: { phase: "staged", nextAttemptAt: "2030-01-01T00:00:00Z", desiredActive: true } }]); + pending.endpoint.status = "attention"; + await expect(pending.prepare(pending.endpoint, pending.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); + it.each(["attention", "retired", "transferred", "stopped_deadline", "missing", "ambiguous", "stale_binding", "foreign_subject"] as const)( + "does not grant budget-free polling to %s forms responsibility", async fault => { + const owner = currentOwner(); + if (fault === "attention") { owner.status = "attention"; owner.result.phase = "staged"; owner.result.nextAttemptAt = null; } + if (fault === "retired") owner.result.desiredActive = false; + if (fault === "transferred") owner.result.phase = "transferred"; + if (fault === "stopped_deadline") { owner.result.phase = "staged"; owner.result.nextAttemptAt = null; } + if (fault === "foreign_subject") owner.payload.subject = "other"; + const f = inputs({ ...confirmedSetup, slackCloudForms: fault === "stale_binding" ? { ...binding, runtimeGeneration: 2 } : binding }, + fault === "missing" ? [] : fault === "ambiguous" ? [owner, { ...owner, id: "duplicate" }] : [owner]); + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); + it.each(["attention", "inactive", "generation", "company", "retired", "connection"] as const)( + "does not grant budget-free polling to a %s events intent", async fault => { + const intent = { ...scheduledIntent, nextAttemptAt: scheduledIntent.nextAttemptAt as string | null }; + if (fault === "attention") intent.nextAttemptAt = null; + if (fault === "inactive") intent.active = false; + if (fault === "generation") intent.runtimeGeneration = 2; + if (fault === "company") intent.companyId = "foreign"; + const f = inputs({ slackCloudInstanceId: "instance", runtimeGeneration: 1, slackCloudIntent: intent, + ...(fault === "retired" ? { slackCloudRetiredGeneration: 1 } : {}) }); + if (fault === "connection") f.connection.enabled = false; + await expect(f.prepare(f.endpoint, f.publication)).rejects.toMatchObject({ code: "form_publication_enrollment_pending", disposition: "retry" }); + }); +}); diff --git a/server/src/__tests__/slack-form-retirement-work.test.ts b/server/src/__tests__/slack-form-retirement-work.test.ts new file mode 100644 index 00000000000..2ef3b751586 --- /dev/null +++ b/server/src/__tests__/slack-form-retirement-work.test.ts @@ -0,0 +1,510 @@ +import { readSlackIdleIncomingProof } from "../services/chat-channels/slack/idle-incoming.js"; +import { idleChatHistoryChecksForOwner } from "../services/chat-channels/idle-history-proof.js"; +import { registerConnectionEventWake, hasConnectionEventWakeReceiver } from "../services/connection-event-wake.js"; +import { slackConfiguredBindingsFingerprint } from "../services/chat-channels/slack/configured-coverage.js"; +import { verifyCloudControlAssertion, CLOUD_CONTROL_AUDIENCE, CLOUD_CONTROL_JWS_TYPE } from "../services/cloud-runtime-identity.js"; +import type { VerifiedIdleCapabilities } from "../services/idle-incoming-wake-capability.js"; +import { retainSlackFormOpeningProof } from "../services/chat-channels/slack/form-opening-proof.js"; +import { readSlackFormRetainedConsumer } from "../services/chat-channels/slack/cloud-consumer.js"; +import { parseSlackFormAckWork, transferSlackFormAckReceipt } from "../services/chat-channels/slack/form-ack-work.js"; +import { parseSlackFormFinalizedReceipt, slackFormFinalizedReceiptDigest } from "../services/chat-channels/slack/ack-receipt.js"; +import { createPaperclipCloudConnector } from "../services/paperclip-cloud-connector.js"; +import { githubConnectionEventService } from "../services/github-connection-events.js"; +import { createHash, createHmac, generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { and, eq, sql } from "drizzle-orm"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; +import { activityLog, agents, applyPendingMigrations, authUsers, chatActions, chatEndpoints, companies, companyMemberships, + toolApplications, toolConnections, createDb, closeCheckpointDb, closeFileStorageDb, closeLivenessDb } from "@paperclipai/db"; +import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js"; +import { stageSlackFormEnrollment, stageSlackFormOwnerConfiguration, reconcileSlackFormEnrollment, reconcilePendingSlackFormWork } from "../services/chat-channels/slack/form-owner.js"; +import { acceptSlackFormRetirementLocally, transferSlackFormRetirement } from "../services/chat-channels/slack/form-retirement-work.js"; +import { slackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "../services/chat-channels/slack/ack-retirement-receipt.js"; +import { parseSlackFormBinding } from "../services/chat-channels/slack/cloud-forms.js"; +import { retrySlackFormAcknowledgement } from "../services/chat-channels/slack/form-retry.js"; +import { connectionManagementService } from "../services/connection-management.js"; + +const external = resolveExternalTestDatabaseUrl("PAPERCLIP_SLACK_RETIREMENT_TEST_DATABASE_URL"); +if (process.env.SLACK_GATE_EXPECT_PRELOAD === "1" && !external) throw new Error("Guarded retirement SQL requires an explicit external database"); +(external ? describe : describe.skip)("Slack retirement original-custody SQL handoff", () => { + let db: ReturnType, companyId: string, endpointId: string, connectionId: string, applicationId: string, agentId: string, actorId: string, actionId: string; + const identity = { instanceId: "retirement-sql-instance", environment: "test" as const }; + let binding: ReturnType, compactJws: string; + const row = async () => (await db.select().from(chatActions).where(and(eq(chatActions.id, actionId), eq(chatActions.companyId, companyId))))[0]!; + const trigger = "slack_retirement_controlled_update"; + function signCurrentReceipt(kid = "retirement-sql-key") { + const pair = generateKeyPairSync("ed25519"); + vi.stubEnv("PAPERCLIP_CLOUD_STACK_ID", identity.instanceId); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] })); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const receipt = slackFormRetirementReceipt(binding, identity.environment, actorId); + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + return body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + } + + beforeAll(async () => { + if (!external) throw new Error("explicit retirement database absent"); + await applyPendingMigrations(external); + db = createDb(external, { maxConnections: 2, idleTimeoutSeconds: 0, maxLifetimeSeconds: 0 }); + }, 60000); + beforeEach(async () => { + companyId = randomUUID(); endpointId = randomUUID(); connectionId = randomUUID(); applicationId = randomUUID(); agentId = randomUUID(); actorId = `retirement-${randomUUID()}`; + const now = new Date(); + await db.insert(companies).values({ id: companyId, name: "Retirement custody", issuePrefix: `R${companyId.slice(0, 7)}` }); + await db.insert(authUsers).values({ id: actorId, name: "Original owner", email: `${actorId}@example.test`, emailVerified: true, createdAt: now, updatedAt: now }); + await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: actorId, membershipRole: "operator", status: "active" }); + await db.insert(agents).values({ id: agentId, companyId, name: "Custody fixture", adapterType: "process", status: "idle", runtimeConfig: { heartbeat: { wakeOnDemand: false, intervalSec: 0 } } }); + await db.insert(toolApplications).values({ id: applicationId, companyId, name: "Slack custody", type: "chat" }); + await db.insert(toolConnections).values({ id: connectionId, companyId, applicationId, name: "Slack custody", uid: `slack-${connectionId}`, + connectionPurpose: "channel", transport: "chat_sdk", status: "active", enabled: true, createdByUserId: actorId, config: { provider: "slack" } }); + binding = parseSlackFormBinding({ version: 1, instanceId: identity.instanceId, companyId, endpointId, connectionId, publicId: "p".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); + const { credentialFingerprint, eventBindingId, ...base } = binding, enrollment = { ...base, bindingId: eventBindingId }; + await db.insert(chatEndpoints).values({ id: endpointId, companyId, connectionId, assignedAgentId: agentId, provider: "slack", publicId: binding.publicId, + sponsorUserId: actorId, status: "active", allowUnlinkedPeople: false, setup: { step: "provider_setup" } }); + await db.update(chatEndpoints).set({ setup: sql`${JSON.stringify({ step: "provider_setup", runtimeGeneration: 1, slackCloudInstanceId: identity.instanceId, + slackCloudSubject: actorId, slackCloudEvents: { runtimeGeneration: 1, bindingId: eventBindingId, teamId: binding.teamId, appId: binding.appId } })}::jsonb` }).where(eq(chatEndpoints.id, endpointId)); + actionId = await db.transaction(tx => stageSlackFormEnrollment(tx, { enrollment, subject: actorId, environment: identity.environment })); + await reconcileSlackFormEnrollment(db, { actionId, identity, resolveBotToken: async () => "xoxb-controlled-test-only", protocol: async input => { + expect(input.operation.kind).toBe("enroll"); expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); return { binding }; + } }); + expect((await row()).result?.phase).toBe("enrolled"); + await db.transaction(async tx => { + await tx.update(chatEndpoints).set({ status: "paused" }).where(and(eq(chatEndpoints.id, endpointId), eq(chatEndpoints.companyId, companyId))); + await stageSlackFormOwnerConfiguration(tx, endpointId, false, actorId, identity); + }); + compactJws = signCurrentReceipt(); + }); + afterEach(async () => { + const errors: unknown[] = []; + for (const cleanup of [ + () => db.execute(sql.raw(`DROP TRIGGER IF EXISTS ${trigger} ON chat_actions`)), + () => db.execute(sql.raw(`DROP FUNCTION IF EXISTS ${trigger}()`)), + () => db.delete(chatActions).where(eq(chatActions.companyId, companyId)), + () => db.delete(chatEndpoints).where(eq(chatEndpoints.companyId, companyId)), + () => db.delete(toolConnections).where(eq(toolConnections.companyId, companyId)), + () => db.delete(toolApplications).where(eq(toolApplications.companyId, companyId)), + () => db.delete(agents).where(eq(agents.companyId, companyId)), + () => db.delete(activityLog).where(eq(activityLog.companyId, companyId)), + () => db.delete(companyMemberships).where(eq(companyMemberships.companyId, companyId)), + () => db.delete(companies).where(eq(companies.id, companyId)), + () => db.delete(authUsers).where(eq(authUsers.id, actorId)), + ]) try { await cleanup(); } catch (error) { errors.push(error); } + vi.unstubAllEnvs(); + if (errors.length) throw new AggregateError(errors, "retirement SQL cleanup failure"); + }); + afterAll(async () => { + const closed = await Promise.allSettled([closeCheckpointDb(db, 1), closeFileStorageDb(db, 1), closeLivenessDb(db, 1)]); + await db?.$client.end({ timeout: 1 }); + const errors = closed.flatMap(value => value.status === "rejected" ? [value.reason] : []); + if (errors.length) throw new AggregateError(errors, "retirement auxiliary closure failure"); + }); + function signedRetainedHistoryCapabilities(): VerifiedIdleCapabilities { + const key=generateKeyPairSync("ed25519"),kid="retained-history-current",now=Math.floor(Date.now()/1000),generation="e".repeat(64); + const capabilities={incomingWake:{version:1,ownerId:randomUUID(),generation,routes:["slack-broker-events-v1"], + slackBroker:{version:1,instanceId:identity.instanceId,generation,bindingsFingerprint:slackConfiguredBindingsFingerprint([]),entries:[]}}}; + const encode=(value:unknown)=>Buffer.from(JSON.stringify(value)).toString("base64url"); + const body=encode({alg:"EdDSA",typ:CLOUD_CONTROL_JWS_TYPE,kid})+"."+encode({v:1,iss:"paperclip-cloud",aud:CLOUD_CONTROL_AUDIENCE, + sub:identity.instanceId,action:"task-drain:read",requestId:randomUUID(),iat:now,exp:now+60,idleCapabilities:capabilities}); + const claims=verifyCloudControlAssertion({compactJws:body+"."+sign(null,Buffer.from(body),key.privateKey).toString("base64url"),expectedAction:"task-drain:read", + env:{PAPERCLIP_CLOUD_STACK_ID:identity.instanceId,PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS:JSON.stringify({keys:[{...key.publicKey.export({format:"jwk"}),kid,use:"sig",alg:"EdDSA"}]})}}); + if(!claims.idleCapabilities)throw new Error("Actual signed retained-history capability absent"); + return claims.idleCapabilities as VerifiedIdleCapabilities; + } + async function finishedHistory(ownerAlreadyTransferred=false) { + // The original owner reaches its real normal transferred state. Additional + // rows explicitly represent already-committed historical custody, not live + // provider calls or a fabricated provider result in this tested journey. + if(!ownerAlreadyTransferred)await acceptSlackFormRetirementLocally(db,await row(),identity,compactJws); + const receipt=slackFormRetirementReceipt(binding,identity.environment,actorId); + const protocol=async(input:{operation:Record})=>input.operation.refreshRetirementReceipt?{retirementReceiptJws:compactJws} + :input.operation.acceptedRetirementReceipt?{acceptedRetirementDigest:receipt.receiptId}:{releasedRetirementDigest:receipt.receiptId}; + if(!ownerAlreadyTransferred){await transferSlackFormRetirement(db,await row(),identity,protocol);await transferSlackFormRetirement(db,await row(),identity,protocol);} + expect(slackFormRetirementHistoryReceipt(await row())).not.toBeNull(); + const history=Array.from({length:1005},(_,index)=>({id:`b1000000-0000-4000-8000-${String(index).padStart(12,"0")}`,companyId,endpointId, + kind:"slack_form_grant",providerActionId:"slack-form-grant-history:"+index,status:"cancelled", + payload:{binding,environment:identity.environment,subject:actorId},result:{phase:"transferred",nextAttemptAt:null, + retirement:{version:1,binding,instanceId:identity.instanceId,environment:identity.environment,subject:actorId,receiptDigest:receipt.receiptId,receipt}}})); + for(const entry of history)expect(slackFormRetirementHistoryReceipt(entry)).not.toBeNull(); + await db.insert(chatActions).values(history); + return{history,receipt}; + } + async function inspectHistory(abortBeforeRead?:number) { + const capabilities=signedRetainedHistoryCapabilities(),unregister=registerConnectionEventWake(db,()=>{}); + let reads=0,selects=0; + try { + return await db.transaction(async tx=>{ + const select=vi.spyOn(tx,"select"); + const started=performance.now(),remaining=()=>{ + const left=2000-(performance.now()-started);if(left<=0)throw new Error("Incoming wake inspection budget exhausted");return Math.max(1,Math.floor(left)); + }; + const beforeRead=async()=>{ + reads++;if(reads===abortBeforeRead)throw new Error("Controlled existing inspection budget exhausted before next page"); + if(reads*2>64)throw new Error("Incoming wake query budget exhausted"); + await tx.execute(sql`SELECT set_config('statement_timeout', ${`${remaining()}ms`}, true)`);remaining(); + }; + try { + const proof=await readSlackIdleIncomingProof(tx,{capabilities,identity,receiverBound:hasConnectionEventWakeReceiver(db),registeredOptions:()=>false,beforeRead,remaining}); + const check=idleChatHistoryChecksForOwner(proof?.completedActionIds).find(value=>value.reason==="chat_actions"); + if(!check)throw new Error("Actual chat action owner query absent"); + await beforeRead();const query=typeof check.query==="string"?sql.raw(check.query):check.query; + const blockers=await tx.execute<{count:number}>(sql`SELECT count(*)::int AS count FROM (${query}) retained_history_blockers`); + return{proof,blockers:blockers[0]!.count,reads}; + }finally{selects=select.mock.calls.length;select.mockRestore();} + }); + }finally{unregister();if(abortBeforeRead!==undefined){expect(reads).toBe(abortBeforeRead);expect(selects).toBe(abortBeforeRead-1);}} + } + it("more than one page of valid finished Slack custody remains provable by the real incoming and action owners",async()=>{ + const{history}=await finishedHistory();const result=await inspectHistory(); + expect(result.proof?.completedActionIds).toEqual([actionId,...history.map(entry=>entry.id)].sort()); + expect(result.proof?.endpointIds).toEqual([]);expect(result.proof?.connectionIds).toEqual([]); + expect(result.blockers).toBe(0);expect(result.reads).toBeGreaterThanOrEqual(5); + },15000); + for(const fault of ["unfinished","malformed","foreign_company"] as const)it(`a ${fault} receipt after the first completed history page still blocks the actual action owner`,async()=>{ + const{history,receipt}=await finishedHistory(),id="ffffffff-ffff-4fff-bfff-ffffffffffff",last=history.at(-1)!; + const payload=fault==="foreign_company"?{...last.payload,binding:{...binding,companyId:randomUUID()}}:last.payload; + const result=fault==="unfinished"?{phase:"issued",nextAttemptAt:null}:fault==="malformed" + ?{...last.result,retirement:{...last.result.retirement,receipt:{...receipt,receiptId:"not-a-retirement-digest"}}}:last.result; + await db.insert(chatActions).values({...last,id,providerActionId:"slack-form-grant-history:last",status:fault==="unfinished"?"issued":last.status,payload,result}); + const inspected=await inspectHistory();expect(inspected.proof?.completedActionIds).toHaveLength(1006); + expect(inspected.proof?.completedActionIds).not.toContain(id);expect(inspected.blockers).toBe(1); + },15000); + it("a deadline before the next history page refuses proof without issuing that read",async()=>{ + await finishedHistory();await expect(inspectHistory(4)).rejects.toThrow("Controlled existing inspection budget exhausted before next page"); + },15000); + + it("a finished session on a later page certifies only its exact earlier open and submit actions",async()=>{ + const ack=await acceptedAckForRefresh("ffffffff-ffff-4fff-bfff-fffffffffffe"); + const readAck=async()=>(await db.select().from(chatActions).where(eq(chatActions.id,ack.id)))[0]!; + const work=parseSlackFormAckWork((await readAck()).result?.formAck); + if(!work.receipt)throw new Error("Normal retained ACK fixture lost its receipt"); + const digest=slackFormFinalizedReceiptDigest(work.receipt); + const protocol=async(input:{operation:Record})=>input.operation.refreshAckReceipt?{ackReceiptJws:ack.currentJws} + :input.operation.acceptedAckReceipt?{acceptedReceiptDigest:digest,needsTerminalOriginal:false}:{releasedAckDigest:digest}; + await transferSlackFormAckReceipt(db,{companyId,actionId:ack.id,instanceId:identity.instanceId,protocol}); + await transferSlackFormAckReceipt(db,{companyId,actionId:ack.id,instanceId:identity.instanceId,protocol}); + const finished=await readAck();expect(parseSlackFormAckWork(finished.result?.formAck).phase).toBe("transferred"); + const openId="00000000-0000-4000-8000-000000000001",submitId="00000000-0000-4000-8000-000000000002"; + const payload={interactionId:finished.payload.interactionId,publicationId:finished.payload.publicationId}; + await db.insert(chatActions).values([ + {id:openId,companyId,endpointId,kind:"question_form_open",status:"processed",providerActionId:"slack-history-linked-open",payload}, + {id:submitId,companyId,endpointId,kind:"question_form_submit",status:"processed",providerActionId:"slack-history-linked-submit",payload}]); + await db.update(chatActions).set({payload:{...finished.payload,openActionRowId:openId,submitActionRowId:submitId}}).where(eq(chatActions.id,ack.id)); + await finishedHistory(true); + const proof=await inspectHistory();expect(proof.proof?.completedActionIds).toHaveLength(1009);expect(proof.blockers).toBe(0); + expect(proof.proof?.completedActionIds).toEqual(expect.arrayContaining([ack.id,openId,submitId])); + await db.update(chatActions).set({payload:{...payload,publicationId:"changed-retained-publication"}}).where(eq(chatActions.id,openId)); + const changed=await inspectHistory();expect(changed.proof?.completedActionIds).not.toContain(openId);expect(changed.blockers).toBe(1); + },15000); + + // Seed retained accepted ACK custody, as in the existing GC controls; this is not a provider-delivery proof. + async function acceptedAckForRefresh(appReceiptId=randomUUID()) { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retire = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: digest } : { releasedRetirementDigest: digest }; + await transferSlackFormRetirement(db, await row(), identity, retire); + await transferSlackFormRetirement(db, await row(), identity, retire); + const id = appReceiptId, operationId = "e".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const originalBody = JSON.stringify({ type: "block_actions", user: { id: "UTEST" } }); + const timestamp = String(Math.floor(Date.now() / 1000)); + const original = { rawBody: Buffer.from(originalBody).toString("base64"), timestamp, + signature: `v0=${createHmac("sha256", "controlled-signing-secret").update(`v0:${timestamp}:${originalBody}`).digest("hex")}` }; + const requestDigest = createHash("sha256").update(JSON.stringify([original.rawBody, original.timestamp, original.signature])).digest("hex"); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: id, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const signAck = (kid: string) => { + const pair = generateKeyPairSync("ed25519"), encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] })); + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + return body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + }; + const oldJws = signAck("ack-before-refresh"); + await db.insert(chatActions).values({ id, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, requestDigest, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent: { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: id, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: id, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "accepted", leaseId: "exact-refresh-lease", deliveryId, receiptJws: oldJws, receipt }, nextAttemptAt: now } }); + vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "9".repeat(64)); + await db.transaction(tx => retainSlackFormOpeningProof(tx, { companyId, appReceiptId: id, original, + operation: { version: 1, binding, grantId: "grant", id: operationId, externalId: "controlled-external-id", deliveryId, requestDigest, triggerDigest: "f".repeat(64), + userId: "UTEST", acceptedAt: now, triggerDeadline: now, phase: "rejected", view: null, closed: null, acknowledgement: null, rejection: null } })); + return { id, oldJws, currentJws: signAck("ack-after-refresh"), original }; + } + for (const kind of ["retirement", "ack"] as const) for (const boundary of ["before_refresh", "after_refresh", "after_concurrent_write"] as const) { + it(`${kind} owner failure ${boundary} retains exact custody and durable retry without overwriting another writer`, async () => { + let id: string, oldJws: string, currentJws: string, original: { rawBody: string; timestamp: string; signature: string } | undefined; + if (kind === "retirement") { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + id = actionId; oldJws = compactJws; currentJws = signCurrentReceipt("retirement-after-refresh"); + } else ({ id, oldJws, currentJws, original } = await acceptedAckForRefresh()); + const read = async () => (await db.select().from(chatActions).where(and(eq(chatActions.id, id), eq(chatActions.companyId, companyId))))[0]!; + const before = await read(), reported: string[] = []; + let refreshes = 0, transfers = 0, concurrent: typeof before | undefined; + const owner = { identity, resolveBotToken: async () => { throw new Error("retained work must not read credentials"); }, + report: (code: string) => { reported.push(code); }, protocol: async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt || input.operation.refreshAckReceipt) { + refreshes++; + if (boundary === "before_refresh") throw new Error("controlled failure before refresh"); + return kind === "retirement" ? { retirementReceiptJws: currentJws } : { ackReceiptJws: currentJws }; + } + transfers++; + expect(input.operation[kind === "retirement" ? "acceptedRetirementReceipt" : "acceptedAckReceipt"]).toBe(currentJws); + if (kind === "ack") expect(input.operation.originalOpening).toEqual(original); + if (boundary === "after_concurrent_write") { + const refreshed = await read(); + await db.update(chatActions).set({ status: "attention", payload: { ...refreshed.payload, protectedConcurrentWriter: true }, + result: { ...refreshed.result, nextAttemptAt: null, protectedConcurrentWriter: true }, updatedAt: new Date() }).where(eq(chatActions.id, id)); + concurrent = await read(); + } + throw new Error("controlled transfer failure after exact signature refresh"); + } }; + const started = Date.now(), due = await reconcilePendingSlackFormWork(db, owner), after = await read(); + expect(refreshes).toBe(1); expect(transfers).toBe(boundary === "before_refresh" ? 0 : 1); + expect(reported).toEqual(["form_owner_broker_unavailable"]); + if (boundary === "after_concurrent_write") { + expect(concurrent).toBeDefined(); expect(after).toEqual(concurrent); expect(due).toBeNull(); + } else { + expect(after.payload).toEqual(before.payload); expect(after.status).toBe(before.status); + const result = kind === "retirement" ? { ...before.result, retirementReceiptJws: boundary === "before_refresh" ? oldJws : currentJws } + : { ...before.result, formAck: { ...parseSlackFormAckWork(before.result?.formAck), receiptJws: boundary === "before_refresh" ? oldJws : currentJws } }; + expect(after.result).toEqual({ ...result, attempts: 1, failureCode: "form_owner_broker_unavailable", failureReason: null, nextAttemptAt: expect.any(String) }); + expect(due).toBe(Date.parse(String(after.result?.nextAttemptAt))); expect(due).toBeGreaterThanOrEqual(started + 5000); + if (due === null) throw new Error("Normal owner lost durable backoff"); + expect(await reconcilePendingSlackFormWork(db, owner, new Date(due - 1))).toBe(due); + expect(refreshes).toBe(1); expect(await read()).toEqual(after); + } + }, 15000); + } + it("commits original-subject acceptance before Cloud compaction and retries a lost transfer after reconstruction", async () => { + let retireCalls = 0, cleanupCalls = 0; + await reconcileSlackFormEnrollment(db, { actionId, identity, resolveBotToken: async () => { throw new Error("retirement must not read credentials"); }, protocol: async input => { + expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); + if (input.operation.kind === "retire") { retireCalls++; return { retired: true }; } + cleanupCalls++; return { cleaned: true, instanceId: identity.instanceId, environment: identity.environment, binding, retirementReceiptJws: compactJws }; + } }); + const accepted = await row(); expect(accepted.result?.phase).toBe("retirement_accepted"); expect(accepted.result?.retirementReceiptJws).toBe(compactJws); + let compacted = false, transfers = 0; const lost = new Error("lost actual transfer response"); + const protocol = async (input: { subject: string; companyId: string; operation: Record }) => { + expect(input.subject).toBe(actorId); expect(input.companyId).toBe(companyId); + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: compactJws }; + if (input.operation.releaseRetirementFact) return { releasedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + expect(input.operation).toEqual({ kind: "cleanup", binding, acceptedRetirementReceipt: compactJws }); + transfers++; compacted = true; if (transfers === 1) throw lost; + return { acceptedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + }; + const reported: string[] = [], owner = { identity, protocol, resolveBotToken: async () => { throw new Error("no new use"); }, report: (code: string) => { reported.push(code); } }; + const started = Date.now(), due = await reconcilePendingSlackFormWork(db, owner); + const pending = await row(); expect(compacted).toBe(true); expect(pending.payload).toEqual(accepted.payload); + expect(pending.result?.phase).toBe("retirement_accepted"); expect(pending.result?.retirementReceiptJws).toBe(compactJws); + expect(pending.result?.attempts).toBe(1); expect(reported).toEqual(["form_owner_broker_unavailable"]); + expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); expect(due).toBeGreaterThanOrEqual(started + 5000); + if (due === null) throw new Error("normal owner did not retain retry deadline"); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + expect(await reconcilePendingSlackFormWork(db, owner)).not.toBeNull(); + expect((await row()).result?.phase).toBe("retirement_gc_pending"); + expect(await reconcilePendingSlackFormWork(db, owner)).toBeNull(); + const complete = await row(); expect(slackFormRetirementHistoryReceipt(complete)?.subject).toBe(actorId); + expect(complete.result?.retirementReceiptJws).toBeUndefined(); expect(retireCalls).toBe(1); expect(cleanupCalls).toBe(1); expect(transfers).toBe(2); + }, 15000); + it("a lost cleanup reply before App acceptance keeps original debt and the normal owner retries it", async () => { + const original = await row(), lost = new Error("cleanup reply lost before local acceptance"); + let cleanups = 0; + const owner = { identity, resolveBotToken: async () => { throw new Error("no credential reads during retirement"); }, report: () => {}, protocol: async (input: { operation: Record }) => { + if (input.operation.kind === "retire") return { retired: true }; + cleanups++; if (cleanups === 1) throw lost; + return { cleaned: true, instanceId: identity.instanceId, environment: identity.environment, binding, retirementReceiptJws: compactJws }; + } }; + const due = await reconcilePendingSlackFormWork(db, owner), pending = await row(); + expect(pending.payload).toEqual(original.payload); expect(pending.result?.phase).toBe("retired_cleanup"); expect(pending.result?.retirementReceiptJws).toBeUndefined(); + expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); if (due === null) throw new Error("cleanup lost its retry deadline"); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + await reconcilePendingSlackFormWork(db, owner); expect((await row()).result?.phase).toBe("retirement_accepted"); expect(cleanups).toBe(2); + }, 15000); + it("acceptance SQL rollback preserves old proof debt and a stale snapshot cannot clear a successor", async () => { + const before = await row(); expect(actionId).toMatch(/^[a-f0-9-]+$/); + await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF NEW.id='${actionId}' AND NEW.result->>'phase'='retirement_accepted' THEN RAISE EXCEPTION 'controlled retirement acceptance rollback'; END IF; RETURN NEW; END $$`)); + await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON chat_actions FOR EACH ROW EXECUTE FUNCTION ${trigger}()`)); + await expect(acceptSlackFormRetirementLocally(db, before, identity, compactJws)).rejects.toThrow(); expect(await row()).toEqual(before); + await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON chat_actions`)); await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`)); + await db.update(chatActions).set({ result: { ...before.result, preservedSuccessor: "exact" } }).where(eq(chatActions.id, actionId)); + const successor = await row(); await expect(acceptSlackFormRetirementLocally(db, before, identity, compactJws)).rejects.toMatchObject({ code: "form_retirement_acceptance_changed" }); + expect(await row()).toEqual(successor); + }); + it("foreign company and a changed accepted digest cannot clear local responsibility", async () => { + const before = await row(); await expect(acceptSlackFormRetirementLocally(db, { ...before, companyId: randomUUID() }, identity, compactJws)).rejects.toMatchObject({ code: "form_retirement_authority_changed" }); + expect(await row()).toEqual(before); await acceptSlackFormRetirementLocally(db, before, identity, compactJws); const accepted = await row(); + await expect(transferSlackFormRetirement(db, accepted, identity, async input => input.operation.refreshRetirementReceipt ? { retirementReceiptJws: compactJws } : { acceptedRetirementDigest: "0".repeat(64) })).rejects.toMatchObject({ code: "form_retirement_transfer_unconfirmed" }); + expect(await row()).toEqual(accepted); + }); + it("current company-authorized Retry rearms retained debt and revoked authority cannot", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + await db.update(chatActions).set({ status: "attention", result: { ...(await row()).result, failureCode: "form_retirement_verifier_unavailable", nextAttemptAt: null } }).where(eq(chatActions.id, actionId)); + const management = await connectionManagementService(db).authorize({ type: "board", userId: actorId, source: "session", ignoreInstanceAdmin: true }, "connections:configure", + { type: "connection_management", companyId, connectionId, purpose: "chat", operation: "configure", readOnly: false }); + const before = await row(); expect((await retrySlackFormAcknowledgement(db, { companyId, endpointId, actionId, identity: () => identity, management })).state).toBe("scheduled"); + const pending = await row(); expect(pending.payload).toEqual(before.payload); expect(pending.result?.retirementReceiptJws).toBe(compactJws); expect(pending.result?.nextAttemptAt).not.toBeNull(); + await db.delete(companyMemberships).where(and(eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, actorId))); + await expect(retrySlackFormAcknowledgement(db, { companyId, endpointId, actionId, identity: () => identity, management })).rejects.toThrow(); + expect(await row()).toEqual(pending); + }); + it("current signature refresh changes only the JWS and a successor CAS cannot be overwritten", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const accepted = await row(), currentJws = signCurrentReceipt("B"); + const protocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: currentJws } : { acceptedRetirementDigest: "0".repeat(64) }; + await expect(transferSlackFormRetirement(db, accepted, identity, protocol)).rejects.toMatchObject({ code: "form_retirement_transfer_unconfirmed" }); + const refreshed = await row(); expect(refreshed.payload).toEqual(accepted.payload); expect(refreshed.status).toBe(accepted.status); + expect(refreshed.result).toEqual({ ...accepted.result, retirementReceiptJws: currentJws }); + const nextJws = signCurrentReceipt("C"); let transfers = 0; + await db.update(chatActions).set({ result: { ...refreshed.result, protectedSuccessor: true } }).where(eq(chatActions.id, actionId)); + const successor = await row(); + await expect(transferSlackFormRetirement(db, refreshed, identity, async input => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: nextJws }; + transfers++; return { acceptedRetirementDigest: slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId }; + })).rejects.toMatchObject({ code: "form_signature_refresh_changed" }); + expect(transfers).toBe(0); expect(await row()).toEqual(successor); + }); + it("durable transfer tracks GC and retries a lost release reply without the old signing key", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); const currentJws = signCurrentReceipt("B"); + const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + let releases = 0; const lost = new Error("lost canonical fact release response"); + const protocol = async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: currentJws }; + if (input.operation.acceptedRetirementReceipt) return { acceptedRetirementDigest: digest }; + expect(input.operation).toEqual({ kind: "cleanup", binding, releaseRetirementFact: { receiptId: digest } }); + releases++; if (releases === 1) throw lost; return { releasedRetirementDigest: digest }; + }; + await transferSlackFormRetirement(db, await row(), identity, protocol); + const transferred = await row(); expect(transferred.result?.phase).toBe("retirement_gc_pending"); expect(transferred.result?.nextAttemptAt).not.toBeNull(); expect(releases).toBe(0); + const owner = { identity, protocol, resolveBotToken: async () => { throw new Error("GC has no new credential use"); }, report: () => {} }; + const due = await reconcilePendingSlackFormWork(db, owner); const pending = await row(); expect(pending.result?.phase).toBe("retirement_gc_pending"); + expect(pending.result?.attempts).toBe(1); expect(due).toBe(Date.parse(String(pending.result?.nextAttemptAt))); if (due === null) throw new Error("GC debt lost its retry deadline"); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", ""); + while (Date.now() < due) await new Promise(resolve => setTimeout(resolve, due - Date.now())); + expect(await reconcilePendingSlackFormWork(db, owner)).toBeNull(); expect((await row()).result?.phase).toBe("transferred"); expect(releases).toBe(2); + }, 15000); + it("transfer rollback cannot issue GC before the actual App SQL commit", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); const currentJws = signCurrentReceipt("B"); + expect(actionId).toMatch(/^[a-f0-9-]+$/); + await db.execute(sql.raw(`CREATE FUNCTION ${trigger}() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF NEW.id='${actionId}' AND NEW.result->>'phase'='retirement_gc_pending' THEN RAISE EXCEPTION 'controlled transfer commit rollback'; END IF; RETURN NEW; END $$`)); + await db.execute(sql.raw(`CREATE TRIGGER ${trigger} BEFORE UPDATE ON chat_actions FOR EACH ROW EXECUTE FUNCTION ${trigger}()`)); + let transfers = 0, releases = 0; const digest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const protocol = async (input: { operation: Record }) => { + if (input.operation.refreshRetirementReceipt) return { retirementReceiptJws: currentJws }; + if (input.operation.acceptedRetirementReceipt) { transfers++; return { acceptedRetirementDigest: digest }; } + releases++; return { releasedRetirementDigest: digest }; + }; + await expect(transferSlackFormRetirement(db, await row(), identity, protocol)).rejects.toThrow(); + expect(transfers).toBe(1); expect(releases).toBe(0); expect((await row()).result?.phase).toBe("retirement_accepted"); + await db.execute(sql.raw(`DROP TRIGGER ${trigger} ON chat_actions`)); await db.execute(sql.raw(`DROP FUNCTION ${trigger}()`)); + await transferSlackFormRetirement(db, await row(), identity, protocol); expect((await row()).result?.phase).toBe("retirement_gc_pending"); expect(releases).toBe(0); + await transferSlackFormRetirement(db, await row(), identity, protocol); expect((await row()).result?.phase).toBe("transferred"); expect(releases).toBe(1); + }); + + it("a real paused endpoint polls reconstructed GC debt but completed exact session history stops recurring empty leases", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const retirementDigest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retirementProtocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: retirementDigest } : { releasedRetirementDigest: retirementDigest }; + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + expect((await row()).result?.phase).toBe("transferred"); + expect((await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)))[0]?.status).toBe("paused"); + const sessionId = randomUUID(), operationId = "c".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: sessionId, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const pair = generateKeyPairSync("ed25519"), kid = "session-canonical-control"; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), pair.privateKey).toString("base64url"); + const terminalIntent = { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: sessionId, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }; + await db.insert(chatActions).values({ id: sessionId, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: sessionId, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "gc_pending", leaseId: "exact-session-lease", deliveryId, receiptJws, receipt }, nextAttemptAt: now } }); + const signing = generateKeyPairSync("ed25519"), sealing = generateKeyPairSync("x25519"); + const signPrivateKey = signing.privateKey.export({ format: "jwk" }).d, sealPrivateKey = sealing.privateKey.export({ format: "jwk" }).d; + if (!signPrivateKey || !sealPrivateKey) throw new Error("normal controlled connector private keys absent"); + const connector = createPaperclipCloudConnector({ config: { baseUrl: "https://broker.example.test", instanceId: identity.instanceId, + environment: identity.environment, signPrivateKey, sealPrivateKey }, request: async () => { throw new Error("Unexpected external connector request"); } }); + const lease = vi.spyOn(connector, "leaseEvents").mockResolvedValue(null); + expect((await readSlackFormRetainedConsumer(db, connector)).owner).toMatchObject({ companyId, requiresDelivery: false }); + const pendingWorker = githubConnectionEventService(db, { connector, env: { PAPERCLIP_GITHUB_EVENT_POLLING: "1" } }); + await pendingWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(1); expect(pendingWorker.nextPollAt()).not.toBeNull(); + let releases = 0; + await transferSlackFormAckReceipt(db, { companyId, actionId: sessionId, instanceId: identity.instanceId, protocol: async values => { + expect(values.subject).toBe("github-bot-events"); expect(values.companyId).toBe(companyId); + expect(values.operation).toEqual({ kind: "receipt", binding, operationId, deliveryId, + releaseAckFact: { leaseId: "exact-session-lease", receiptDigest: slackFormFinalizedReceiptDigest(receipt), appReceiptId: sessionId } }); + releases++; return { releasedAckDigest: slackFormFinalizedReceiptDigest(receipt) }; + } }); + const [completed] = await db.select().from(chatActions).where(and(eq(chatActions.id, sessionId), eq(chatActions.companyId, companyId))); + expect(completed?.result?.formAck).toMatchObject({ phase: "transferred", leaseId: "exact-session-lease", receipt }); + expect(completed?.result?.nextAttemptAt).toBeNull(); expect(releases).toBe(1); + expect(await readSlackFormRetainedConsumer(db, connector)).toEqual({ owner: null, attention: [] }); + const completedWorker = githubConnectionEventService(db, { connector, env: { PAPERCLIP_GITHUB_EVENT_POLLING: "1" } }); + await completedWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(2); expect(completedWorker.nextPollAt()).toBeNull(); + await completedWorker.pollOnce(); expect(lease).toHaveBeenCalledTimes(2); + }); + it("bounded retained-consumer pages pass more than32 completed rows to discover an unfinished tail without dropping attention", async () => { + await acceptSlackFormRetirementLocally(db, await row(), identity, compactJws); + const retirementDigest = slackFormRetirementReceipt(binding, identity.environment, actorId).receiptId; + const retirementProtocol = async (input: { operation: Record }) => input.operation.refreshRetirementReceipt + ? { retirementReceiptJws: compactJws } : input.operation.acceptedRetirementReceipt + ? { acceptedRetirementDigest: retirementDigest } : { releasedRetirementDigest: retirementDigest }; + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + await transferSlackFormRetirement(db, await row(), identity, retirementProtocol); + expect((await row()).result?.phase).toBe("transferred"); + expect((await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)))[0]?.status).toBe("paused"); + const sessionId = randomUUID(), operationId = "c".repeat(64), deliveryId = `slack_form_${operationId}`, now = new Date().toISOString(); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: identity.environment, binding, + operationId, deliveryId, appReceiptId: sessionId, grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", + viewDigest: null, terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: now }); + const pair = generateKeyPairSync("ed25519"), kid = "session-canonical-control"; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), pair.privateKey).toString("base64url"); + const terminalIntent = { version: 1, binding, operationId, deliveryId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction", + appReceiptId: sessionId, view: null, outcome: "no_effect", submitReceiptId: null, rejectionDigest: receipt.terminalDigest, encryptedEvidence: null }; + await db.insert(chatActions).values({ id: sessionId, companyId, endpointId, kind: "slack_form_session", providerActionId: `slack-form-session:${operationId}`, + status: "failed", payload: { binding, environment: identity.environment, operationId, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "not_opened", terminalIntent, + cloudProofAcceptance: { version: 1, binding, operationId, appReceiptId: sessionId, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "gc_pending", leaseId: "exact-session-lease", deliveryId, receiptJws, receipt }, nextAttemptAt: now } }); + await transferSlackFormAckReceipt(db, { companyId, actionId: sessionId, instanceId: identity.instanceId, + protocol: async () => ({ releasedAckDigest: slackFormFinalizedReceiptDigest(receipt) }) }); + const [completed] = await db.select().from(chatActions).where(and(eq(chatActions.id, sessionId), eq(chatActions.companyId, companyId))); + if (!completed) throw new Error("Actual GC transition lost its session"); + for (let n = 1; n <= 33; n++) { + const id = `00000000-0000-4000-8000-${n.toString(16).padStart(12, "0")}`, op = n.toString(16).padStart(64, "0"), delivery = `slack_form_${op}`; + const historyReceipt = { ...receipt, operationId: op, deliveryId: delivery, appReceiptId: id }; + const body = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: identity.instanceId, receipt: historyReceipt }); + const historyJws = body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"); + await db.insert(chatActions).values({ ...completed, id, providerActionId: `slack-form-session:${op}`, + payload: { ...completed.payload, operationId: op }, result: { phase: "not_opened", + terminalIntent: { ...terminalIntent, operationId: op, deliveryId: delivery, appReceiptId: id }, + cloudProofAcceptance: { version: 1, binding, operationId: op, appReceiptId: id, outcome: "no_effect", terminalDigest: receipt.terminalDigest, acceptedAt: now }, + formAck: { version: 1, phase: "transferred", leaseId: "exact-session-lease", deliveryId: delivery, receiptJws: historyJws, receipt: historyReceipt }, nextAttemptAt: null } }); + } + const tailId = "ffffffff-ffff-4fff-8fff-ffffffffffff", tailOp = "f".repeat(64); + await db.insert(chatActions).values({ id: tailId, companyId, endpointId, kind: "slack_form_session", status: "processing", providerActionId: `slack-form-session:${tailOp}`, + payload: { binding, environment: identity.environment, operationId: tailOp, userId: "UTEST", grantId: "grant", publicationId: "publication", interactionId: "interaction" }, + result: { phase: "update_attempted", view: { id: "VTAIL", hash: "same-loading-hash", externalId: `pc-form-${tailOp}` } } }); + const connector = { instanceIdentity: () => identity }; + expect((await readSlackFormRetainedConsumer(db, connector)).owner).toEqual({ subject: "github-bot-events", companyId, binding, requiresDelivery: true }); + await db.insert(chatActions).values({ id: "00000000-0000-4000-8000-000000000000", companyId, endpointId, kind: "slack_form_session", status: "attention", providerActionId: "malformed-retained-page", + payload: { binding: { instanceId: identity.instanceId }, environment: identity.environment }, result: {} }); + const withMalformed = await readSlackFormRetainedConsumer(db, connector); + expect(withMalformed.owner).toMatchObject({ companyId, requiresDelivery: true }); + expect(withMalformed.attention).toContainEqual({ actionId: "00000000-0000-4000-8000-000000000000", code: "form_retained_owner_invalid" }); + expect(withMalformed.attention.length).toBeLessThanOrEqual(32); + }); + +}); diff --git a/server/src/http/body-limits.ts b/server/src/http/body-limits.ts index fde64e5daaa..44a7f6d2dbb 100644 --- a/server/src/http/body-limits.ts +++ b/server/src/http/body-limits.ts @@ -1,4 +1,5 @@ export const DEFAULT_JSON_BODY_LIMIT = "10mb"; +export const DEFAULT_JSON_BODY_LIMIT_BYTES = 10 * 1024 * 1024; export const CHAT_WEBHOOK_BODY_LIMIT = "1mb"; export const CHAT_WEBHOOK_BODY_LIMIT_BYTES = 1024 * 1024; export const PORTABLE_JSON_BODY_LIMIT = "64mb"; diff --git a/server/src/middleware/cloud-control.ts b/server/src/middleware/cloud-control.ts index 90a6ba00936..69d3edb290b 100644 --- a/server/src/middleware/cloud-control.ts +++ b/server/src/middleware/cloud-control.ts @@ -23,6 +23,7 @@ export function verifiedIdleCapabilities(req: Request): VerifiedIdleCapabilities const ACTIONS_BY_ENDPOINT: Record> = { "/api/instance/ephemeral-enrollment/prepare": { POST: "ephemeral-enrollment:prepare" }, "/api/instance/connector-events/wake": { POST: "connector-events:wake" }, + "/api/instance/task-drain/status": { POST: "task-drain:read-body" }, "/api/instance/task-drain": { GET: "task-drain:read", POST: "task-drain:start", @@ -64,7 +65,9 @@ export function cloudControlMiddleware(): RequestHandler { return; } try { - const claims = verifyCloudControlAssertion({ compactJws: assertion, expectedAction }); + const rawBody: unknown = Reflect.get(req, "rawBody"); + const claims = verifyCloudControlAssertion({ compactJws: assertion, expectedAction, + ...(Buffer.isBuffer(rawBody) ? { rawBody } : {}) }); controlClaims.set(req, claims); if (claims.idleCapabilities) idleCapabilities.set(req, claims.idleCapabilities as VerifiedIdleCapabilities); } catch (error) { diff --git a/server/src/middleware/cloud-tenant-actor.test.ts b/server/src/middleware/cloud-tenant-actor.test.ts index c0c13cc6c42..eff8f83b671 100644 --- a/server/src/middleware/cloud-tenant-actor.test.ts +++ b/server/src/middleware/cloud-tenant-actor.test.ts @@ -219,7 +219,7 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => { expect(insertedTables).toContain(companyMemberships); }); - it.each(["/api/health", "/api/instance/task-drain"])("counts unfinished authentication on exempt %s requests", async (path) => { + it.each(["/api/health", "/api/instance/task-drain", "/api/instance/task-drain/status"])("counts unfinished authentication on exempt %s requests", async (path) => { let enterWrite!: () => void, finishWrite!: () => void; const entered = new Promise(resolve => { enterWrite = resolve; }); const pending = new Promise(resolve => { finishWrite = resolve; }); @@ -235,8 +235,9 @@ describe("resolveCloudTenantActor (shared-pool hardening)", () => { res.json(await readIdleSleepSafety(emptyDb, () => ({ ...hold, draining: true, activeRuns: 0, pendingWakes: 0 }), Date.now, hold.ownerId, async () => "none")); }); + server.post("/api/instance/task-drain/status", (_req, res) => res.sendStatus(204)); trackIdleRequestHandlers(server); - const accepted = request(server).get(path).set(VALID_HEADERS).then(response => response); + const accepted = (path.endsWith("/status") ? request(server).post(path) : request(server).get(path)).set(VALID_HEADERS).then(response => response); try { await entered; expect(idleWorkSnapshot().active).toBeGreaterThan(0); diff --git a/server/src/middleware/idle-admission.ts b/server/src/middleware/idle-admission.ts index cbcf5cb7275..64fc2beb9a0 100644 --- a/server/src/middleware/idle-admission.ts +++ b/server/src/middleware/idle-admission.ts @@ -55,6 +55,7 @@ export function completeIdleReadOnlyResponse(req: Request, application: Applicat function isControlRequest(req: Request) { const path = req.path.replace(/\/$/, ""); return (path === "/api/instance/task-drain" && ["GET", "POST", "DELETE"].includes(req.method)) || + (path === "/api/instance/task-drain/status" && req.method === "POST") || (path === "/api/health" && ["GET", "HEAD"].includes(req.method)); } @@ -71,7 +72,8 @@ export const idleAdmissionMiddleware: RequestHandler = (req, res, next) => { res.set("Retry-After", "1").status(503).json({ error: "instance_preparing_to_sleep" }); return; } - if (controlRequest && req.method === "GET" && req.path.replace(/\/$/, "") === "/api/instance/task-drain") { + if (controlRequest && ((req.method === "GET" && req.path.replace(/\/$/, "") === "/api/instance/task-drain") + || (req.method === "POST" && req.path.replace(/\/$/, "") === "/api/instance/task-drain/status"))) { next(); return; } diff --git a/server/src/routes/chat-channels.ts b/server/src/routes/chat-channels.ts index caf44d8e435..d75e0f27acb 100644 --- a/server/src/routes/chat-channels.ts +++ b/server/src/routes/chat-channels.ts @@ -473,6 +473,11 @@ export function chatChannelRoutes(db: Db, options: ChatChannelRouteOptions) { ); }); + router.post("/chat-endpoints/:endpointId/provider-work/:actionId/retry", managementGuard("connections:configure", "POST /chat-endpoints/:endpointId/provider-work/:actionId/retry"), validate(z.object({}).strict()), async (req, res) => { + if (!(await assertEndpointManagementAccess(req, res))) return; + res.json(await service.retrySlackFormReceipt(endpointId(req), String(req.params.actionId), connectionRequestContext(req))); + }); + router.post("/chat-endpoints/:endpointId/test", managementGuard("connections:configure", "POST /chat-endpoints/:endpointId/test"), async (req, res) => { if (!(await assertEndpointManagementAccess(req, res))) return; res.json(await service.test(endpointId(req), undefined, connectionRequestContext(req))); diff --git a/server/src/routes/instance-settings.ts b/server/src/routes/instance-settings.ts index 3372b295792..260d507215f 100644 --- a/server/src/routes/instance-settings.ts +++ b/server/src/routes/instance-settings.ts @@ -1,7 +1,7 @@ import type { IdleDeadlineOwners } from "../services/idle-deadline-types.js"; -import { verifiedIdleCapabilities } from "../middleware/cloud-control.js"; +import { verifiedCloudControlClaims, verifiedIdleCapabilities } from "../middleware/cloud-control.js"; import { trackIdleWork } from "../services/task-admission.js"; -import { Router, type Request } from "express"; +import { Router, type Request, type RequestHandler } from "express"; import { companies, type Db } from "@paperclipai/db"; import { patchInstanceSettingsSchema, @@ -302,7 +302,7 @@ export function instanceSettingsRoutes( }, ); - router.get("/instance/task-drain", async (req, res) => { + const readTaskDrainStatus: RequestHandler = async (req, res) => { assertBoardOrgAccess(req); if (req.query.upgradeReadiness === "1") { // This covers all companies and must not expose instance-wide work to a member. @@ -348,6 +348,13 @@ export function instanceSettingsRoutes( return; } res.json(heartbeat.getTaskDrainStatus()); + }; + router.get("/instance/task-drain", readTaskDrainStatus); + router.post("/instance/task-drain/status", async (req, res, next) => { + if (req.actor.source !== "cloud_control" || verifiedCloudControlClaims(req)?.action !== "task-drain:read-body") { + throw forbidden("Cloud body-bound read authority is required"); + } + await readTaskDrainStatus(req, res, next); }); router.post( diff --git a/server/src/routes/openapi.ts b/server/src/routes/openapi.ts index 773d9acb745..ca8657c54eb 100644 --- a/server/src/routes/openapi.ts +++ b/server/src/routes/openapi.ts @@ -2553,6 +2553,19 @@ for (const operation of githubBotOperations) { }); } +registry.registerPath({ + "x-paperclip-transport": "rest", + "x-paperclip-connection-management": "connections:configure", + method: "post", + path: "/api/chat-endpoints/{endpointId}/provider-work/{actionId}/retry", + tags: ["chat-channels"], + summary: "Retry retained provider receipt work", + description: "Requires current company connection-management access. Rearms only the same authorized retained Slack receipt or cleanup responsibility. Does not create a new form or spend a trigger. Revoked, foreign, or changed ownership is refused.", + request: { params: z.object({ endpointId: z.string().uuid(), actionId: z.string().uuid() }), body: jsonBody(z.object({}).strict()) }, + responses: { 200: r.ok(z.object({ actionId: z.string().uuid(), state: z.enum(["scheduled", "already_scheduled", "complete"]) })), + 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden, 404: r.notFound, 409: r.conflict }, +}); + registry.registerPath({ "x-paperclip-transport": "rest", "x-paperclip-connection-management": "connections:configure", @@ -7312,6 +7325,17 @@ registry.registerPath({ responses: { 200: r.ok(), 401: r.unauthorized, 403: r.forbidden }, }); +registry.registerPath({ + method: "post", + path: "/api/instance/task-drain/status", + tags: ["instance"], + summary: "Read task-drain status with body-bound signed Cloud idle capabilities", + description: "Requires the task-drain:read-body Cloud control assertion bound to the exact raw JSON SHA256. The body alone and browser administrator access do not grant this authority. Uses the same status/idle-safety reader and owner/scheduled-wake semantics as GET; does not start or stop a drain.", + request: { body: jsonBody(z.object({ idleCapabilities: z.unknown().describe("Complete closed incoming-wake and deadline catalogs, validated by the existing capability parser after signature and exact raw-body binding") }).strict()), + query: z.object({ upgradeReadiness: z.literal("1").optional(), ownerId: z.string().uuid().optional(), idleSleepSafety: z.literal("1").optional(), scheduledWake: z.enum(["1", "2", "domain-deadlines-v1"]).optional() }) }, + responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 403: r.forbidden }, +}); + registry.registerPath({ method: "post", path: "/api/instance/task-drain", diff --git a/server/src/services/chat-channels.ts b/server/src/services/chat-channels.ts index bbc5a8b18bf..55fb04afc3f 100644 --- a/server/src/services/chat-channels.ts +++ b/server/src/services/chat-channels.ts @@ -1,3 +1,9 @@ +import { retrySlackFormAcknowledgement } from "./chat-channels/slack/form-retry.js"; +import { createSlackFormConnectorAccess, registerSlackFormWorkOwner, reconcilePendingSlackFormWork, stageSlackFormOwnerConfiguration } from "./chat-channels/slack/form-owner.js"; +import { createSlackFormDeliveryOwner } from "./chat-channels/slack/form-delivery.js"; +import { createSlackFormPublicationStage } from "./chat-channels/slack/form-publication.js"; +import { withSlackCloudAcceptedAt, SlackCloudReceiptError } from "./chat-channels/slack/cloud-event.js"; +import { currentSlackFormOwnerBinding, withSlackFormSubmissionEvidence } from "./chat-channels/slack/cloud-forms.js"; import { githubReviewCheckService } from "./chat-github-checks.js"; import { createSlackCloudEventIngress } from "./chat-channels/slack/cloud-ingress.js"; import { createSlackCloudEnrollment, stageSlackCloudEnrollmentIntent, registerSlackCloudEnrollmentOwner } from "./chat-channels/slack/cloud-enrollment.js"; @@ -844,7 +850,30 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { } const withCredentialMutationLease = chatCredentialMutationLease(db, options); + const slackForms = createSlackFormConnectorAccess(); + const handleSlackFormReceipt = createSlackFormDeliveryOwner(db, { + connector: slackForms.connector, protocol: slackForms.protocol, withCredentialMutationLease, resolveCredentials, + webhookUrl: publicId => `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${publicId}/slack`, + handleWebhook: (publicId, request, generation, acceptedAt) => withSlackCloudAcceptedAt(request, acceptedAt, + () => handleWebhook(publicId, "slack", request, { slackCloudGeneration: generation })), + }); + const unregisterSlackFormWork = registerSlackFormWorkOwner(db, now => reconcilePendingSlackFormWork(db, { + identity: slackForms.identity(), protocol: slackForms.protocol, connector: slackForms.connector, + resolveBotToken: async endpointId => { + const record = await endpointRecord(endpointId); + if (!record || record.endpoint.provider !== "slack") throw new SlackCloudReceiptError("form_endpoint_custody_missing", "attention"); + return withCredentialMutationLease(record.endpoint, async lease => { + const credentials = await resolveCredentials(record.endpoint, db, ["botToken"]); + await lease.assertOwned(); + if (!credentials.botToken) throw new SlackCloudReceiptError("form_bot_credential_unavailable", "attention"); + return credentials.botToken; + }); + }, + report: (reason, actionId) => logger.warn({ reason, actionId }, "Slack form custody requires attention"), + }, now)); const unregisterSlackCloudIngress = createSlackCloudEventIngress(db, { + connectorIdentity: () => slackForms.identity() ?? undefined, + handleFormReceipt: handleSlackFormReceipt, withCredentialMutationLease, resolveCredentials, runtimeGeneration: endpoint => runtimeGeneration(endpoint.setup), handleWebhook: (publicId, provider, request, generation) => handleWebhook(publicId, provider, request, { slackCloudGeneration: generation }), }); @@ -3022,12 +3051,11 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { const response = await handleGitHubWebhookEvent(endpoint, request, endpointRuntime, runtimeContext); if (response) return response; } - const response = await endpointRuntime.handleWebhook( - request, - undefined, - githubResponseDeadlineAt ?? undefined, - provider === "telegram" ? serviceReceivedAtMs : undefined, - ); + const invokeWebhook = () => endpointRuntime.handleWebhook(request, undefined, githubResponseDeadlineAt ?? undefined, + provider === "telegram" ? serviceReceivedAtMs : undefined); + const response = provider === "slack" + ? await withSlackFormSubmissionEvidence(request, endpoint.id, currentSlackFormOwnerBinding({ ...endpoint.setup }) !== null, invokeWebhook) + : await invokeWebhook(); let lifecyclePayload: unknown = null; if (response.ok) { try { @@ -3418,14 +3446,21 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { : null; } - const slackCloudEnrollment = createSlackCloudEnrollment(db, { resolveCredentials, runtimeGeneration }); + const slackCloudEnrollment = createSlackCloudEnrollment(db, { resolveCredentials, runtimeGeneration, + onCommitted: async (tx, values) => { await stageSlackFormOwnerConfiguration(tx, values.endpointId, values.active, values.subject, + { instanceId: values.instanceId, environment: values.environment }); }, + }); const unregisterSlackCloudEnrollment = registerSlackCloudEnrollmentOwner(db, async endpointId => { const record = await endpointRecord(endpointId); if (!record) throw new Error("Slack enrollment endpoint is absent"); await withCredentialMutationLease(record.endpoint, lease => slackCloudEnrollment(endpointId, undefined, lease)); }); const { configure, configureWithCredentialLease, runtimeConfig } = createChatConfiguration(db, { - stageConfigurationEffects: stageSlackCloudEnrollmentIntent, + stageConfigurationEffects: async (tx, endpointId, actorUserId) => { + await stageSlackCloudEnrollmentIntent(tx, endpointId, actorUserId); + const identity = slackForms.identity(); + if (identity) await stageSlackFormOwnerConfiguration(tx, endpointId, false, actorUserId ?? "", identity); + }, endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime, nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials, @@ -3493,6 +3528,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { get publicationTransportPreparationBarrier() { return options.publicationTransportPreparationBarrier; }, }); const publicationWorker = createChatPublicationWorker(db, { + prepareSlackFormPublication: createSlackFormPublicationStage(db), CAPABILITIES, authorizeRetainedChatSourcePublication, claimPublicationTransportAuthorization, closeProgressToReplace, commitTaskControlCompletion, currentTaskControlPayload, endpointRecord, ensureDurablePublicationTransport, finalizePublicationFailure, hasCommittedTaskControlCompletion, @@ -3510,6 +3546,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, get heartbeat() { return options.heartbeat; }, }); const { handleAction, handleModalSubmit, reconcileTerminalConfirmationActions, registerTerminalConfirmationWork } = createChatInteractionCallbacks(db, { + withCredentialMutationLease, heartbeat: options.heartbeat, CAPABILITIES, MAX_ERROR_TEXT, actionThreadMatchesConversation, nonDirectDestinationAllowed, sameTeamsThreadIdentity, ensurePrincipal, endpointRecord, runtimeCallbackRecord, runtimeCallbackEndpoint, @@ -3563,6 +3600,11 @@ return { configure, inspectPhoton, test, + retrySlackFormReceipt: async (endpointId: string, actionId: string, management?: ConnectionManagementContext) => { + if (!management) throw forbidden("Connection management authority is required"); + return retrySlackFormAcknowledgement(db, { companyId: management.resource.companyId, endpointId, actionId, + identity: slackForms.identity, management }); + }, finishSlackSetup: (endpointId: string, userId: string, managementContext?: ConnectionManagementContext) => test(endpointId, { optionalSlackTestForUser: userId }, managementContext), setupTestStatus, handleWebhook, @@ -3634,6 +3676,7 @@ return { voice.unregisterAgentTools(); unregisterGitHubCloudIngress(); unregisterSlackCloudIngress(); + unregisterSlackFormWork(); unregisterSlackCloudEnrollment(); unregisterCommittedResponseAuthority(); await Promise.allSettled([...publicationEndpointTasks.values()]); diff --git a/server/src/services/chat-channels/idle-history-proof.ts b/server/src/services/chat-channels/idle-history-proof.ts index 992c71d8645..31e23ecd1db 100644 --- a/server/src/services/chat-channels/idle-history-proof.ts +++ b/server/src/services/chat-channels/idle-history-proof.ts @@ -1,3 +1,4 @@ +import { sql, type SQL } from "drizzle-orm"; import { finishedIdleRun, finishedIdleWake } from "../idle-work-proof.js"; import { completedGithubAction } from "./github/idle-history-proof.js"; import { completedSlackAction } from "./slack/idle-history-proof.js"; @@ -8,15 +9,19 @@ import { completedVoiceAction } from "../voice/idle-history-proof.js"; import { completedTeamsAction } from "./microsoft-teams/idle-history-proof.js"; import { completedTelegramAction } from "./telegram/idle-history-proof.js"; +function actionGraph(a: string): string { + return `jsonb_typeof(${a}.payload) = 'object' + AND EXISTS (SELECT 1 FROM chat_endpoints e WHERE e.id = ${a}.endpoint_id AND e.company_id = ${a}.company_id) + AND (${a}.delivery_id IS NULL OR EXISTS (SELECT 1 FROM chat_deliveries d + WHERE d.id = ${a}.delivery_id AND d.company_id = ${a}.company_id AND d.endpoint_id = ${a}.endpoint_id))`; +} + /** A processed inbound receipt can still own an unconsumed wake. */ function completedAction(a: string): string { const wake = `EXISTS (SELECT 1 FROM agent_wakeup_requests w WHERE w.company_id = ${a}.company_id AND w.id = ${a}.id AND w.agent_id::text = ${a}.payload->>'agentId' AND w.payload->>'issueId' = ${a}.payload->>'issueId' AND ${finishedIdleWake('w')})`; - return `jsonb_typeof(${a}.payload) = 'object' - AND EXISTS (SELECT 1 FROM chat_endpoints e WHERE e.id = ${a}.endpoint_id AND e.company_id = ${a}.company_id) - AND (${a}.delivery_id IS NULL OR EXISTS (SELECT 1 FROM chat_deliveries d - WHERE d.id = ${a}.delivery_id AND d.company_id = ${a}.company_id AND d.endpoint_id = ${a}.endpoint_id)) + return `${actionGraph(a)} AND CASE WHEN ${a}.kind = 'inbound_wakeup' THEN (${a}.status IN ('processed', 'failed') AND ${wake}) OR (${a}.status = 'failed' AND ${a}.result->>'code' = 'inbound_wakeup_delivery_rejected' @@ -127,3 +132,18 @@ export const idleChatHistoryChecks = [ AND (r.native_issue_id = h.conversation_id OR r.context_snapshot->>'issueId' = h.conversation_id::text) AND ${finishedIdleRun('r')}))) IS NOT TRUE` }, ] as const; + +export function idleChatHistoryChecksForOwner(completedActionIds: readonly string[] = []): readonly { reason: string; query: string | SQL }[] { + if (!completedActionIds.length) return idleChatHistoryChecks; + const ids = [...new Set(completedActionIds)]; + if (ids.some(id => !/^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$/i.test(id))) throw new Error("Invalid completed chat action identity"); + const normal = completedAction("a"); + // One bound UUID array; no retained-history-sized SQL literal/parameter list. + const completed = sql`(${sql.raw(normal)}) OR (${sql.raw(actionGraph("a"))} AND a.id IN (SELECT completed.id FROM unnest(${sql.param(ids)}::uuid[]) AS completed(id)))`; + return idleChatHistoryChecks.map(check => { + if (check.reason !== "chat_actions" && check.reason !== "chat_deliveries") return check; + const pieces = check.query.split(normal); + if (pieces.length !== 2) throw new Error("Chat history completion predicate changed"); + return { ...check, query: sql`${sql.raw(pieces[0]!)}${completed}${sql.raw(pieces[1]!)}` }; + }); +} diff --git a/server/src/services/chat-channels/interactions/callbacks.ts b/server/src/services/chat-channels/interactions/callbacks.ts index 24dfb13fede..1db05e4f726 100644 --- a/server/src/services/chat-channels/interactions/callbacks.ts +++ b/server/src/services/chat-channels/interactions/callbacks.ts @@ -1,3 +1,9 @@ +import { stageSlackFormSubmission } from "../slack/form-session.js"; +import { slackFormSubmissionEvidence } from "../slack/cloud-forms.js"; +import { slackFormOpenContext } from "../slack/cloud-forms.js"; +import { stageSlackFormOpenCallback, confirmSlackFormOpenCallback, isRetryableSlackFormUpdate } from "../slack/form-open-callback.js"; +import { SlackCloudReceiptError } from "../slack/cloud-event.js"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; import { notifyChatEndpointConfiguration } from "../configuration-work-notifications.js"; import { notifyChatConnectionHealth } from "../connection-health-notifications.js"; import { slackQuestionFormValidationError } from "../slack/question-forms.js"; @@ -115,6 +121,7 @@ import { createChatConfirmationActions } from "./confirmation-actions.js"; type ResourceRow = typeof chatEndpointResources.$inferSelect; type PrincipalResolution = { principal: typeof chatExternalPrincipals.$inferSelect; userId: string | null; linkedDenied: boolean }; type InteractionCallbacksOptions = { + withCredentialMutationLease?: ReturnType; heartbeat: import("../../issue-assignment-wakeup.js").IssueAssignmentWakeupDeps; CAPABILITIES: Record; MAX_ERROR_TEXT: number; @@ -986,6 +993,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb await options.questionFormOpenAuthorizationBarrier?.(); let providerAttempted = false; let attemptActionId: string | null = null; + const cloudForm = event.provider === "slack" ? slackFormOpenContext() : undefined; try { const attemptProviderActionId = event.event.triggerId ? `question_form_open_attempt:${createHash("sha256") @@ -998,7 +1006,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb ) .digest("hex")}` : null; - const preflight = await db.transaction(async (tx) => { + const runPreflight = (lease?: CredentialMutationLeaseGuard) => db.transaction(async (tx) => { await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: record.endpoint.id, @@ -1037,6 +1045,9 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb }) .onConflictDoNothing() .returning({ id: chatActions.id }); + if (cloudForm) return stageSlackFormOpenCallback(tx, { context: cloudForm, lease, resolved, + companyId: record.endpoint.companyId, endpointId: record.endpoint.id, conversationId: conversation.id, + principalId: principal.principal.id, attemptProviderActionId, attemptActionId: attempt?.id ?? null }); return attempt ? { kind: "ready" as const, @@ -1045,6 +1056,10 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb } : { kind: "duplicate" as const }; }); + if (cloudForm && !options.withCredentialMutationLease) throw new SlackCloudReceiptError("form_credential_lease_unavailable", "retry"); + const preflight = cloudForm + ? await options.withCredentialMutationLease!(record.endpoint, lease => runPreflight(lease)) + : await runPreflight(); if (preflight.kind === "denied") return deny(safelyKnown); if (preflight.kind === "duplicate") { if (event.provider === "discord") { @@ -1097,6 +1112,8 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb if (event.provider === "discord") return deny(safelyKnown); return; } + if (cloudForm) await db.transaction(tx => confirmSlackFormOpenCallback(tx, { context: cloudForm, + companyId: record.endpoint.companyId, endpointId: record.endpoint.id, principalId: principal.principal.id })); if (attemptActionId) { const [settledAttempt] = await db .update(chatActions) @@ -1123,6 +1140,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb return deny(safelyKnown); } if (!providerAttempted) throw error; + if (isRetryableSlackFormUpdate(cloudForm, error)) throw error; await recordModalOpenFailure( record.endpoint, event, @@ -1713,6 +1731,12 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb } return { action: "clear" }; }; + const persistSlackFormTerminal = async (tx: DbTransaction, resolved: IssueThreadInteraction) => { + if (event.provider !== "slack" || !slackFormSubmissionEvidence()) return; + await stageSlackFormSubmission(tx, { companyId: record.endpoint.companyId, endpointId: record.endpoint.id, + conversationId: conversation.id, principalId: principal.principal.id, authorizedUserId: principal.userId!, + runtimeGeneration: runtimeContext.generation, submitReceiptId: loaded.actionRowId, publicationId: originalPublication.id, resolved }); + }; if (loaded.status === "processed") { if (!isExactProcessedReplay(loaded)) { return deny( @@ -1724,6 +1748,13 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb loaded.payload, ); } + if (event.provider === "slack" && slackFormSubmissionEvidence()) { + await db.transaction(async tx => { + await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: event.endpointId, + expectedUserId: principal.userId!, principalId: principal.principal.id, runtimeContext }); + await persistSlackFormTerminal(tx, interaction!); + }); + } if (event.event.relatedThread) { await recordCurrentMicrosoftTeamsRoute( record.endpoint, @@ -1874,6 +1905,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb interactionId: resolved.id, }); if (!completed) throw denialError(); + await persistSlackFormTerminal(tx, resolved); await tx .update(chatActions) .set({ @@ -1959,6 +1991,12 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb if (!currentInteraction || currentInteraction.status === "pending") { return false; } + if (event.provider === "slack" && slackFormSubmissionEvidence() && currentInteraction.status === "answered" + && currentInteraction.resolvedByUserId === principal.userId) { + await requireCurrentExternalActionAuthorization(tx, { conversationId: conversation.id, endpointId: event.endpointId, + expectedUserId: principal.userId!, principalId: principal.principal.id, runtimeContext }); + await persistSlackFormTerminal(tx, currentInteraction); + } await enqueueTerminalIssueInteractionChatPublications( tx, currentInteraction, diff --git a/server/src/services/chat-channels/publications/failures.ts b/server/src/services/chat-channels/publications/failures.ts index c7e22f1c680..777bac1fe34 100644 --- a/server/src/services/chat-channels/publications/failures.ts +++ b/server/src/services/chat-channels/publications/failures.ts @@ -1,3 +1,4 @@ +import type { ChatPublicationPreparationWait } from "./preparation.js"; import { notifyChatResponseAvailability } from "../../chat-response-availability.js"; import { notifyChatEndpointConfiguration } from "../configuration-work-notifications.js"; import { notifyChatConnectionHealth } from "../connection-health-notifications.js"; @@ -58,14 +59,19 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur providerAccepted = false, runtimeContext?: LifecycleRuntimeFence | null, credentialLease?: CredentialMutationLeaseGuard, + providerInvocation: "possible" | "not_started" = "possible", + preparationWait?: ChatPublicationPreparationWait, ): Promise { const attempts = publication.attempts + 1; + const waitingBeforeSend = !providerAccepted && providerInvocation === "not_started" && preparationWait?.kind === "wait"; const disposition = providerAccepted ? { kind: "delivery_unknown" as const, reason: "Provider accepted the publication, but Paperclip could not confirm its durable result", } + : waitingBeforeSend && preparationWait + ? { kind: "retry" as const, retryAfterMs: preparationWait.retryAfterMs, reason: preparationWait.reason } : error instanceof NativeChatReviewPresentationContentionError ? { kind: "retry" as const, retryAfterMs: 250, reason: error.message } : error instanceof FailedChatRetryPublicationReadError @@ -88,7 +94,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur ), reason: error.message, } - : classifyChatPublicationError(error, attempts); + : classifyChatPublicationError(error, attempts, providerInvocation); const failure = redactSensitiveText(disposition.reason).slice( 0, MAX_ERROR_TEXT, @@ -100,6 +106,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur const terminalRetry = disposition.kind === "retry" && disposition.providerRateLimit !== true && + !waitingBeforeSend && attempts >= 5; const attachmentFailure = attachmentFailureKind(publication.payload); const attachmentFailureRuntimeContext = @@ -127,6 +134,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur .update(chatPublications) .set({ state, + ...(waitingBeforeSend ? { attempts: publication.attempts } : {}), nextAttemptAt: disposition.kind === "retry" && !terminalRetry ? new Date(Date.now() + disposition.retryAfterMs) @@ -161,7 +169,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur : state === "cancelled" ? "task_control_destination_unavailable" : "task_control_transport_failed", - attempts, + attempts: waitingBeforeSend ? publication.attempts : attempts, retryable: state === "retry", }, updatedAt: new Date(), diff --git a/server/src/services/chat-channels/publications/preparation.ts b/server/src/services/chat-channels/publications/preparation.ts index 54df34ef5bd..6bd11b3134b 100644 --- a/server/src/services/chat-channels/publications/preparation.ts +++ b/server/src/services/chat-channels/publications/preparation.ts @@ -24,6 +24,13 @@ import { splitNativePublicationText, } from "../../chat-publication-text-parts.js"; +/** Returned only before provider invocation; waiting does not consume a delivery attempt. */ +export type ChatPublicationPreparationWait = Readonly<{ + kind: "wait"; + reason: string; + retryAfterMs: number; +}>; + type PublicationPreparationOptions = { runPublicationToReplace: ReturnType["runPublicationToReplace"]; publicationTransportPreparationBarrier?: (input: { publicationId: string }) => Promise; diff --git a/server/src/services/chat-channels/publications/worker.ts b/server/src/services/chat-channels/publications/worker.ts index 831a041fd39..93085a937bd 100644 --- a/server/src/services/chat-channels/publications/worker.ts +++ b/server/src/services/chat-channels/publications/worker.ts @@ -1,3 +1,4 @@ +import type { ChatPublicationPreparationWait } from "./preparation.js"; import { notifyChatResponseAvailability } from "../../chat-response-availability.js"; import { notifyVoicePublication } from "../../voice/voice-reply-notifications.js"; import { createGitHubPublicationPolicy } from "../github/publication-policy.js"; @@ -90,6 +91,7 @@ import type { createChatPublicationPreparation } from "./preparation.js"; import type { createChatPublicationTransport } from "./transport.js"; type PublicationWorkerOptions = Pick, "runOwnershipMilestoneSupersessionReason" | "runPublicationToReplace" | "receiptReactionCompletionRunId" | "inboundWakePublicationToReplace" | "interactionResolutionPublicationToReplace" | "interactionPromptPublicationToReplace" | "progressSupersededByFastResponse" | "runProgressSupersededByPublishedInteraction" | "taskStatusPublicationToReplace" | "closeProgressToReplace"> & { CAPABILITIES: Record; + prepareSlackFormPublication?: ReturnType; authorizeRetainedChatSourcePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise; claimPublicationTransportAuthorization: ReturnType["claimPublicationTransportAuthorization"]; commitTaskControlCompletion: ReturnType["commitTaskControlCompletion"]; @@ -424,6 +426,8 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp const settlePublicationFailure = async ( error: unknown, credentialLease?: CredentialMutationLeaseGuard, + providerInvocation: "possible" | "not_started" = "possible", + preparationWait?: ChatPublicationPreparationWait, ) => { try { await finalizePublicationFailure( @@ -432,6 +436,8 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp providerAccepted, publicationRuntimeContext, credentialLease ?? publicationLease, + providerInvocation, + preparationWait, ); } catch (finalizationError) { // If the provider accepted the message, a failed receipt write @@ -782,6 +788,19 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp runtimeContext: currentPublicationRuntimeContext, }) : null; + let stageSlackForm: Exclude>>, ChatPublicationPreparationWait> | undefined; + try { + const preparation = await options.prepareSlackFormPublication?.(authorizationClaim.endpoint, publication); + if (preparation == null || typeof preparation === "function") { + stageSlackForm = preparation; + } else { + await settlePublicationFailure(new Error(preparation.reason), credentialLease, "not_started", preparation); + return; + } + } catch (error) { + await settlePublicationFailure(error, credentialLease, "not_started"); + return; + } const sent = await postSafePublication({ endpoint: authorizationClaim.endpoint, conversation: authorizationClaim.conversation, @@ -886,6 +905,7 @@ export function createChatPublicationWorker(db: Db, options: PublicationWorkerOp "Chat publication ownership changed before commit", ); } + await stageSlackForm?.(tx); if (telegramDraft) { const completedDraft = await tx .update(chatActions) diff --git a/server/src/services/chat-channels/runtime-types.ts b/server/src/services/chat-channels/runtime-types.ts index 4b9e9e5f5ab..bae9a9221f4 100644 --- a/server/src/services/chat-channels/runtime-types.ts +++ b/server/src/services/chat-channels/runtime-types.ts @@ -69,6 +69,7 @@ export type InternalSetupState = ChatEndpointSetupState & { nextAttemptAt: string | null; failureCode?: string; }; + slackCloudForms?: import("./slack/cloud-forms.js").SlackFormBinding; slackCloudEvents?: { version: 1; bindingId: string; diff --git a/server/src/services/chat-channels/slack/ack-receipt.ts b/server/src/services/chat-channels/slack/ack-receipt.ts new file mode 100644 index 00000000000..15ac8eda507 --- /dev/null +++ b/server/src/services/chat-channels/slack/ack-receipt.ts @@ -0,0 +1,67 @@ +import { createHash } from "node:crypto"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormBinding } from "./cloud-forms.js"; + +export type SlackFormFinalizedReceipt = Readonly<{ + version: 1; + kind: "slack_form_finalized"; + binding: SlackFormBinding; + environment: "development" | "test" | "staging" | "production"; + operationId: string; + deliveryId: string; + appReceiptId: string; + grantId: string; + publicationId: string; + interactionId: string; + userId: string; + viewDigest: string | null; + terminalDigest: string; + outcome: "submitted" | "closed" | "no_effect"; + finalizedAt: string; +}>; + +function invalid(): never { throw new Error("slack_form_ack_receipt_invalid"); } +export function parseSlackFormFinalizedReceipt(value: unknown): SlackFormFinalizedReceipt { + if (!value || typeof value !== "object" || Array.isArray(value)) return invalid(); + if (Buffer.byteLength(JSON.stringify(value), "utf8") > 8192) return invalid(); + const field = (key: string): unknown => Reflect.get(value, key); + const keys = ["version", "kind", "binding", "environment", "operationId", "deliveryId", "appReceiptId", "terminalDigest", "outcome", "finalizedAt", "grantId", "publicationId", "interactionId", "userId", "viewDigest"]; + if (Object.keys(value).sort().join() !== keys.sort().join()) return invalid(); + const binding = parseSlackFormBinding(field("binding")); + const environment = field("environment"); + const operationId = field("operationId"), deliveryId = field("deliveryId"); + const appReceiptId = field("appReceiptId"), terminalDigest = field("terminalDigest"); + const grantId = field("grantId"), publicationId = field("publicationId"), interactionId = field("interactionId"), userId = field("userId"), viewDigest = field("viewDigest"); + const outcome = field("outcome"), finalizedAt = field("finalizedAt"); + if (field("version") !== 1 || field("kind") !== "slack_form_finalized" + || environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production" + || typeof operationId !== "string" || !/^[a-f0-9]{64}$/.test(operationId) + || typeof deliveryId !== "string" || deliveryId !== `slack_form_${operationId}` || typeof appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(appReceiptId) + || typeof grantId !== "string" || !grantId.length || grantId.length > 200 + || typeof publicationId !== "string" || !publicationId.length || publicationId.length > 200 + || typeof interactionId !== "string" || !interactionId.length || interactionId.length > 200 + || typeof userId !== "string" || !userId.length || userId.length > 200 + || viewDigest !== null && (typeof viewDigest !== "string" || !/^[a-f0-9]{64}$/.test(viewDigest)) + || typeof terminalDigest !== "string" || !/^[a-f0-9]{64}$/.test(terminalDigest) + || outcome !== "submitted" && outcome !== "closed" && outcome !== "no_effect" + || typeof finalizedAt !== "string" || !Number.isFinite(Date.parse(finalizedAt))) return invalid(); + return Object.freeze({ version: 1, kind: "slack_form_finalized", binding: Object.freeze(binding), environment, + operationId, deliveryId, appReceiptId, grantId, publicationId, interactionId, userId, viewDigest, terminalDigest, outcome, finalizedAt }); +} + +export function assertSameSlackFormFinalization(actual: SlackFormFinalizedReceipt, expected: SlackFormFinalizedReceipt): void { + if (!sameSlackFormBinding(actual.binding, expected.binding) + || actual.environment !== expected.environment || actual.operationId !== expected.operationId + || actual.deliveryId !== expected.deliveryId || actual.appReceiptId !== expected.appReceiptId + || actual.grantId !== expected.grantId || actual.publicationId !== expected.publicationId + || actual.interactionId !== expected.interactionId || actual.userId !== expected.userId || actual.viewDigest !== expected.viewDigest + || actual.terminalDigest !== expected.terminalDigest || actual.outcome !== expected.outcome + || actual.finalizedAt !== expected.finalizedAt) return invalid(); +} + +export function slackFormFinalizedReceiptDigest(receipt: SlackFormFinalizedReceipt): string { + const b = receipt.binding; + return createHash("sha256").update(JSON.stringify(["slack-form-finalized-receipt/v1", b.version, b.instanceId, b.companyId, b.endpointId, + b.publicId, b.connectionId, b.runtimeGeneration, b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint, + receipt.environment, receipt.operationId, receipt.deliveryId, receipt.appReceiptId, receipt.grantId, receipt.publicationId, + receipt.interactionId, receipt.userId, receipt.viewDigest, receipt.terminalDigest, receipt.outcome, receipt.finalizedAt]), "utf8").digest("hex"); +} diff --git a/server/src/services/chat-channels/slack/ack-retirement-receipt.ts b/server/src/services/chat-channels/slack/ack-retirement-receipt.ts new file mode 100644 index 00000000000..0627f1aa9ac --- /dev/null +++ b/server/src/services/chat-channels/slack/ack-retirement-receipt.ts @@ -0,0 +1,60 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { isDeepStrictEqual } from "node:util"; +import { createHash } from "node:crypto"; +import { parseSlackFormBinding as parseFormBinding, parseSlackFormPublicEnrollment, type SlackFormBinding as FormBinding } from "./cloud-forms.js"; +function closedRecord(value: unknown, keys: readonly string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== [...keys].sort().join()) throw new Error("Invalid retirement receipt shape"); + return Object.fromEntries(Object.entries(value)); +} +const formOwnerId = (b: FormBinding) => `slack-form-owner:${createHash("sha256").update(JSON.stringify([b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, b.runtimeGeneration, b.credentialFingerprint])).digest("hex")}`; + +export type SlackFormRetirementReceipt = Readonly<{ + version: 1; kind: "slack_form_retired"; binding: FormBinding; + environment: "development" | "test" | "staging" | "production"; + subject: string; ownerId: string; receiptId: string; +}>; +function identity(binding: FormBinding, environment: string, subject: string) { + const b = binding; + return ["slack-form-retirement/v1", b.version, b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, + b.runtimeGeneration, b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint, environment, subject, formOwnerId(binding)]; +} +export function slackFormRetirementReceipt(binding: FormBinding, environment: string, subject: string): SlackFormRetirementReceipt { + const parsed = parseFormBinding(binding); + return parseSlackFormRetirementReceipt({ version: 1, kind: "slack_form_retired", binding: parsed, environment, subject, + ownerId: formOwnerId(parsed), receiptId: createHash("sha256").update(JSON.stringify(identity(parsed, environment, subject))).digest("hex") }); +} +export function parseSlackFormRetirementReceipt(value: unknown): SlackFormRetirementReceipt { + const v = closedRecord(value, ["version", "kind", "binding", "environment", "subject", "ownerId", "receiptId"]); + if (Buffer.byteLength(JSON.stringify(value), "utf8") > 8192 || v.version !== 1 || v.kind !== "slack_form_retired" + || typeof v.subject !== "string" || !v.subject.length || v.subject.length > 200 + || !["development", "test", "staging", "production"].includes(String(v.environment))) throw new Error("Invalid Slack retirement receipt"); + const binding = parseFormBinding(v.binding), environment = v.environment; + if (environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production") throw new Error("Invalid Slack retirement environment"); + const receiptId = createHash("sha256").update(JSON.stringify(identity(binding, environment, v.subject))).digest("hex"); + if (v.ownerId !== formOwnerId(binding) || v.receiptId !== receiptId) throw new Error("Slack retirement identity changed"); + return Object.freeze({ version: 1, kind: "slack_form_retired", binding: Object.freeze(binding), environment, subject: v.subject, + ownerId: formOwnerId(binding), receiptId }); +} + +export type SlackRetirementHistoryRow = { kind: string; companyId: string; endpointId: string; providerActionId?: string; status: string; + payload: Record; result: Record | null }; +export function slackFormRetirementHistoryReceipt(row: SlackRetirementHistoryRow): SlackFormRetirementReceipt | null { + if (!["slack_form_owner", "slack_form_grant"].includes(row.kind) || row.result?.phase !== "transferred" || row.result?.nextAttemptAt !== null) return null; + const retired = closedRecord(row.result.retirement, ["version", "binding", "instanceId", "environment", "subject", "receiptDigest", "receipt"]); + const receipt = parseSlackFormRetirementReceipt(retired.receipt), binding = receipt.binding; + if (row.companyId !== binding.companyId || row.endpointId !== binding.endpointId || row.payload.subject !== receipt.subject + || row.payload.environment !== receipt.environment || retired.version !== 1 || retired.instanceId !== binding.instanceId + || retired.environment !== receipt.environment || retired.subject !== receipt.subject || retired.receiptDigest !== receipt.receiptId + || !isDeepStrictEqual(parseFormBinding(retired.binding), binding) + || row.kind === "slack_form_owner" && row.status !== "processed" || row.kind === "slack_form_grant" && row.status !== "cancelled") throw new Error("Slack retirement history authority changed"); + if (row.payload.binding !== undefined) { + if (!isDeepStrictEqual(parseFormBinding(row.payload.binding), binding)) throw new Error("Slack retirement history binding changed"); + } else { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(enrollment, key) !== Reflect.get(binding, key))) throw new Error("Slack retirement enrollment history changed"); + } + if (row.kind === "slack_form_owner" && !isSlackFormEnrollmentActionId(row.providerActionId, parseSlackFormPublicEnrollment(row.payload.enrollment))) + throw new Error("Slack retirement history owner changed"); + return receipt; +} diff --git a/server/src/services/chat-channels/slack/adapter.ts b/server/src/services/chat-channels/slack/adapter.ts index 9e76796544b..1238473468c 100644 --- a/server/src/services/chat-channels/slack/adapter.ts +++ b/server/src/services/chat-channels/slack/adapter.ts @@ -1,8 +1,29 @@ +import { createHash } from "node:crypto"; +import type { ModalElement } from "chat"; +import { slackDeferredCallbackActive } from "./cloud-callback.js"; +import { slackFormOpenContext } from "./cloud-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; import { SlackAdapter } from "@chat-adapter/slack"; import type { AdapterPostableMessage, LinkPreview, RawMessage } from "chat"; import { slackSelectionMessage } from "./cards.js"; export class PaperclipSlackAdapter extends SlackAdapter { + override async openModal(triggerId: string, modal: ModalElement, contextId?: string): Promise<{ viewId: string }> { + const current = slackFormOpenContext(); + if (!current) { + if (slackDeferredCallbackActive()) throw new SlackCloudReceiptError("callback_trigger_owner_required", "attention"); + return super.openModal(triggerId, modal, contextId); + } + const view = current.operation.view; + if (!view || current.operation.closed || current.updateAttempted || createHash("sha256").update(triggerId).digest("hex") !== current.operation.triggerDigest) + throw new SlackCloudReceiptError("form_provider_attempt_changed", "attention"); + current.updateAttempted = true; + const result = await super.updateModal(view.id, { ...modal, notifyOnClose: true }, { hash: view.hash, externalId: view.externalId, contextId }); + if (result.viewId !== view.id || !result.viewHash) throw new SlackCloudReceiptError("form_update_receipt_invalid", "attention"); + current.updatedView = { ...view, hash: result.viewHash }; + return { viewId: result.viewId }; + } + override async postMessage(threadId: string, message: AdapterPostableMessage): Promise { const payload = slackSelectionMessage(message); if (!payload) return super.postMessage(threadId, message); diff --git a/server/src/services/chat-channels/slack/cloud-callback.test.ts b/server/src/services/chat-channels/slack/cloud-callback.test.ts new file mode 100644 index 00000000000..b6f3088f93c --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-callback.test.ts @@ -0,0 +1,105 @@ +import { guardSlackFixtureNetwork, stubSlackFixtureTransport } from "../../../__tests__/helpers/slack-cloud/transport.js"; +import { verifySlackSignature } from "@chat-adapter/slack/webhook"; +import type { ChatSdkRuntimeCallbacks } from "../../chat-sdk-runtime.js"; +import { createChatSdkEndpointRuntime } from "../../chat-sdk-runtime.js"; +import { createHash, createHmac } from "node:crypto"; +import { describe, expect, it, vi } from "vitest"; +import { receiveSlackCloudCallback, slackDeferredCallbackActive } from "./cloud-callback.js"; +import { slackCloudAcceptedAt, type SlackCloudEvent } from "./cloud-event.js"; +const acceptedAt = Date.parse("2099-01-01T00:00:00Z"); +function fixture(lane: "command" | "action" = "command") { + const current = { instanceId: "instance", environment: "test", companyId: "company", endpointId: "endpoint", connectionId: "connection", publicId: "a".repeat(43), runtimeGeneration: 1, + teamId: "TTEST", appId: "ATEST", bindingId: `slack-event:${"b".repeat(64)}`, signingSecret: "0123456789abcdef0123456789abcdef" }; + const body = lane === "command" ? new URLSearchParams({ command: "/paperclip", text: "useful original task", trigger_id: "original-one-use-trigger", team_id: "TTEST", api_app_id: "ATEST", user_id: "UTEST", channel_id: "CTEST", response_url: "https://hooks.slack.com/commands/controlled" }).toString() + : new URLSearchParams({ payload: JSON.stringify({ type: "block_actions", api_app_id: "ATEST", team: { id: "TTEST" }, user: { id: "UTEST" }, channel: { id: "CTEST" }, actions: [{ action_id: "opaque-confirmation", action_ts: "4070908800.000001" }], trigger_id: "original-one-use-trigger" }) }).toString(); + const rawBody = Buffer.from(body).toString("base64"), timestamp = String(Math.floor(acceptedAt / 1000)); + const signature = `v0=${createHmac("sha256", current.signingSecret).update(`v0:${timestamp}:`).update(body).digest("hex")}`; + const { signingSecret: _secret, ...identity } = current; + const event: SlackCloudEvent = { id: `slack_callback_${createHash("sha256").update(JSON.stringify([current.bindingId, timestamp, rawBody])).digest("hex")}`, provider: "slack", event: lane, + installationId: current.bindingId, bindingIds: [current.bindingId], createdAt: new Date(acceptedAt).toISOString(), payload: { slackCallback: { version: 1, ...identity, lane, userId: "UTEST", channelId: "CTEST", acceptedAt: new Date(acceptedAt).toISOString(), rawBody, timestamp, signature } } }; + return { current, event, body }; +} +describe("deferred Slack command and action custody", () => { + it.each(["command", "action"] as const)("preserves original %s bytes and authentication while awaiting actual callback completion", async lane => { + const f = fixture(lane); let completed = false; + await receiveSlackCloudCallback(f.event, f.current, async request => { + expect(slackDeferredCallbackActive()).toBe(true); expect(slackCloudAcceptedAt(request)).toBe(acceptedAt); + expect(await request.text()).toBe(f.body); expect(request.headers.get("content-type")).toBe("application/x-www-form-urlencoded"); + await Promise.resolve(); completed = true; return new Response(null, { status: 200 }); + }, acceptedAt + 3500); + expect(completed).toBe(true); expect(slackDeferredCallbackActive()).toBe(false); + }); + it("does not acknowledge unaccepted or failed callback work", async () => { + const f = fixture(); + await expect(receiveSlackCloudCallback(f.event, f.current, async () => new Response(null, { status: 503 }), acceptedAt)).rejects.toMatchObject({ disposition: "retry" }); + const original = new Error("actual admission failed"); + await expect(receiveSlackCloudCallback(f.event, f.current, async () => { throw original; }, acceptedAt)).rejects.toBe(original); + expect(slackDeferredCallbackActive()).toBe(false); + }); + it("rejects forged original authentication and foreign instance without invoking the SDK", async () => { + const f = fixture(), handler = vi.fn(async () => new Response(null, { status: 200 })); + const packet = f.event.payload.slackCallback as Record; + packet.signature = "v0=" + "0".repeat(64); + await expect(receiveSlackCloudCallback(f.event, f.current, handler, acceptedAt)).rejects.toMatchObject({ disposition: "attention" }); + const foreign = fixture(); + await receiveSlackCloudCallback(foreign.event, { ...foreign.current, instanceId: "foreign" }, handler, acceptedAt); + expect(handler).not.toHaveBeenCalled(); + }); + it("does not accept a callback profile as a view submission or options response owner", async () => { + const f = fixture(), handler = vi.fn(async () => new Response(null, { status: 200 })); + f.event.event = "view_submission"; + await expect(receiveSlackCloudCallback(f.event, f.current, handler, acceptedAt)).rejects.toMatchObject({ disposition: "invalid" }); + expect(handler).not.toHaveBeenCalled(); + }); + it("crosses the actual registered Slash SDK owner and refuses ordinary expired external evidence", async () => { + const network = guardSlackFixtureNetwork(), f = fixture(); + let runtime: ReturnType | undefined; + let primary: unknown; + try { + const slash: NonNullable = async event => { + await event.event.channel.postEphemeral(event.event.user, { markdown: "Useful normal bot notice" }, { fallbackToDM: false }); + }; + const onSlashCommand = vi.fn(slash); + runtime = createChatSdkEndpointRuntime({ companyId: f.current.companyId, endpointId: f.current.endpointId, logger: "silent", + callbacks: { onMessage: async () => {}, onSlashCommand }, persistence: { read: async () => null, compareAndSet: async () => true, deleteIfVersion: async () => true }, + providerConfig: { provider: "slack", userName: "paperclip-agent", credentials: { apiUrl: "http://127.0.0.1:1/api/", botToken: "xoxb-synthetic", botUserId: "UBOT", signingSecret: f.current.signingSecret } } }); + await runtime.initialize(); + const transport = stubSlackFixtureTransport(runtime.getProviderAdapter()); + const delegate = transport.getMockImplementation(); + if (!delegate) throw new Error("Slack fixture transport is missing"); + let notices = 0; + transport.mockImplementation(async config => { + const url = new URL(config.url, config.baseURL); + if (url.hostname === "127.0.0.1" && url.pathname.endsWith("/chat.postEphemeral")) { + notices++; + const body = new URLSearchParams(String(Reflect.get(config, "data"))); + expect(body.get("channel")).toBe("CTEST"); expect(body.get("user")).toBe("UTEST"); + expect(body.has("response_url")).toBe(false); + return { data: { ok: true, message_ts: "1.2" }, status: 200, statusText: "OK", headers: {}, config, request: { path: url.pathname } }; + } + return delegate(config); + }); + const owner = runtime; + await receiveSlackCloudCallback(f.event, f.current, request => owner.handleWebhook(request), acceptedAt + 31 * 60_000); + expect(notices).toBe(1); + expect(onSlashCommand).toHaveBeenCalledOnce(); + expect(onSlashCommand).toHaveBeenCalledWith(expect.objectContaining({ endpointId: f.current.endpointId, provider: "slack", + event: expect.objectContaining({ command: "/paperclip", text: "useful original task", triggerId: "original-one-use-trigger" }) })); + await transport.assertNoErrors(); + const packet = f.event.payload.slackCallback as Record; + await expect(verifySlackSignature(f.body, new Headers({ "x-slack-request-timestamp": String(packet.timestamp), "x-slack-signature": String(packet.signature) }), + { signingSecret: f.current.signingSecret, now: () => acceptedAt + 31 * 60_000 })).rejects.toThrow("too old"); + const response = await runtime.handleWebhook(new Request("https://external.invalid", { method: "POST", body: f.body, + headers: { "content-type": "application/x-www-form-urlencoded", "x-slack-request-timestamp": String(packet.timestamp), "x-slack-signature": String(packet.signature) } })); + expect(response.status).toBe(401); expect(onSlashCommand).toHaveBeenCalledOnce(); + } catch (error) { primary = error; throw error; } + finally { + const failures: unknown[] = []; + try { await runtime?.shutdown(); } catch (error) { failures.push(error); } + try { network.assertQuiet(); } catch (error) { failures.push(error); } + try { network.restore(); } catch (error) { failures.push(error); } + if (failures.length) throw new AggregateError(primary === undefined ? failures : [primary, ...failures], "Slack registered callback cleanup failed"); + } + }); + +}); diff --git a/server/src/services/chat-channels/slack/cloud-callback.ts b/server/src/services/chat-channels/slack/cloud-callback.ts new file mode 100644 index 00000000000..cda9398f440 --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-callback.ts @@ -0,0 +1,56 @@ +import { AsyncLocalStorage } from "node:async_hooks"; +import { createHash, createHmac, timingSafeEqual } from "node:crypto"; +import { SlackCloudReceiptError, withSlackCloudAcceptedAt, type CurrentSlackCloudBinding, type SlackCloudEvent } from "./cloud-event.js"; + +const deferred = new AsyncLocalStorage(); +export const slackDeferredCallbackActive = () => deferred.getStore() === true; +export async function receiveSlackCloudCallback(event: SlackCloudEvent, current: (CurrentSlackCloudBinding & { instanceId: string; environment: string }) | null, + handleWebhook: (request: Request) => Promise, now = Date.now()): Promise { + if (!current) return; + const raw = event.payload.slackCallback; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const packet = raw as Record; + const expectedKeys = ["version", "lane", "teamId", "appId", "userId", "channelId", "instanceId", "environment", "bindingId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "acceptedAt", "timestamp", "signature", "rawBody"]; + if (Object.keys(packet).sort().join() !== expectedKeys.sort().join() || packet.version !== 1 || event.provider !== "slack" + || !["command", "action"].includes(String(packet.lane)) || event.event !== packet.lane) + throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + if (["companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "bindingId", "instanceId", "environment"].some(key => packet[key] !== Reflect.get(current, key))) return; + if (event.installationId !== current.bindingId || event.bindingIds.length !== 1 || event.bindingIds[0] !== current.bindingId + || typeof packet.timestamp !== "string" || !/^[0-9]{1,16}$/.test(packet.timestamp) + || typeof packet.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(packet.signature) + || typeof packet.rawBody !== "string" || packet.rawBody.length > 1_398_104 || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(packet.rawBody)) + throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const acceptedAt = Date.parse(event.createdAt); + if (packet.acceptedAt !== event.createdAt || !Number.isFinite(acceptedAt) || acceptedAt > now + 300_000 || now - acceptedAt >= 30 * 24 * 60 * 60_000 + || Math.abs(Math.floor(acceptedAt / 1000) - Number(packet.timestamp)) > 300) + throw new SlackCloudReceiptError("callback_acceptance_invalid", "attention"); + const body = Buffer.from(packet.rawBody, "base64"); + if (body.byteLength > 1_048_576) throw new SlackCloudReceiptError("callback_packet_invalid", "invalid"); + const digest = createHash("sha256").update(JSON.stringify([current.bindingId, packet.timestamp, packet.rawBody])).digest("hex"); + if (event.id !== `slack_callback_${digest}` || !timingSafeEqual(createHmac("sha256", current.signingSecret).update(`v0:${packet.timestamp}:`).update(body).digest(), Buffer.from(packet.signature.slice(3), "hex"))) + throw new SlackCloudReceiptError("callback_original_authentication_failed", "attention"); + const params = new URLSearchParams(body.toString("utf8")); + for (const key of params.keys()) if (params.getAll(key).length !== 1) throw new SlackCloudReceiptError("callback_duplicate_field", "invalid"); + if (packet.lane === "command") { + if (params.has("payload") || !/^\/[A-Za-z0-9_-]{1,32}$/.test(params.get("command") ?? "") || !params.get("trigger_id") || !params.has("text") + || params.get("team_id") !== current.teamId || params.get("api_app_id") !== current.appId || params.get("user_id") !== packet.userId || params.get("channel_id") !== packet.channelId) + throw new SlackCloudReceiptError("callback_original_identity_changed", "invalid"); + } else { + let payload: unknown; + try { payload = JSON.parse(params.get("payload") ?? "null"); } catch { throw new SlackCloudReceiptError("callback_original_invalid", "invalid"); } + if (!payload || typeof payload !== "object" || Array.isArray(payload) || [...params.keys()].join() !== "payload") throw new SlackCloudReceiptError("callback_original_invalid", "invalid"); + const actions = Reflect.get(payload, "actions"); + const field = (key: string) => { const value = Reflect.get(payload, key); return value && typeof value === "object" ? Reflect.get(value, "id") : undefined; }; + if (Reflect.get(payload, "type") !== "block_actions" || Reflect.get(payload, "api_app_id") !== current.appId || field("team") !== current.teamId || field("user") !== packet.userId || field("channel") !== packet.channelId + || !Array.isArray(actions) || actions.length !== 1 || !actions[0] || typeof actions[0] !== "object" + || typeof Reflect.get(actions[0], "action_id") !== "string" || !Reflect.get(actions[0], "action_id") || String(Reflect.get(actions[0], "action_id")).length > 255 || String(Reflect.get(actions[0], "action_id")).startsWith("pcf:")) + throw new SlackCloudReceiptError("callback_trigger_owner_required", "attention"); + } + const request = new Request("https://paperclip.invalid/slack-cloud-ingress", { method: "POST", headers: { "content-type": "application/x-www-form-urlencoded", + "x-slack-request-timestamp": packet.timestamp, "x-slack-signature": packet.signature }, body: new Uint8Array(body) }); + await deferred.run(true, () => withSlackCloudAcceptedAt(request, acceptedAt, async () => { + const response = await handleWebhook(request); + await response.body?.cancel(); + if (!response.ok) throw new SlackCloudReceiptError("callback_unaccepted", "retry"); + })); +} diff --git a/server/src/services/chat-channels/slack/cloud-consumer.ts b/server/src/services/chat-channels/slack/cloud-consumer.ts new file mode 100644 index 00000000000..71559e9e4b1 --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-consumer.ts @@ -0,0 +1,58 @@ +import { completedSlackFormSessionReceipt } from "./form-idle-history.js"; +import { and, gt, inArray, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { projectSlackFormRetainedOwner, type SlackFormRetainedRow } from "./form-session.js"; + +export type SlackFormConnectorIdentity = Readonly<{ instanceId: string; environment: "development" | "test" | "staging" | "production" }>; +export function slackFormConnectorIdentity(connector: unknown): SlackFormConnectorIdentity | null { + if (!connector || typeof connector !== "object") return null; + let value: unknown; + try { + const getter = Reflect.get(connector, "instanceIdentity"); + if (typeof getter !== "function") return null; + value = Reflect.apply(getter, connector, []); + } catch { return null; } + if (!value || typeof value !== "object" || Array.isArray(value) + || Object.keys(value).sort().join() !== "environment,instanceId") return null; + const instanceId = Reflect.get(value, "instanceId"), environment = Reflect.get(value, "environment"); + if (typeof instanceId !== "string" || !instanceId.length || instanceId.length > 200 + || environment !== "development" && environment !== "test" && environment !== "staging" && environment !== "production") return null; + return Object.freeze({ instanceId, environment }); +} + +export function selectSlackFormRetainedConsumer(rows: readonly SlackFormRetainedRow[], identity: SlackFormConnectorIdentity) { + let owner: ReturnType = null; + const attention: { actionId: string; code: string }[] = []; + for (const row of rows) { + try { + const value = projectSlackFormRetainedOwner(row, identity.instanceId, identity.environment); + if (value && row.kind === "slack_form_session" && row.result?.formAck && typeof row.result.formAck === "object" + && Reflect.get(row.result.formAck, "phase") === "transferred" && completedSlackFormSessionReceipt(row)) continue; + if (value && !owner) owner = value; + } catch { + attention.push({ actionId: row.id, code: "form_retained_owner_invalid" }); + } + } + return { owner, attention }; +} + +export async function readSlackFormRetainedConsumer(db: Pick, connector: unknown) { + const identity = slackFormConnectorIdentity(connector); + if (!identity) return { owner: null, attention: [] }; + let after: string | undefined; + const attention: { actionId: string; code: string }[] = []; + for (;;) { + const rows = await db.select().from(chatActions).where(and( + inArray(chatActions.kind, ["slack_form_session", "slack_form_owner", "slack_form_grant"]), + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${identity.instanceId}`, + after === undefined ? undefined : gt(chatActions.id, after))) + .orderBy(chatActions.id).limit(32); + const selected = selectSlackFormRetainedConsumer(rows, identity); + attention.push(...selected.attention.slice(0, Math.max(0, 32 - attention.length))); + if (selected.owner) return { owner: selected.owner, attention }; + if (rows.length < 32) return { owner: null, attention }; + const next = rows[rows.length - 1]?.id; + if (!next || next === after) throw new Error("Slack retained consumer cursor did not advance"); + after = next; + } +} diff --git a/server/src/services/chat-channels/slack/cloud-enrollment.ts b/server/src/services/chat-channels/slack/cloud-enrollment.ts index be8e33394e3..89881195fc5 100644 --- a/server/src/services/chat-channels/slack/cloud-enrollment.ts +++ b/server/src/services/chat-channels/slack/cloud-enrollment.ts @@ -145,6 +145,7 @@ export function createSlackCloudEnrollment(db: Db, options: { resolveCredentials: ReturnType["resolveCredentials"]; runtimeGeneration(setup: InternalSetupState): number; connector?: Pick; + onCommitted?(tx: Parameters[0]>[0], values: { endpointId: string; active: boolean; subject: string; instanceId: string; environment: import("../../paperclip-cloud-connector.js").PaperclipCloudConnectorEnvironment }): Promise; }) { return async (endpointId: string, _actorUserId: string | null | undefined, lease: CredentialMutationLeaseGuard) => { const [current] = await db.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints) @@ -189,6 +190,7 @@ export function createSlackCloudEnrollment(db: Db, options: { sql`${chatEndpoints.setup}->'slackCloudIntent'->>'id'=${intent.id}`, sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer,0)=${intent.runtimeGeneration}`)).returning({ id: chatEndpoints.id }); if (updated.length !== 1) throw new Error("Slack configuration changed before event enrollment committed"); + if (change.receipt) await options.onCommitted?.(tx, { endpointId, active: intent.active, subject: intent.subject, instanceId: intent.instanceId, environment: config.environment }); await lease.assertOwned(tx); }); try { diff --git a/server/src/services/chat-channels/slack/cloud-event.ts b/server/src/services/chat-channels/slack/cloud-event.ts index b589de638bc..878c2dcb096 100644 --- a/server/src/services/chat-channels/slack/cloud-event.ts +++ b/server/src/services/chat-channels/slack/cloud-event.ts @@ -3,6 +3,14 @@ import { createHash, createHmac, timingSafeEqual } from "node:crypto"; const acceptedTimes = new WeakMap(); export const slackCloudAcceptedAt = (request: Request): number | undefined => acceptedTimes.get(request); +export async function withSlackCloudAcceptedAt(request: Request, acceptedAt: number, work: () => Promise): Promise { + if (!Number.isFinite(acceptedAt)) throw new SlackCloudReceiptError("form_accepted_time_invalid", "invalid"); + const before = acceptedTimes.get(request); + acceptedTimes.set(request, acceptedAt); + try { return await work(); } + finally { if (before === undefined) acceptedTimes.delete(request); else acceptedTimes.set(request, before); } +} + export class SlackCloudReceiptError extends Error { constructor(readonly code: string, readonly disposition: "invalid" | "attention" | "retry") { super(code); } } diff --git a/server/src/services/chat-channels/slack/cloud-forms.ts b/server/src/services/chat-channels/slack/cloud-forms.ts new file mode 100644 index 00000000000..5e8e73055df --- /dev/null +++ b/server/src/services/chat-channels/slack/cloud-forms.ts @@ -0,0 +1,268 @@ +import { AsyncLocalStorage } from "node:async_hooks"; +import { createHash, createHmac, timingSafeEqual } from "node:crypto"; +import { and, eq, inArray } from "drizzle-orm"; +import { chatActions, chatConversations, chatPublications, type Db } from "@paperclipai/db"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import { slackThreadChannelId } from "./messages.js"; +import { SlackCloudReceiptError, type SlackCloudEvent } from "./cloud-event.js"; + +export type SlackFormBinding = { + version: 1; instanceId: string; companyId: string; endpointId: string; publicId: string; + connectionId: string; runtimeGeneration: number; teamId: string; appId: string; + eventBindingId: string; credentialFingerprint: string; +}; +export type SlackFormPublicEnrollment = Omit & { bindingId: string }; +export function parseSlackFormPublicEnrollment(value: unknown): SlackFormPublicEnrollment { + const v = record(value, ["version", "instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "bindingId"]); + if (v.version !== 1 || !Number.isSafeInteger(v.runtimeGeneration) || Number(v.runtimeGeneration) < 1 + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "teamId", "appId", "bindingId"].some(k => typeof v[k] !== "string" || !String(v[k]).length || String(v[k]).length > 200) + || !/^[A-Za-z0-9_-]{43}$/.test(String(v.publicId)) || !/^T[A-Z0-9]+$/.test(String(v.teamId)) || !/^A[A-Z0-9]+$/.test(String(v.appId)) + || !/^slack-event:[a-f0-9]{64}$/.test(String(v.bindingId))) throw new SlackCloudReceiptError("invalid_form_enrollment", "invalid"); + return v as unknown as SlackFormPublicEnrollment; +} +export type SlackIssuedFormGrant = { + version: 1; binding: SlackFormBinding; grantId: string; + conversationId: string; conversationGeneration: number; publicationId: string; interactionId: string; + openActionRowId: string; openActionDigest: string; submitActionRowId: string; submitActionDigest: string; + expiresAt: string; channelId: string; threadTs: string | null; providerMessageId: string; +}; +export type SlackFormView = { id: string; hash: string; externalId: string }; +export type SlackFormOperation = { + version: 1; binding: SlackFormBinding; grantId: string; id: string; externalId: string; + deliveryId: string; requestDigest: string; triggerDigest: string; userId: string; + acceptedAt: string; triggerDeadline: string; + phase: "reserved" | "attempted" | "confirmed" | "rejected" | "ambiguous" | "terminal_proven" | "acknowledged"; + view: SlackFormView | null; + closed: { kind: "closed" | "submitted"; view: SlackFormView; acceptedAt: string; requestDigest: string; evidence: { + version: 1; kind: "close" | "submission"; submitReceiptId: string | null; instanceId: string; environment: string; operationId: string; binding: SlackFormBinding; + acceptedAt: string; rawBody: string; timestamp: string; signature: string; + } } | null; + acknowledgement: { appReceiptId: string; acceptedAt: string } | null; + rejection: { code: "expired_before_spend" | "expired_trigger_id" | "invalid_auth" | "token_revoked" | "not_authed" | "invalid_arguments"; + observedAt: string; responseDigest: string; evidence: Record } | null; +}; +const hash = (s: string) => createHash("sha256").update(s).digest("hex"); +function record(value: unknown, keys: string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== keys.sort().join()) + throw new SlackCloudReceiptError("invalid_form_receipt", "invalid"); + return value as Record; +} +export function parseSlackFormBinding(value: unknown): SlackFormBinding { + const v = record(value, ["version", "instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId", "eventBindingId", "credentialFingerprint"]); + if (v.version !== 1 || !Number.isSafeInteger(v.runtimeGeneration) || Number(v.runtimeGeneration) < 1 + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "teamId", "appId", "eventBindingId", "credentialFingerprint"].some(k => typeof v[k] !== "string" || !(v[k] as string).length || (v[k] as string).length > 200) + || !/^[A-Za-z0-9_-]{43}$/.test(String(v.publicId)) || !/^T[A-Z0-9]+$/.test(String(v.teamId)) || !/^A[A-Z0-9]+$/.test(String(v.appId)) + || !/^slack-event:[a-f0-9]{64}$/.test(String(v.eventBindingId)) || !/^[a-f0-9]{64}$/.test(String(v.credentialFingerprint))) + throw new SlackCloudReceiptError("invalid_form_binding", "invalid"); + return v as unknown as SlackFormBinding; +} +export function slackFormSetupRecord(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("form_setup_invalid", "attention"); + const setup: Record = { ...value }; + return setup; +} +export function currentSlackFormOwnerBinding(setup: Record): SlackFormBinding | null { + if (setup.slackCloudForms === undefined) return null; + return parseSlackFormBinding(setup.slackCloudForms); +} +export const sameSlackFormBinding = (a: SlackFormBinding, b: SlackFormBinding) => Object.keys(a).every(k => Reflect.get(a, k) === Reflect.get(b, k)); +function view(value: unknown): SlackFormView { + const v = record(value, ["id", "hash", "externalId"]); + if (!/^V[A-Z0-9]+$/.test(String(v.id)) || typeof v.hash !== "string" || !v.hash.length || v.hash.length > 200 + || typeof v.externalId !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(v.externalId)) throw new SlackCloudReceiptError("invalid_form_view", "invalid"); + return v as unknown as SlackFormView; +} +export function parseSlackFormOperation(value: unknown): SlackFormOperation { + const v = record(value, ["version", "binding", "grantId", "id", "externalId", "deliveryId", "requestDigest", "triggerDigest", "userId", "acceptedAt", "triggerDeadline", "phase", "view", "closed", "acknowledgement", "rejection"]); + parseSlackFormBinding(v.binding); + if (v.version !== 1 || ![v.id, v.grantId, v.requestDigest, v.triggerDigest].every(x => typeof x === "string" && /^[a-f0-9]{64}$/.test(x)) + || v.externalId !== `pc-form-${v.id}` || v.deliveryId !== `slack_form_${v.id}` + || !/^[UW][A-Z0-9]+$/.test(String(v.userId)) || typeof v.acceptedAt !== "string" || !Number.isFinite(Date.parse(v.acceptedAt)) + || typeof v.triggerDeadline !== "string" || !Number.isFinite(Date.parse(v.triggerDeadline)) + || !["reserved", "attempted", "confirmed", "rejected", "ambiguous", "terminal_proven", "acknowledged"].includes(String(v.phase))) + throw new SlackCloudReceiptError("invalid_form_operation", "invalid"); + if (v.view !== null) view(v.view); + if (v.closed !== null) { + const c = record(v.closed, ["kind", "view", "acceptedAt", "requestDigest", "evidence"]); view(c.view); + const e = record(c.evidence, ["version", "kind", "submitReceiptId", "instanceId", "environment", "operationId", "binding", "acceptedAt", "rawBody", "timestamp", "signature"]); + parseSlackFormBinding(e.binding); + if (e.version !== 1 || !["close", "submission"].includes(String(e.kind)) || (e.kind === "close" ? e.submitReceiptId !== null : typeof e.submitReceiptId !== "string") || typeof e.instanceId !== "string" || typeof e.environment !== "string" || typeof e.operationId !== "string" + || typeof e.acceptedAt !== "string" || !Number.isFinite(Date.parse(e.acceptedAt)) || typeof e.rawBody !== "string" || e.rawBody.length > 1398104 + || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(e.rawBody) + || typeof e.timestamp !== "string" || !/^[0-9]{1,16}$/.test(e.timestamp) || typeof e.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(e.signature)) + throw new SlackCloudReceiptError("invalid_form_close_evidence", "invalid"); + if (!["closed", "submitted"].includes(String(c.kind)) || typeof c.acceptedAt !== "string" || !Number.isFinite(Date.parse(c.acceptedAt)) || typeof c.requestDigest !== "string" + || !/^[a-f0-9]{64}$/.test(c.requestDigest)) throw new SlackCloudReceiptError("invalid_form_close", "invalid"); + } + if (v.rejection !== null) { + const n = record(v.rejection, ["code", "observedAt", "responseDigest", "evidence"]); + if (!["expired_before_spend", "expired_trigger_id", "invalid_auth", "token_revoked", "not_authed", "invalid_arguments"].includes(String(n.code)) + || typeof n.observedAt !== "string" || !Number.isFinite(Date.parse(n.observedAt)) || typeof n.responseDigest !== "string" || !/^[a-f0-9]{64}$/.test(n.responseDigest) + || !n.evidence || typeof n.evidence !== "object" || Array.isArray(n.evidence)) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + const e = record(n.evidence, ["version", "kind", "instanceId", "environment", "operationId", "binding", "observedAt", "code", "responseDigest", "observation"]); + parseSlackFormBinding(e.binding); + if (e.version !== 1 || e.kind !== "rejection" || typeof e.instanceId !== "string" || typeof e.environment !== "string" || typeof e.operationId !== "string" + || !e.observation || typeof e.observation !== "object" || Array.isArray(e.observation)) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + } + if ((v.phase === "rejected" && !v.rejection) || (v.rejection && (v.closed || v.view || !["rejected", "terminal_proven", "acknowledged"].includes(String(v.phase))))) throw new SlackCloudReceiptError("invalid_form_no_effect", "invalid"); + if (v.acknowledgement !== null) { + const a = record(v.acknowledgement, ["appReceiptId", "acceptedAt"]); + if (typeof a.appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(a.appReceiptId) || typeof a.acceptedAt !== "string" || !Number.isFinite(Date.parse(a.acceptedAt))) + throw new SlackCloudReceiptError("invalid_form_acknowledgement", "invalid"); + } + const op = v as unknown as SlackFormOperation; + if (["terminal_proven", "acknowledged"].includes(op.phase) !== (op.acknowledgement !== null) || (["terminal_proven", "acknowledged"].includes(op.phase) && !op.closed && !op.rejection)) + throw new SlackCloudReceiptError("invalid_form_acknowledgement", "invalid"); + if (((op.phase === "confirmed" || ["terminal_proven", "acknowledged"].includes(op.phase) && op.closed) && !op.view) || (op.view && op.view.externalId !== op.externalId) + || (op.closed && (op.closed.view.externalId !== op.externalId || (op.view && op.view.id !== op.closed.view.id)))) + throw new SlackCloudReceiptError("form_view_changed", "attention"); + return op; +} +export function slackIssuedFormGrantId(binding: SlackFormBinding, openActionRowId: string, submitActionRowId: string, publicationId: string, interactionId: string): string { + return hash(JSON.stringify([[binding.instanceId, binding.companyId, binding.endpointId, binding.publicId, binding.connectionId, binding.runtimeGeneration, + binding.teamId, binding.appId, binding.eventBindingId, binding.credentialFingerprint], openActionRowId, submitActionRowId, publicationId, interactionId])); +} +export async function projectSlackIssuedFormGrant(db: Pick, binding: SlackFormBinding, + resolved: ResolvedChatQuestionFormOpen): Promise { + const [publication] = await db.select().from(chatPublications).where(and(eq(chatPublications.id, resolved.publicationId), + eq(chatPublications.companyId, binding.companyId), eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.state, "published"))).limit(1); + if (!publication?.providerMessageId || !publication.publishedAt) return null; + const [conversation] = await db.select().from(chatConversations).where(and(eq(chatConversations.id, publication.conversationId), + eq(chatConversations.companyId, binding.companyId), eq(chatConversations.endpointId, binding.endpointId), eq(chatConversations.state, "active"))).limit(1); + if (!conversation) return null; + const channelId = slackThreadChannelId(conversation.externalThreadId); + const thread = /^slack:[^:]+:([0-9]+[.][0-9]+)?$/.exec(conversation.externalThreadId); + if (!channelId || !thread || conversation.externalConversationId !== `slack:${channelId}`) return null; + const rows = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), eq(chatActions.conversationId, conversation.id), inArray(chatActions.id, [resolved.openActionRowId, resolved.submitActionRowId]))); + const open = rows.find(a => a.id === resolved.openActionRowId && a.kind === "question_form_open" && a.status === "issued" && a.providerActionId === resolved.openActionId); + const submit = rows.find(a => a.id === resolved.submitActionRowId && a.kind === "question_form_submit" && a.status === "issued" && a.providerActionId === resolved.submitActionId); + if (!open || !submit || open.payload.version !== 1 || submit.payload.version !== 1 + || open.payload.interactionId !== resolved.interactionId || submit.payload.interactionId !== resolved.interactionId + || open.payload.publicationId !== resolved.publicationId || submit.payload.publicationId !== resolved.publicationId + || open.payload.formActionId !== submit.providerActionId || submit.payload.formActionId !== submit.providerActionId + || typeof open.payload.expiresAt !== "string" || open.payload.expiresAt !== submit.payload.expiresAt) return null; + const grantId = slackIssuedFormGrantId(binding, resolved.openActionRowId, resolved.submitActionRowId, resolved.publicationId, resolved.interactionId); + return { version: 1, binding, grantId, conversationId: conversation.id, conversationGeneration: conversation.sessionGeneration, + publicationId: publication.id, interactionId: resolved.interactionId, openActionRowId: open.id, openActionDigest: hash(open.providerActionId), + submitActionRowId: submit.id, submitActionDigest: hash(submit.providerActionId), expiresAt: open.payload.expiresAt, + channelId, threadTs: thread[1] || null, providerMessageId: publication.providerMessageId }; +} +export type SlackFormLookup = (values: { binding: SlackFormBinding; operationId: string; deliveryId: string }) => Promise; +export type SlackFormDeliveryAuthority = { binding: SlackFormBinding; instanceId: string; environment: string } & + ({ kind: "current"; signingSecret: string } | { kind: "retained" }); +export async function lookupSlackFormDelivery(event: SlackCloudEvent, current: SlackFormDeliveryAuthority, + lookup: SlackFormLookup, now = Date.now()): Promise<{ operation: SlackFormOperation; originalPayload: Record; authority: "current" | "retained" }> { + if (event.provider !== "slack" || event.event !== "form_open") throw new SlackCloudReceiptError("unsupported_form_receipt", "attention"); + const packet = record(event.payload.slackFormOpen, ["version", "instanceId", "environment", "operation", "acceptedAt", "rawBody", "timestamp", "signature"]); + const op = parseSlackFormOperation(packet.operation); + if (packet.version !== 1 || current.binding.instanceId !== current.instanceId || !sameSlackFormBinding(op.binding, current.binding) || packet.instanceId !== current.instanceId || packet.environment !== current.environment + || event.id !== op.deliveryId || event.createdAt !== packet.acceptedAt || event.installationId !== `slack-form-open:${op.id}` + || !event.bindingIds.includes(`slack-form-open:${op.id}`)) throw new SlackCloudReceiptError("form_binding_changed", "attention"); + const acceptedAt = typeof packet.acceptedAt === "string" ? Date.parse(packet.acceptedAt) : Number.NaN; + if (typeof packet.rawBody !== "string" || packet.rawBody.length > 1398104 || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(packet.rawBody) + || typeof packet.timestamp !== "string" || !/^[0-9]{1,16}$/.test(packet.timestamp) || typeof packet.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(packet.signature) + || !Number.isFinite(acceptedAt) || acceptedAt > now + 300000 || now - acceptedAt >= 30 * 24 * 60 * 60000 || Math.abs(Math.floor(acceptedAt / 1000) - Number(packet.timestamp)) > 300) + throw new SlackCloudReceiptError("form_original_evidence_invalid", "attention"); + const raw = Buffer.from(packet.rawBody, "base64"); + if (raw.byteLength > 1048576) throw new SlackCloudReceiptError("form_original_too_large", "invalid"); + if (packet.acceptedAt !== op.acceptedAt || hash(JSON.stringify([packet.rawBody, packet.timestamp, packet.signature])) !== op.requestDigest) + throw new SlackCloudReceiptError("form_original_authentication_failed", "attention"); + if (current.kind === "current") { + const signature = createHmac("sha256", current.signingSecret).update(`v0:${packet.timestamp}:`).update(raw).digest(); + if (!timingSafeEqual(signature, Buffer.from(packet.signature.slice(3), "hex"))) throw new SlackCloudReceiptError("form_original_authentication_failed", "attention"); + } + let original: unknown; + try { original = JSON.parse(new URLSearchParams(raw.toString("utf8")).get("payload") ?? "null"); } + catch { throw new SlackCloudReceiptError("form_original_payload_invalid", "invalid"); } + if (!original || typeof original !== "object" || Array.isArray(original)) throw new SlackCloudReceiptError("form_original_payload_invalid", "invalid"); + const body = original as Record; + if (body.type !== "block_actions" || typeof body.trigger_id !== "string" || hash(body.trigger_id) !== op.triggerDigest + || !body.user || typeof body.user !== "object" || Reflect.get(body.user, "id") !== op.userId + || !body.team || typeof body.team !== "object" || Reflect.get(body.team, "id") !== current.binding.teamId + || body.api_app_id !== current.binding.appId) throw new SlackCloudReceiptError("form_original_identity_changed", "attention"); + const receipt = record(await lookup({ binding: current.binding, operationId: op.id, deliveryId: op.deliveryId }), ["version", "instanceId", "environment", "operation"]); + let latest = parseSlackFormOperation(receipt.operation); + if (receipt.version !== 1 || receipt.instanceId !== current.instanceId || receipt.environment !== current.environment + || !sameSlackFormBinding(latest.binding, op.binding) + || ["id", "grantId", "externalId", "deliveryId", "requestDigest", "triggerDigest", "userId", "acceptedAt", "triggerDeadline"].some(k => Reflect.get(latest, k) !== Reflect.get(op, k))) + throw new SlackCloudReceiptError("form_lookup_identity_changed", "attention"); + if (latest.rejection) { + const n = latest.rejection, evidence = n.evidence; + if (evidence.instanceId !== current.instanceId || evidence.environment !== current.environment || evidence.operationId !== op.id + || !sameSlackFormBinding(parseSlackFormBinding(evidence.binding), op.binding) || evidence.observedAt !== n.observedAt || evidence.code !== n.code + || evidence.responseDigest !== n.responseDigest || hash(JSON.stringify([op.id, n.code, evidence.observation])) !== n.responseDigest) + throw new SlackCloudReceiptError("form_no_effect_evidence_changed", "attention"); + } + if (latest.closed) { + const c = latest.closed, evidence = c.evidence; + if (evidence.instanceId !== current.instanceId || evidence.environment !== current.environment || evidence.operationId !== op.id + || !sameSlackFormBinding(evidence.binding, op.binding) || evidence.acceptedAt !== c.acceptedAt + || Math.abs(Math.floor(Date.parse(evidence.acceptedAt) / 1000) - Number(evidence.timestamp)) > 300 + || hash(JSON.stringify([evidence.rawBody, evidence.timestamp, evidence.signature])) !== c.requestDigest) + throw new SlackCloudReceiptError("form_close_evidence_changed", "attention"); + const closeRaw = Buffer.from(evidence.rawBody, "base64"); + if (closeRaw.length > 1048576) throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); + if (current.kind === "current") { + const signature = createHmac("sha256", current.signingSecret).update(`v0:${evidence.timestamp}:`).update(closeRaw).digest(); + if (!timingSafeEqual(signature, Buffer.from(evidence.signature.slice(3), "hex"))) throw new SlackCloudReceiptError("form_close_authentication_failed", "attention"); + } + let closeBody: unknown; + try { closeBody = JSON.parse(new URLSearchParams(closeRaw.toString("utf8")).get("payload") ?? "null"); } + catch { throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); } + if (!closeBody || typeof closeBody !== "object" || Array.isArray(closeBody)) throw new SlackCloudReceiptError("form_close_evidence_invalid", "invalid"); + const v = Reflect.get(closeBody, "view"), user = Reflect.get(closeBody, "user"), team = Reflect.get(closeBody, "team"); + if (Reflect.get(closeBody, "type") !== (c.kind === "submitted" ? "view_submission" : "view_closed") + || evidence.kind !== (c.kind === "submitted" ? "submission" : "close") || Reflect.get(closeBody, "api_app_id") !== op.binding.appId + || !v || typeof v !== "object" || !user || typeof user !== "object" || !team || typeof team !== "object" + || Reflect.get(user, "id") !== op.userId || Reflect.get(team, "id") !== op.binding.teamId || Reflect.get(v, "id") !== c.view.id + || Reflect.get(v, "external_id") !== op.externalId || Reflect.get(v, "hash") !== c.view.hash || Reflect.get(v, "app_id") !== op.binding.appId) + throw new SlackCloudReceiptError("form_close_identity_changed", "attention"); + } + if (current.kind === "retained" && !latest.closed && !latest.rejection) throw new SlackCloudReceiptError("form_retained_resource_unresolved", "attention"); + if (["reserved", "attempted"].includes(latest.phase) && !latest.closed) throw new SlackCloudReceiptError("form_receipt_pending", "retry"); + return { operation: latest, originalPayload: body, authority: current.kind }; +} + +export function assertSlackFormTerminalAck(operation: SlackFormOperation, appReceiptId: string): void { + if (["terminal_proven", "acknowledged"].includes(operation.phase) && operation.acknowledgement?.appReceiptId === appReceiptId && (operation.closed || operation.rejection)) return; + throw new SlackCloudReceiptError(operation.phase === "confirmed" ? "form_view_active" : "form_resource_unresolved", + operation.phase === "confirmed" || operation.closed ? "retry" : "attention"); +} + +export type SlackFormSubmissionEvidence = Readonly<{ endpointId: string; acceptedAt: string; rawBody: string; timestamp: string; signature: string }>; +const submissions = new AsyncLocalStorage(); +export const slackFormSubmissionEvidence = (): SlackFormSubmissionEvidence | undefined => submissions.getStore(); +export async function withSlackFormSubmissionEvidence(request: Request, endpointId: string, enabled: boolean, work: () => Promise): Promise { + if (!enabled) return work(); + const acceptedAt = new Date().toISOString(), copy = request.clone(); + if (!copy.body) return work(); + const reader = copy.body.getReader(), chunks: Uint8Array[] = []; let size = 0; + try { + for (;;) { + const part = await reader.read(); if (part.done) break; + size += part.value.byteLength; + if (size > 1048576) throw new SlackCloudReceiptError("form_submission_too_large", "invalid"); + chunks.push(part.value); + } + } finally { void reader.cancel().catch(() => {}); reader.releaseLock(); } + const bytes = Buffer.concat(chunks), encoded = new URLSearchParams(bytes.toString("utf8")); + if (encoded.getAll("payload").length !== 1) return work(); + let body: unknown; + try { body = JSON.parse(encoded.get("payload")!); } catch { return work(); } + if (!body || typeof body !== "object" || Reflect.get(body, "type") !== "view_submission") return work(); + const view = Reflect.get(body, "view"); + if (!view || typeof view !== "object" || typeof Reflect.get(view, "external_id") !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(Reflect.get(view, "external_id"))) return work(); + const timestamp = request.headers.get("x-slack-request-timestamp"), signature = request.headers.get("x-slack-signature"); + if (!timestamp || !/^[0-9]{1,16}$/.test(timestamp) || !signature || !/^v0=[a-f0-9]{64}$/.test(signature)) return work(); + return submissions.run(Object.freeze({ endpointId, acceptedAt, rawBody: bytes.toString("base64"), timestamp, signature }), work); +} + +export type SlackFormOpenContext = { operation: SlackFormOperation; currentBinding: SlackFormBinding; environment?: string; + sessionActionId?: string; updateAttempted?: boolean; updatedView?: SlackFormView }; +const formOpens = new AsyncLocalStorage(); +export const slackFormOpenContext = (): SlackFormOpenContext | undefined => formOpens.getStore(); +export function withSlackFormOpenContext(operation: SlackFormOperation, currentBinding: SlackFormBinding, work: () => Promise, environment?: string): Promise { + if (operation.phase !== "confirmed" || !operation.view || operation.closed || operation.rejection || !sameSlackFormBinding(operation.binding, currentBinding)) + throw new SlackCloudReceiptError("form_new_use_not_current", "attention"); + return formOpens.run({ operation, currentBinding, environment }, work); +} diff --git a/server/src/services/chat-channels/slack/cloud-ingress.ts b/server/src/services/chat-channels/slack/cloud-ingress.ts index a4368552abc..6d701665111 100644 --- a/server/src/services/chat-channels/slack/cloud-ingress.ts +++ b/server/src/services/chat-channels/slack/cloud-ingress.ts @@ -1,3 +1,4 @@ +import { receiveSlackCloudCallback } from "./cloud-callback.js"; import type { Db } from "@paperclipai/db"; import { chatEndpoints, toolConnections } from "@paperclipai/db"; import { and, eq } from "drizzle-orm"; @@ -26,10 +27,19 @@ export function createSlackCloudEventIngress(db: Db, options: { withCredentialMutationLease: ReturnType; resolveCredentials: ReturnType["resolveCredentials"]; runtimeGeneration(endpoint: EndpointRow): number; + handleFormReceipt?(event: SlackCloudEvent): Promise; + connectorIdentity?(): { instanceId: string; environment: string } | undefined; handleWebhook(publicId: string, provider: "slack", request: Request, expectedGeneration: number): Promise; }) { return registerSlackCloudEventIngress(db, async event => { - const value = event.payload.slackEvent; + if (event.payload.slackFormOpen !== undefined) { + if (!options.handleFormReceipt) throw new SlackCloudReceiptError("form_owner_not_ready", "retry"); + return options.handleFormReceipt(event); + } + const callback = event.payload.slackCallback !== undefined; + const identity = callback ? options.connectorIdentity?.() : undefined; + if (callback && !identity) throw new SlackCloudReceiptError("callback_identity_unavailable", "retry"); + const value = callback ? event.payload.slackCallback : event.payload.slackEvent; if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("invalid_receipt", "invalid"); const packet = value as Record; if (typeof packet.endpointId !== "string" || !/^[a-f0-9-]{36}$/i.test(packet.endpointId)) @@ -45,7 +55,9 @@ export function createSlackCloudEventIngress(db: Db, options: { const endpoint = current.endpoint; if (packet.companyId !== endpoint.companyId || packet.publicId !== endpoint.publicId || packet.connectionId !== endpoint.connectionId || packet.runtimeGeneration !== options.runtimeGeneration(endpoint)) return null; - const enrolled = (endpoint.setup as InternalSetupState).slackCloudEvents; + const setup = endpoint.setup as InternalSetupState; + if (callback && setup.slackCloudInstanceId !== identity?.instanceId) throw new SlackCloudReceiptError("callback_instance_changed", "attention"); + const enrolled = setup.slackCloudEvents; if (!enrolled) throw new Error("Slack Cloud enrollment is not locally committed"); if (enrolled.version !== 1 || enrolled.runtimeGeneration !== options.runtimeGeneration(endpoint) @@ -57,6 +69,11 @@ export function createSlackCloudEventIngress(db: Db, options: { bindingId: enrolled.bindingId, teamId: enrolled.teamId, appId: enrolled.appId, signingSecret }; }); if (!binding) return; + if (callback) { + if (!binding || !identity) return; + return receiveSlackCloudCallback(event, { ...binding, ...identity }, + request => options.handleWebhook(binding.publicId, "slack", request, binding.runtimeGeneration)); + } await receiveSlackCloudEvent(event, binding, request => options.handleWebhook(binding.publicId, "slack", request, binding.runtimeGeneration)); }); diff --git a/server/src/services/chat-channels/slack/configured-coverage.test.ts b/server/src/services/chat-channels/slack/configured-coverage.test.ts new file mode 100644 index 00000000000..2a7482eeb8a --- /dev/null +++ b/server/src/services/chat-channels/slack/configured-coverage.test.ts @@ -0,0 +1,23 @@ +import { expect, it } from "vitest"; +import { CAPABILITIES } from "../provider-catalog.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { requiredSlackIngressLanes, parseSlackConfiguredCoverage, slackConfiguredBindingsFingerprint, matchSlackConfiguredCoverage } from "./configured-coverage.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const requirement = { binding, capabilities: CAPABILITIES.slack, registeredOptions: false }; +it("requires every default configured Slack lane rather than treating a form slice as complete", () => { + expect(requiredSlackIngressLanes(requirement)).toEqual(["actions", "commands", "events", "forms", "modal_close", "modal_submit"]); +}); +it("requires options whenever a real options handler is registered", () => { + expect(requiredSlackIngressLanes({ ...requirement, registeredOptions: true })).toContain("options"); +}); +it("the closed coverage parser refuses foreign instance and altered exact binding fingerprint", () => { + const proof = { version: 1, instanceId: "instance", generation: "c".repeat(64), bindingsFingerprint: slackConfiguredBindingsFingerprint([binding]), + entries: [{ binding, lanes: requiredSlackIngressLanes(requirement) }] }; + expect(parseSlackConfiguredCoverage(proof).entries[0]?.binding.connectionId).toBe("connection"); + expect(() => parseSlackConfiguredCoverage({ ...proof, instanceId: "foreign" })).toThrow(); + expect(() => parseSlackConfiguredCoverage({ ...proof, bindingsFingerprint: "d".repeat(64) })).toThrow(); +}); +it("absence of a genuinely verified capability never discharges endpoint or connection IDs", () => { + expect(matchSlackConfiguredCoverage([requirement], undefined, { instanceId: "instance", generation: "c".repeat(64) })).toBeUndefined(); +}); diff --git a/server/src/services/chat-channels/slack/configured-coverage.ts b/server/src/services/chat-channels/slack/configured-coverage.ts new file mode 100644 index 00000000000..e6573de1e4a --- /dev/null +++ b/server/src/services/chat-channels/slack/configured-coverage.ts @@ -0,0 +1,66 @@ +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { createHash } from "node:crypto"; +import type { ChatAdapterCapabilities } from "@paperclipai/shared"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormBinding } from "./cloud-forms.js"; +export const SLACK_INGRESS_LANES = ["actions", "commands", "events", "forms", "modal_close", "modal_submit", "options"] as const; +export type SlackIngressLane = typeof SLACK_INGRESS_LANES[number]; +export type SlackConfiguredRequirement = Readonly<{ binding: SlackFormBinding; capabilities: Pick; registeredOptions: boolean }>; +export type SlackConfiguredCoverage = Readonly<{ version: 1; instanceId: string; generation: string; bindingsFingerprint: string; + entries: readonly Readonly<{ binding: SlackFormBinding; lanes: readonly SlackIngressLane[] }>[] }>; +const hash = (value: unknown) => createHash("sha256").update(JSON.stringify(value)).digest("hex"); +function exact(value: unknown, keys: string[]): Record { + if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).sort().join() !== [...keys].sort().join()) throw new Error("Invalid Slack configured coverage"); + return Object.fromEntries(Object.entries(value)); +} +export function requiredSlackIngressLanes(input: SlackConfiguredRequirement): readonly SlackIngressLane[] { + const lanes: SlackIngressLane[] = ["events"]; + if (input.capabilities.actions) lanes.push("actions"); + if (input.capabilities.slashCommands) lanes.push("commands"); + if (input.capabilities.modals) lanes.push("forms", "modal_close", "modal_submit"); + if (input.registeredOptions) lanes.push("options"); + return lanes.sort(); +} +export function slackConfiguredBindingsFingerprint(bindings: readonly SlackFormBinding[]): string { + return hash(bindings.map(b => [b.instanceId, b.companyId, b.endpointId, b.publicId, b.connectionId, b.runtimeGeneration, + b.teamId, b.appId, b.eventBindingId, b.credentialFingerprint]).sort((a, b) => JSON.stringify(a) < JSON.stringify(b) ? -1 : JSON.stringify(a) > JSON.stringify(b) ? 1 : 0)); +} +export function parseSlackConfiguredCoverage(value: unknown): SlackConfiguredCoverage { + const v = exact(value, ["version", "instanceId", "generation", "bindingsFingerprint", "entries"]); + if (v.version !== 1 || typeof v.instanceId !== "string" || !v.instanceId || typeof v.generation !== "string" || !/^[a-f0-9]{64}$/.test(v.generation) + || typeof v.bindingsFingerprint !== "string" || !/^[a-f0-9]{64}$/.test(v.bindingsFingerprint) || !Array.isArray(v.entries) || v.entries.length > 1000) throw new Error("Invalid Slack configured coverage"); + const seen = new Set(); + const entries = v.entries.map(value => { + const entry = exact(value, ["binding", "lanes"]), binding = parseSlackFormBinding(entry.binding); + const key = JSON.stringify([binding.companyId, binding.endpointId, binding.connectionId]); + if (binding.instanceId !== v.instanceId || seen.has(key) || !Array.isArray(entry.lanes) || !entry.lanes.length || new Set(entry.lanes).size !== entry.lanes.length + || entry.lanes.join() !== [...entry.lanes].sort().join()) throw new Error("Invalid Slack configured coverage"); + seen.add(key); + const lanes = entry.lanes.map(lane => { + const known = SLACK_INGRESS_LANES.find(value => value === lane); + if (!known) throw new Error("Unknown Slack configured lane"); + return known; + }); + return Object.freeze({ binding: Object.freeze(binding), lanes: Object.freeze(lanes) }); + }); + if (slackConfiguredBindingsFingerprint(entries.map(entry => entry.binding)) !== v.bindingsFingerprint) throw new Error("Slack configured fingerprint changed"); + return Object.freeze({ version: 1, instanceId: v.instanceId, generation: v.generation, bindingsFingerprint: v.bindingsFingerprint, entries: Object.freeze(entries) }); +} +export function matchSlackConfiguredCoverage(requirements: readonly SlackConfiguredRequirement[], verifiedCapabilities: VerifiedIdleCapabilities | undefined, + expected: { instanceId: string; generation: string }): { endpointIds: readonly string[]; connectionIds: readonly string[] } | undefined { + const incoming = verifiedCapabilities?.incomingWake; + if (!incoming || !incoming.routes.some(route => String(route) === "slack-broker-events-v1")) return; + const raw = Reflect.get(incoming, "slackBroker"); + if (raw === undefined) return; + const verifiedProof = parseSlackConfiguredCoverage(raw); + if (incoming.generation !== expected.generation || verifiedProof.instanceId !== expected.instanceId || verifiedProof.generation !== expected.generation || requirements.length > 1000) return; + const seen = new Set(); + for (const requirement of requirements) { + const binding = requirement.binding; + if (binding.instanceId !== expected.instanceId || seen.has(binding.endpointId)) return; + seen.add(binding.endpointId); + const entry = verifiedProof.entries.find(value => sameSlackFormBinding(value.binding, binding)); + if (!entry || requiredSlackIngressLanes(requirement).some(lane => !entry.lanes.includes(lane))) return; + } + if (verifiedProof.bindingsFingerprint !== slackConfiguredBindingsFingerprint(requirements.map(value => value.binding))) return; + return { endpointIds: [...seen].sort(), connectionIds: [...new Set(requirements.map(value => value.binding.connectionId))].sort() }; +} diff --git a/server/src/services/chat-channels/slack/form-ack-work.test.ts b/server/src/services/chat-channels/slack/form-ack-work.test.ts new file mode 100644 index 00000000000..cbef64c5b3a --- /dev/null +++ b/server/src/services/chat-channels/slack/form-ack-work.test.ts @@ -0,0 +1,30 @@ +import { generateKeyPairSync, sign } from "node:crypto"; +import { afterEach, expect, it, vi } from "vitest"; +import { acceptSlackFormAckReceipt } from "./form-ack-work.js"; +import { parseSlackFormFinalizedReceipt } from "./ack-receipt.js"; + +const original = { stack: process.env.PAPERCLIP_CLOUD_STACK_ID, jwks: process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS }; +afterEach(() => { + if (original.stack === undefined) delete process.env.PAPERCLIP_CLOUD_STACK_ID; else process.env.PAPERCLIP_CLOUD_STACK_ID = original.stack; + if (original.jwks === undefined) delete process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS; else process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS = original.jwks; +}); +it("rejects a genuinely signed foreign-company receipt and its action ID before any SQL", async () => { + const keys = generateKeyPairSync("ed25519"), kid = "ack-company-test"; + process.env.PAPERCLIP_CLOUD_STACK_ID = "instance"; + process.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS = JSON.stringify({ keys: [{ ...keys.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] }); + const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: "test", + binding: { version: 1, instanceId: "instance", companyId: "foreign", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "T1", appId: "A1", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }, + operationId: "c".repeat(64), deliveryId: "slack_form_" + "c".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "U1", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-06T00:00:00.000Z" }); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const input = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: "instance", receipt }); + const receiptJws = input + "." + sign(null, Buffer.from(input), keys.privateKey).toString("base64url"); + const tx = { select: vi.fn(), update: vi.fn() }; + await expect(acceptSlackFormAckReceipt(tx, { companyId: "current", instanceId: "instance", leaseId: "lease", receiptJws })) + .rejects.toMatchObject({ code: "form_ack_company_changed" }); + expect(tx.select).not.toHaveBeenCalled(); + expect(tx.update).not.toHaveBeenCalled(); +}); diff --git a/server/src/services/chat-channels/slack/form-ack-work.ts b/server/src/services/chat-channels/slack/form-ack-work.ts new file mode 100644 index 00000000000..c038d19659e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-ack-work.ts @@ -0,0 +1,201 @@ +import { replaceSlackFormReceiptSignature } from "./form-signature-refresh.js"; +import { readSlackFormOriginalProofs } from "./form-opening-proof.js"; +import { createHash } from "node:crypto"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { verifySlackFormFinalizedReceipt } from "./receipt-authority.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormProofAcceptance } from "./form-session.js"; +import { parseSlackFormFinalizedReceipt, slackFormFinalizedReceiptDigest, type SlackFormFinalizedReceipt } from "./ack-receipt.js"; + +type Tx = Pick; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function record(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return fail("form_ack_record_invalid"); + return Object.fromEntries(Object.entries(value)); +} +export type SlackFormAckWork = Readonly<{ + version: 1; phase: "pending" | "accepted" | "gc_pending" | "transferred"; leaseId: string; deliveryId: string; + receiptJws: string | null; receipt: SlackFormFinalizedReceipt | null; +}>; +export function parseSlackFormAckWork(value: unknown): SlackFormAckWork { + const v = record(value); + if (Object.keys(v).sort().join() !== "deliveryId,leaseId,phase,receipt,receiptJws,version" || v.version !== 1 + || typeof v.leaseId !== "string" || !v.leaseId.length || v.leaseId.length > 200 + || typeof v.deliveryId !== "string" || !/^slack_form_[a-f0-9]{64}$/.test(v.deliveryId) + || v.phase !== "pending" && v.phase !== "accepted" && v.phase !== "gc_pending" && v.phase !== "transferred" + || v.receiptJws !== null && (typeof v.receiptJws !== "string" || Buffer.byteLength(v.receiptJws, "ascii") > 12000 || !/^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$/.test(v.receiptJws))) return fail("form_ack_work_invalid"); + const receipt = v.receipt === null ? null : parseSlackFormFinalizedReceipt(v.receipt); + if (v.phase === "pending" ? receipt !== null || v.receiptJws !== null : !receipt || typeof v.receiptJws !== "string") return fail("form_ack_work_invalid"); + if (receipt && receipt.deliveryId !== v.deliveryId) return fail("form_ack_delivery_changed"); + return { version: 1, phase: v.phase, leaseId: v.leaseId, deliveryId: v.deliveryId, receiptJws: typeof v.receiptJws === "string" ? v.receiptJws : null, receipt }; +} +export function assertSlackFormReceiptHistory(row: Pick, receipt: SlackFormFinalizedReceipt): void { + const intent = record(row.result?.terminalIntent), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance); + const binding = parseSlackFormBinding(row.payload.binding), view = intent.view; + let viewDigest: string | null = null; + if (view !== null) { + const parsed = record(view); + if (Object.keys(parsed).sort().join() !== "externalId,hash,id" || typeof parsed.id !== "string" || typeof parsed.hash !== "string" || typeof parsed.externalId !== "string") + return fail("form_ack_view_changed"); + viewDigest = createHash("sha256").update(JSON.stringify([parsed.id, parsed.hash, parsed.externalId]), "utf8").digest("hex"); + } + if (row.kind !== "slack_form_session" || row.companyId !== receipt.binding.companyId || row.endpointId !== receipt.binding.endpointId + || !sameSlackFormBinding(binding, receipt.binding) || !sameSlackFormBinding(proof.binding, binding) + || row.id !== receipt.appReceiptId || proof.appReceiptId !== row.id || intent.appReceiptId !== row.id + || receipt.environment !== row.payload.environment || receipt.operationId !== row.payload.operationId || receipt.operationId !== intent.operationId + || receipt.deliveryId !== intent.deliveryId || receipt.grantId !== row.payload.grantId || receipt.publicationId !== row.payload.publicationId + || receipt.interactionId !== row.payload.interactionId || receipt.userId !== row.payload.userId || receipt.outcome !== intent.outcome + || receipt.terminalDigest !== proof.terminalDigest || receipt.outcome !== proof.outcome || receipt.viewDigest !== viewDigest + || row.result?.phase !== (receipt.outcome === "no_effect" ? "not_opened" : receipt.outcome)) + fail("form_ack_terminal_identity_changed"); +} +export async function stageSlackFormNormalAck(tx: Tx, input: { companyId: string; instanceId: string; leaseId: string; deliveryIds: readonly string[] }): Promise { + const ids = [...new Set(input.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)))]; + if (!ids.length) return; + if (ids.length > 50 || !input.leaseId.length || input.leaseId.length > 200) return fail("form_ack_batch_invalid"); + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"), + inArray(chatActions.providerActionId, ids.map(id => `slack-form-session:${id.slice("slack_form_".length)}`)))).for("update"); + if (rows.length !== ids.length) return fail("form_ack_session_missing"); + for (const row of rows) { + const intent = record(row.result?.terminalIntent), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance); + if (row.companyId !== input.companyId || proof.binding.companyId !== input.companyId || proof.binding.instanceId !== input.instanceId || proof.appReceiptId !== row.id || !ids.includes(String(intent.deliveryId)) + || row.companyId !== proof.binding.companyId || row.endpointId !== proof.binding.endpointId + || !["submitted", "closed", "not_opened"].includes(String(row.result?.phase))) return fail("form_ack_authority_changed"); + if (row.result?.formAck !== undefined) { + const old = parseSlackFormAckWork(row.result.formAck); + if (old.deliveryId !== intent.deliveryId) return fail("form_ack_delivery_changed"); + if (old.phase !== "pending") continue; + } + const work: SlackFormAckWork = { version: 1, phase: "pending", leaseId: input.leaseId, deliveryId: String(intent.deliveryId), receiptJws: null, receipt: null }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const staged = await tx.update(chatActions).set({ result: { ...row.result, formAck: work, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (staged.length !== 1) return fail("form_ack_stage_changed"); + } +} +export async function acceptSlackFormAckReceipt(tx: Tx, input: { companyId: string; instanceId: string; leaseId: string; receiptJws: string }): Promise { + const receipt = verifyRetainedSlackAckReceipt(input.receiptJws); + if (receipt.binding.companyId !== input.companyId) return fail("form_ack_company_changed"); + if (receipt.binding.instanceId !== input.instanceId) return fail("form_ack_instance_changed"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, receipt.appReceiptId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, receipt.binding.endpointId), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row) return fail("form_ack_session_missing"); + assertSlackFormReceiptHistory(row, receipt); + const old = parseSlackFormAckWork(row.result?.formAck); + if (old.leaseId !== input.leaseId || old.deliveryId !== receipt.deliveryId) return fail("form_ack_lease_changed"); + if (old.receipt && !isDeepStrictEqual(old.receipt, receipt)) return fail("form_ack_receipt_changed"); + if (old.phase === "transferred" || old.phase === "gc_pending") return row.id; + const work: SlackFormAckWork = { ...old, phase: "accepted", receiptJws: input.receiptJws, receipt }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...row.result, formAck: work, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return fail("form_ack_acceptance_changed"); + return row.id; +} +export async function transferSlackFormAckReceipt(db: Db, input: { companyId: string; actionId: string; instanceId: string; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + let [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"))).limit(1); + if (!row) return fail("form_ack_session_missing"); + if (row.companyId !== input.companyId) return fail("form_ack_company_changed"); + let work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase === "transferred") return; + if (work.phase === "gc_pending") return releaseSlackAckFact(db, row, input); + if (work.phase !== "accepted" || !work.receipt || !work.receiptJws || work.receipt.binding.companyId !== input.companyId || work.receipt.binding.instanceId !== input.instanceId) + return fail("form_ack_acceptance_missing"); + assertSlackFormReceiptHistory(row, work.receipt); + const refreshResponse = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, operation: { + kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + refreshAckReceipt: { leaseId: work.leaseId, receiptDigest: slackFormFinalizedReceiptDigest(work.receipt), appReceiptId: row.id } } })); + if (Object.keys(refreshResponse).join() !== "ackReceiptJws" || typeof refreshResponse.ackReceiptJws !== "string") + throw new SlackCloudReceiptError("form_signature_refresh_unconfirmed", "retry"); + const currentJws = refreshResponse.ackReceiptJws; + const refreshed = verifyRetainedSlackAckReceipt(currentJws); + if (!isDeepStrictEqual(refreshed, work.receipt)) return fail("form_signature_canonical_changed"); + row = await replaceSlackFormReceiptSignature(db, row, "formAck", currentJws); + input.onSnapshot?.(row); + work = parseSlackFormAckWork(row.result?.formAck); + if (!work.receipt) return fail("form_ack_acceptance_missing"); + const originals = readSlackFormOriginalProofs(row); + const operation = { kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + acceptedAckReceipt: currentJws }; + let response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, + operation: { ...operation, originalOpening: originals.opening } })); + const digest = slackFormFinalizedReceiptDigest(work.receipt); + if (response.acceptedReceiptDigest !== digest || typeof response.needsTerminalOriginal !== "boolean") return fail("form_ack_transfer_receipt_changed"); + if (response.needsTerminalOriginal) { + if (!originals.terminal) return fail("form_terminal_original_proof_missing"); + response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, + operation: { ...operation, terminalOriginal: originals.terminal } })); + if (response.acceptedReceiptDigest !== digest || response.needsTerminalOriginal !== false) return fail("form_ack_transfer_receipt_changed"); + } + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return fail("form_ack_transfer_custody_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const retainedResult = Object.fromEntries(Object.entries(current.result ?? {}).filter(([key]) => key !== "openingProof")); + const completed = await tx.update(chatActions).set({ result: { ...retainedResult, formAck: { ...work, phase: "gc_pending" }, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (completed.length !== 1) throw new SlackCloudReceiptError("form_ack_transfer_custody_changed", "retry"); + }); +} + +export async function reconcileSlackFormNormalAck(db: Db, input: { companyId: string; actionId: string; instanceId: string; connector: unknown; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "slack_form_session"))).limit(1); + if (!row) return fail("form_ack_session_missing"); + if (row.companyId !== input.companyId) return fail("form_ack_company_changed"); + const work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase === "transferred") return; + if (work.phase === "pending") { + const binding = parseSlackFormBinding(row.payload.binding); + if (binding.companyId !== input.companyId || binding.instanceId !== input.instanceId || !input.connector || typeof input.connector !== "object") return fail("form_ack_instance_changed"); + const acknowledge = Reflect.get(input.connector, "acknowledgeEventsWithReceipts"); + if (typeof acknowledge !== "function") return fail("form_ack_receipts_unsupported"); + const response = record(await Reflect.apply(acknowledge, input.connector, [{ subject: "github-bot-events", companyId: row.companyId, + leaseId: work.leaseId, deliveryIds: [work.deliveryId] }])); + if (!Array.isArray(response.formReceipts) || response.formReceipts.length !== 1) return fail("form_ack_receipt_missing"); + const receipt = record(response.formReceipts[0]); + if (receipt.deliveryId !== work.deliveryId || typeof receipt.receiptJws !== "string") return fail("form_ack_receipt_changed"); + await db.transaction(tx => acceptSlackFormAckReceipt(tx, { companyId: input.companyId, instanceId: input.instanceId, leaseId: work.leaseId, receiptJws: String(receipt.receiptJws) })); + } + await transferSlackFormAckReceipt(db, input); +} + +function verifyRetainedSlackAckReceipt(compactJws: string): SlackFormFinalizedReceipt { + try { return verifySlackFormFinalizedReceipt({ compactJws }); } + catch (error) { + if (error instanceof Error && ["PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is not configured", "PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is invalid", + "Cloud runtime identity uses an unknown signing key"].includes(error.message)) throw new SlackCloudReceiptError("form_ack_verifier_unavailable", "retry"); + return fail("form_ack_receipt_invalid"); + } +} +async function releaseSlackAckFact(db: Db, row: typeof chatActions.$inferSelect, input: { companyId: string; actionId: string; instanceId: string; + protocol(values: { subject: string; companyId: string; operation: Record }): Promise }): Promise { + const work = parseSlackFormAckWork(row.result?.formAck); + if (work.phase !== "gc_pending" || !work.receipt || work.receipt.binding.companyId !== input.companyId || work.receipt.binding.instanceId !== input.instanceId) + return fail("form_ack_gc_authority_changed"); + assertSlackFormReceiptHistory(row, work.receipt); + const digest = slackFormFinalizedReceiptDigest(work.receipt); + const response = record(await input.protocol({ subject: "github-bot-events", companyId: row.companyId, operation: { + kind: "receipt", binding: work.receipt.binding, operationId: work.receipt.operationId, deliveryId: work.deliveryId, + releaseAckFact: { leaseId: work.leaseId, receiptDigest: digest, appReceiptId: row.id } } })); + if (Object.keys(response).join() !== "releasedAckDigest" || response.releasedAckDigest !== digest) + throw new SlackCloudReceiptError("form_ack_gc_unconfirmed", "retry"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) + throw new SlackCloudReceiptError("form_ack_gc_changed", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...current.result, formAck: { ...work, phase: "transferred" }, nextAttemptAt: null }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw new SlackCloudReceiptError("form_ack_gc_changed", "retry"); + }); +} diff --git a/server/src/services/chat-channels/slack/form-canonical-state.test.ts b/server/src/services/chat-channels/slack/form-canonical-state.test.ts new file mode 100644 index 00000000000..7262b53fe82 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-canonical-state.test.ts @@ -0,0 +1,26 @@ +import { expect, it } from "vitest"; +import { parseSlackFormAckWork } from "./form-ack-work.js"; +import { parseSlackFormFinalizedReceipt } from "./ack-receipt.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; + +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const receipt = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: "test", binding, + operationId: "c".repeat(64), deliveryId: "slack_form_" + "c".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-06T00:00:00.000Z" }); +it("keeps GC debt explicitly distinct from completed ACK history", () => { + const pending = parseSlackFormAckWork({ version: 1, phase: "gc_pending", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }); + expect(pending.phase).toBe("gc_pending"); expect(pending.leaseId).toBe("exact-lease"); expect(pending.receipt).toEqual(receipt); + expect(parseSlackFormAckWork({ ...pending, phase: "transferred" }).phase).toBe("transferred"); +}); +it("GC cannot be constructed from absent proof or a different delivery", () => { + const work = { version: 1, phase: "gc_pending", leaseId: "exact-lease", deliveryId: receipt.deliveryId, receiptJws: "a.b.c", receipt }; + expect(() => parseSlackFormAckWork({ ...work, receipt: null })).toThrow(); + expect(() => parseSlackFormAckWork({ ...work, deliveryId: "slack_form_" + "e".repeat(64) })).toThrow(); + expect(() => parseSlackFormAckWork({ ...work, phase: "released_by_absence" })).toThrow(); +}); +it("ordinary old pending and completed states remain closed and do not acquire new authority", () => { + expect(parseSlackFormAckWork({ version: 1, phase: "pending", leaseId: "old", deliveryId: receipt.deliveryId, receipt: null, receiptJws: null }).phase).toBe("pending"); + expect(() => parseSlackFormAckWork({ version: 1, phase: "pending", leaseId: "old", deliveryId: receipt.deliveryId, receipt, receiptJws: "a.b.c" })).toThrow(); +}); diff --git a/server/src/services/chat-channels/slack/form-delivery.ts b/server/src/services/chat-channels/slack/form-delivery.ts new file mode 100644 index 00000000000..c03a709b792 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-delivery.ts @@ -0,0 +1,85 @@ +import { retainSlackFormOpeningProof } from "./form-opening-proof.js"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, toolConnections, type Db } from "@paperclipai/db"; +import type { chatCredentialMutationLease } from "../../chat-credential-mutation-lease.js"; +import type { createChatCredentials } from "../credentials.js"; +import { SlackCloudReceiptError, type SlackCloudEvent } from "./cloud-event.js"; +import { slackFormSetupRecord, currentSlackFormOwnerBinding, parseSlackFormOperation, sameSlackFormBinding, lookupSlackFormDelivery, withSlackFormOpenContext, + type SlackFormDeliveryAuthority } from "./cloud-forms.js"; +import { slackFormConnectorIdentity } from "./cloud-consumer.js"; +import { acceptSlackFormClosed, acceptSlackFormNoEffect, acceptSlackFormTerminalProof, readSlackFormTerminalIntent, readSlackFormTerminalRequest } from "./form-session.js"; +import type { SlackFormProtocol } from "./form-owner.js"; +import { resolveSlackCloudSigningSecret } from "./cloud-signing-secret.js"; + +export function createSlackFormDeliveryOwner(db: Db, options: { + connector(): unknown; protocol: SlackFormProtocol; + withCredentialMutationLease: ReturnType; + resolveCredentials: ReturnType["resolveCredentials"]; + webhookUrl(publicId: string): string; + handleWebhook(publicId: string, request: Request, generation: number, acceptedAt: number): Promise; +}) { + return async (event: SlackCloudEvent): Promise => { + const identity = slackFormConnectorIdentity(options.connector()); + if (!identity) throw new SlackCloudReceiptError("form_connector_identity_unavailable", "attention"); + const packet = event.payload.slackFormOpen; + if (!packet || typeof packet !== "object" || Array.isArray(packet)) throw new SlackCloudReceiptError("form_receipt_invalid", "invalid"); + const original = parseSlackFormOperation(Reflect.get(packet, "operation")), binding = original.binding; + if (binding.instanceId !== identity.instanceId || Reflect.get(packet, "environment") !== identity.environment) throw new SlackCloudReceiptError("form_instance_changed", "attention"); + const [endpoint] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId))).limit(1); + if (!endpoint || endpoint.provider !== "slack") throw new SlackCloudReceiptError("form_endpoint_custody_missing", "attention"); + let authority: SlackFormDeliveryAuthority; + const local = currentSlackFormOwnerBinding(slackFormSetupRecord(endpoint.setup)); + if (local && sameSlackFormBinding(local, binding) && ["active", "verifying", "attention"].includes(endpoint.status)) { + const signingSecret = await options.withCredentialMutationLease(endpoint, async lease => { + const [current] = await db.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints) + .innerJoin(toolConnections, and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, chatEndpoints.companyId))) + .where(and(eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId))).limit(1); + const owner = current && currentSlackFormOwnerBinding(slackFormSetupRecord(current.endpoint.setup)); + if (!current || !owner || !sameSlackFormBinding(owner, binding) || !["active", "verifying", "attention"].includes(current.endpoint.status) + || current.connection.status !== "active" || !current.connection.enabled) throw new SlackCloudReceiptError("form_current_authority_changed", "attention"); + const secret = await resolveSlackCloudSigningSecret(options.resolveCredentials, current.endpoint, db, ["signingSecret"]); + await lease.assertOwned(); return secret; + }); + authority = { kind: "current", binding, ...identity, signingSecret }; + } else { + const [owner] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + sql`${chatActions.kind} in ('slack_form_owner','slack_form_session')`, sql`${chatActions.payload}->'binding' = ${JSON.stringify(binding)}::jsonb`, + sql`${chatActions.payload}->>'environment' = ${identity.environment}`)).limit(1); + if (!owner) throw new SlackCloudReceiptError("form_retained_owner_missing", "attention"); + authority = { kind: "retained", binding, ...identity }; + } + const lookup = await lookupSlackFormDelivery(event, authority, values => options.protocol({ subject: "github-bot-events", companyId: binding.companyId, + operation: { kind: "receipt", ...values } })); + const op = lookup.operation; + let terminal = await readSlackFormTerminalRequest(db, binding, op.id); + const actions = Reflect.get(lookup.originalPayload, "actions"); + const actionId = Array.isArray(actions) && actions.length === 1 && actions[0] && typeof actions[0] === "object" ? Reflect.get(actions[0], "action_id") : null; + if (typeof actionId !== "string") throw new SlackCloudReceiptError("form_issued_action_invalid", "attention"); + if (!terminal && op.closed?.kind === "closed") await db.transaction(tx => acceptSlackFormClosed(tx, op, actionId, identity.environment)); + else if (!terminal && op.rejection) await db.transaction(tx => acceptSlackFormNoEffect(tx, op, actionId, identity.environment)); + else if (!terminal && authority.kind === "current" && op.phase === "confirmed" && !op.closed) { + const rawBody = Reflect.get(packet, "rawBody"), timestamp = Reflect.get(packet, "timestamp"), signature = Reflect.get(packet, "signature"); + if (typeof rawBody !== "string" || typeof timestamp !== "string" || typeof signature !== "string") throw new SlackCloudReceiptError("form_original_invalid", "invalid"); + const url = new URL(options.webhookUrl(binding.publicId)); + if (!url.pathname.endsWith(`/api/chat-webhooks/${binding.publicId}/slack`)) throw new SlackCloudReceiptError("form_webhook_url_changed", "attention"); + const request = new Request(url, { method: "POST", + headers: { "content-type": "application/x-www-form-urlencoded", "x-slack-request-timestamp": timestamp, "x-slack-signature": signature }, body: Buffer.from(rawBody, "base64") }); + const response = await withSlackFormOpenContext(op, binding, + () => options.handleWebhook(binding.publicId, request, binding.runtimeGeneration, Date.parse(op.acceptedAt)), identity.environment); + if (!response.ok) throw new SlackCloudReceiptError("form_action_callback_failed", "retry"); + } + terminal = await readSlackFormTerminalRequest(db, binding, op.id); + if (!terminal) throw new SlackCloudReceiptError(op.phase === "confirmed" ? "form_view_active" : "form_resource_unresolved", op.phase === "confirmed" ? "retry" : "attention"); + const intent = await readSlackFormTerminalIntent(db, op); + if (!intent || intent.deliveryId !== event.id) throw new SlackCloudReceiptError("form_terminal_intent_missing", "attention"); + const proof = await options.protocol({ subject: "github-bot-events", companyId: binding.companyId, operation: terminal }); + if (!proof || typeof proof !== "object" || Reflect.get(proof, "instanceId") !== identity.instanceId || Reflect.get(proof, "environment") !== identity.environment) + throw new SlackCloudReceiptError("form_terminal_proof_invalid", "attention"); + const proven = parseSlackFormOperation(Reflect.get(proof, "operation")); + await db.transaction(async tx => { + await acceptSlackFormTerminalProof(tx, { intent, operation: Reflect.get(proof, "operation"), currentInstanceId: identity.instanceId }); + await retainSlackFormOpeningProof(tx, { companyId: binding.companyId, appReceiptId: intent.appReceiptId, operation: proven, + original: { rawBody: Reflect.get(packet, "rawBody"), timestamp: Reflect.get(packet, "timestamp"), signature: Reflect.get(packet, "signature") } }); + }); + }; +} diff --git a/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts b/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts new file mode 100644 index 00000000000..6d7d03da359 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-enrollment-identity.test.ts @@ -0,0 +1,39 @@ +import { createHash } from "node:crypto"; +import { expect, it } from "vitest"; +import { parseSlackFormPublicEnrollment } from "./cloud-forms.js"; +import { slackFormEnrollmentActionId, slackFormEnrollmentCompatibleActionIds, isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; + +const enrollment = parseSlackFormPublicEnrollment({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", + publicId: "p".repeat(43), connectionId: "connection", runtimeGeneration: 1, bindingId: "slack-event:" + "a".repeat(64), teamId: "TTEST", appId: "ATEST" }); +const legacyId = (value: unknown) => `slack-form-owner:${createHash("sha256").update(JSON.stringify(value)).digest("hex")}`; +it("uses a complete typed tuple independent of input and JSONB object order", () => { + const reversed = parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(enrollment).reverse())); + const postgresOrder = parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(enrollment).sort(([a], [b]) => Buffer.byteLength(a) - Buffer.byteLength(b) || a.localeCompare(b)))); + expect(slackFormEnrollmentActionId(reversed)).toBe(slackFormEnrollmentActionId(enrollment)); + expect(slackFormEnrollmentActionId(postgresOrder)).toBe(slackFormEnrollmentActionId(enrollment)); + expect(legacyId(reversed)).not.toBe(legacyId(enrollment)); +}); +it("recognizes only the exact known legacy producer orders for the same full authority tuple", () => { + const e = enrollment; + const connectionFirst = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, endpointId: e.endpointId, + connectionId: e.connectionId, publicId: e.publicId, runtimeGeneration: e.runtimeGeneration, teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + const publicFirst = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, endpointId: e.endpointId, + publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + for (const previous of [enrollment, connectionFirst, publicFirst]) + expect(isSlackFormEnrollmentActionId(legacyId(previous), parseSlackFormPublicEnrollment(Object.fromEntries(Object.entries(previous).reverse())))).toBe(true); + expect(slackFormEnrollmentCompatibleActionIds(enrollment)).toHaveLength(4); + expect(isSlackFormEnrollmentActionId(legacyId(Object.fromEntries(Object.entries(enrollment).reverse())), enrollment)).toBe(false); + expect(isSlackFormEnrollmentActionId("slack-form-owner:" + "0".repeat(64), enrollment)).toBe(false); +}); +it("every authority field is bound for canonical and compatible identities", () => { + const variants = { instanceId: "other-instance", companyId: "other-company", endpointId: "other-endpoint", publicId: "q".repeat(43), + connectionId: "other-connection", runtimeGeneration: 2, bindingId: "slack-event:" + "b".repeat(64), teamId: "TOTHER", appId: "AOTHER" }; + for (const [key, value] of Object.entries(variants)) { + const changed = parseSlackFormPublicEnrollment({ ...enrollment, [key]: value }); + for (const id of slackFormEnrollmentCompatibleActionIds(enrollment)) expect(isSlackFormEnrollmentActionId(id, changed)).toBe(false); + } +}); +it("does not accept a different schema version or extra authority keys", () => { + expect(() => parseSlackFormPublicEnrollment({ ...enrollment, version: 2 })).toThrow(); + expect(() => parseSlackFormPublicEnrollment({ ...enrollment, unexpected: "authority" })).toThrow(); +}); diff --git a/server/src/services/chat-channels/slack/form-enrollment-identity.ts b/server/src/services/chat-channels/slack/form-enrollment-identity.ts new file mode 100644 index 00000000000..b5f893449d3 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-enrollment-identity.ts @@ -0,0 +1,27 @@ +import { createHash } from "node:crypto"; +import { parseSlackFormPublicEnrollment, type SlackFormPublicEnrollment } from "./cloud-forms.js"; + +function actionId(value: readonly unknown[] | SlackFormPublicEnrollment): string { + return `slack-form-owner:${createHash("sha256").update(JSON.stringify(value)).digest("hex")}`; +} +export function slackFormEnrollmentActionId(input: SlackFormPublicEnrollment): string { + const e = parseSlackFormPublicEnrollment(input); + return actionId(["slack-form-owner/v2", e.version, e.instanceId, e.companyId, e.endpointId, e.publicId, + e.connectionId, e.runtimeGeneration, e.bindingId, e.teamId, e.appId]); +} +export function slackFormEnrollmentCompatibleActionIds(input: SlackFormPublicEnrollment): readonly string[] { + const e = parseSlackFormPublicEnrollment(input); + const projected: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, + bindingId: e.bindingId, teamId: e.teamId, appId: e.appId }; + const bindingDerived: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, connectionId: e.connectionId, publicId: e.publicId, runtimeGeneration: e.runtimeGeneration, + teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + const publicFirstBinding: SlackFormPublicEnrollment = { version: e.version, instanceId: e.instanceId, companyId: e.companyId, + endpointId: e.endpointId, publicId: e.publicId, connectionId: e.connectionId, runtimeGeneration: e.runtimeGeneration, + teamId: e.teamId, appId: e.appId, bindingId: e.bindingId }; + return [slackFormEnrollmentActionId(e), actionId(projected), actionId(bindingDerived), actionId(publicFirstBinding)]; +} +export function isSlackFormEnrollmentActionId(providerActionId: unknown, enrollment: SlackFormPublicEnrollment): boolean { + return typeof providerActionId === "string" && slackFormEnrollmentCompatibleActionIds(enrollment).includes(providerActionId); +} diff --git a/server/src/services/chat-channels/slack/form-idle-history-sql.ts b/server/src/services/chat-channels/slack/form-idle-history-sql.ts new file mode 100644 index 00000000000..5644e40b31e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-idle-history-sql.ts @@ -0,0 +1,47 @@ +import { sql, type SQL, type SQLWrapper } from "drizzle-orm"; + +type Columns = { id: SQLWrapper; companyId: SQLWrapper; endpointId: SQLWrapper; kind: SQLWrapper; status: SQLWrapper; payload: SQLWrapper; result: SQLWrapper }; +function objectKeys(value: SQL, keys: readonly string[]): SQL { + return sql`jsonb_typeof(${value}) = 'object' AND ${value} ?& ${[...keys]}::text[] + AND (SELECT count(*) FROM jsonb_object_keys(CASE WHEN jsonb_typeof(${value})='object' THEN ${value} ELSE '{}'::jsonb END)) = ${keys.length}`; +} +export function completedSlackFormSessionSql(action: Columns): SQL { + const p = sql`${action.payload}`, r = sql`${action.result}`, work = sql`(${r}->'formAck')`, receipt = sql`(${work}->'receipt')`, + binding = sql`(${p}->'binding')`, proof = sql`(${r}->'cloudProofAcceptance')`, intent = sql`(${r}->'terminalIntent')`, view = sql`(${intent}->'view')`; + const digest = sql`encode(sha256(convert_to('[' || to_json(${view}->>'id')::text || ',' || to_json(${view}->>'hash')::text || ',' || to_json(${view}->>'externalId')::text || ']', 'UTF8')), 'hex')`; + return sql`coalesce(${action.kind}='slack_form_session' AND (${action.status} IN ('processed','cancelled') OR (${action.status}='failed' AND ${receipt}->>'outcome'='no_effect')) + AND ${r}->'nextAttemptAt' = 'null'::jsonb + AND ${objectKeys(work, ['version','phase','leaseId','deliveryId','receiptJws','receipt'])} + AND ${work}->'version' = '1'::jsonb AND ${work}->>'phase'='transferred' + AND jsonb_typeof(${work}->'receiptJws')='string' AND length(${work}->>'receiptJws') BETWEEN 1 AND 12000 + AND ${work}->>'receiptJws' ~ '^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$' + AND ${objectKeys(receipt, ['version','kind','binding','environment','operationId','deliveryId','appReceiptId','grantId','publicationId','interactionId','userId','viewDigest','terminalDigest','outcome','finalizedAt'])} + AND ${receipt}->'version'='1'::jsonb AND ${receipt}->>'kind'='slack_form_finalized' + AND ${objectKeys(binding, ['version','instanceId','companyId','endpointId','publicId','connectionId','runtimeGeneration','teamId','appId','eventBindingId','credentialFingerprint'])} + AND ${binding}->'version'='1'::jsonb AND jsonb_typeof(${binding}->'runtimeGeneration')='number' + AND ${binding}->>'runtimeGeneration' ~ '^[1-9][0-9]{0,15}$' + AND (length(${binding}->>'runtimeGeneration') < 16 OR ${binding}->>'runtimeGeneration' <= '9007199254740991') + AND ${binding}->>'publicId' ~ '^[A-Za-z0-9_-]{43}$' AND ${binding}->>'teamId' ~ '^T[A-Z0-9]+$' AND ${binding}->>'appId' ~ '^A[A-Z0-9]+$' + AND ${binding}->>'eventBindingId' ~ '^slack-event:[a-f0-9]{64}$' AND ${binding}->>'credentialFingerprint' ~ '^[a-f0-9]{64}$' + AND NOT EXISTS (SELECT 1 FROM unnest(ARRAY['instanceId','companyId','endpointId','publicId','connectionId','teamId','appId','eventBindingId','credentialFingerprint']) key + WHERE jsonb_typeof(${binding}->key) IS DISTINCT FROM 'string' OR length(${binding}->>key) NOT BETWEEN 1 AND 200) + AND ${objectKeys(proof, ['version','binding','operationId','appReceiptId','outcome','terminalDigest','acceptedAt'])} + AND ${proof}->'version'='1'::jsonb AND ${receipt}->'binding'=${binding} AND ${proof}->'binding'=${binding} + AND ${proof}->>'acceptedAt' ~ '^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9][.][0-9]{3}Z$' + AND ${receipt}->>'finalizedAt' ~ '^[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01])T([01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9][.][0-9]{3}Z$' + AND ${receipt}->>'environment' IN ('development','test','staging','production') + AND ${binding}->>'companyId'=${action.companyId}::text AND ${binding}->>'endpointId'=${action.endpointId}::text + AND ${receipt}->>'appReceiptId'=${action.id}::text AND ${proof}->>'appReceiptId'=${action.id}::text AND ${intent}->>'appReceiptId'=${action.id}::text + AND ${receipt}->'environment'=${p}->'environment' + AND ${receipt}->'operationId'=${p}->'operationId' AND ${receipt}->'operationId'=${intent}->'operationId' AND ${receipt}->'operationId'=${proof}->'operationId' + AND ${receipt}->>'operationId' ~ '^[a-f0-9]{64}$' AND ${receipt}->>'deliveryId'='slack_form_'||(${receipt}->>'operationId') + AND ${receipt}->'deliveryId'=${intent}->'deliveryId' AND ${receipt}->'deliveryId'=${work}->'deliveryId' + AND ${receipt}->'grantId'=${p}->'grantId' AND ${receipt}->'publicationId'=${p}->'publicationId' AND ${receipt}->'interactionId'=${p}->'interactionId' + AND ${receipt}->'userId'=${p}->'userId' AND ${receipt}->'outcome'=${intent}->'outcome' AND ${receipt}->'outcome'=${proof}->'outcome' + AND ${receipt}->'terminalDigest'=${proof}->'terminalDigest' AND ${receipt}->>'terminalDigest' ~ '^[a-f0-9]{64}$' + AND ${r}->>'phase'=CASE WHEN ${receipt}->>'outcome'='no_effect' THEN 'not_opened' ELSE ${receipt}->>'outcome' END + AND CASE WHEN ${intent}->>'outcome'='no_effect' THEN ${view}='null'::jsonb AND ${receipt}->'viewDigest'='null'::jsonb + WHEN ${intent}->>'outcome' IN ('submitted','closed') THEN ${objectKeys(view, ['id','hash','externalId'])} + AND jsonb_typeof(${view}->'id')='string' AND jsonb_typeof(${view}->'hash')='string' AND jsonb_typeof(${view}->'externalId')='string' + AND ${receipt}->>'viewDigest'=${digest} ELSE false END, false)`; +} diff --git a/server/src/services/chat-channels/slack/form-idle-history.ts b/server/src/services/chat-channels/slack/form-idle-history.ts new file mode 100644 index 00000000000..da10215d85b --- /dev/null +++ b/server/src/services/chat-channels/slack/form-idle-history.ts @@ -0,0 +1,85 @@ +import { slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { isDeepStrictEqual } from "node:util"; +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { matchSlackConfiguredCoverage, type SlackConfiguredRequirement } from "./configured-coverage.js"; +import type { chatActions } from "@paperclipai/db"; +import { parseSlackFormBinding, parseSlackFormPublicEnrollment, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormAckWork, assertSlackFormReceiptHistory } from "./form-ack-work.js"; + +export type SlackFormHistoryRow = Pick; +export type SlackFormHistoryCoverage = { requirements: readonly SlackConfiguredRequirement[]; verified: VerifiedIdleCapabilities; + expected: { instanceId: string; generation: string; environment: string }; currentOwners?: ReadonlyMap }; + +/** Prepare bounded current coverage once; history pages retain no protocol rows. + * Session-related actions are supplied from the same page or indexed DB joins. + */ +export function createSlackFormHistoryClassifier(coverage?: SlackFormHistoryCoverage) { + const matched = coverage ? matchSlackConfiguredCoverage(coverage.requirements, coverage.verified, coverage.expected) : undefined; + const endpoints = new Set(matched?.endpointIds), connections = new Set(matched?.connectionIds); + const bindings = new Map(coverage?.requirements.map(requirement => [ + `${requirement.binding.companyId}:${requirement.binding.endpointId}`, requirement.binding])); + return (rows: readonly SlackFormHistoryRow[], companyId: string, endpointId: string, check?: () => void): readonly string[] => { + const scoped: SlackFormHistoryRow[] = [], byId = new Map(); + for (const row of rows) { + check?.(); + if (row.companyId !== companyId || row.endpointId !== endpointId) continue; + const prior = byId.get(row.id); + if (prior && !isDeepStrictEqual(prior, row)) return []; + if (!prior) { scoped.push(row); byId.set(row.id, row); } + } + const completed = new Set(); + for (const row of scoped) { + check?.(); + if (!["slack_form_owner", "slack_form_grant"].includes(row.kind) || row.result?.nextAttemptAt !== null) continue; + try { + if (row.result?.phase === "transferred") { + if (slackFormRetirementHistoryReceipt(row)) completed.add(row.id); + continue; + } + const binding = parseSlackFormBinding(row.payload.binding); + const current = bindings.get(`${companyId}:${endpointId}`); + if (binding.companyId !== companyId || binding.endpointId !== endpointId || !coverage + || !endpoints.has(endpointId) || !connections.has(binding.connectionId) + || row.payload.environment !== coverage.expected.environment || !current || !sameSlackFormBinding(current, binding)) continue; + if (row.kind === "slack_form_owner" && row.result?.phase === "enrolled") { + // Owner uniqueness was certified separately. A new or changed owner + // between keyset passes remains unclassified for the fresh shared gate. + if (!coverage.currentOwners || isDeepStrictEqual(coverage.currentOwners.get(row.id), row)) completed.add(row.id); + } else if (row.kind === "slack_form_grant" && row.result?.phase === "published") completed.add(row.id); + } catch { continue; } + } + for (const session of scoped) { + check?.(); + if (session.kind !== "slack_form_session" || !["processed", "cancelled", "failed"].includes(session.status)) continue; + try { + const binding = parseSlackFormBinding(session.payload.binding), work = parseSlackFormAckWork(session.result?.formAck); + if (session.status === "failed" && work.receipt?.outcome !== "no_effect") continue; + if (binding.companyId !== companyId || binding.endpointId !== endpointId || work.phase !== "transferred" || !work.receiptJws || !work.receipt + || session.result?.nextAttemptAt !== null) continue; + assertSlackFormReceiptHistory(session, work.receipt); + completed.add(session.id); + for (const [kind, id] of [["question_form_open", session.payload.openActionRowId], ["question_form_submit", session.payload.submitActionRowId]] as const) { + const action = typeof id === "string" ? byId.get(id) : undefined; + if (action?.kind === kind && action.status === "processed" && action.conversationId === session.conversationId + && action.principalId === session.principalId && action.payload.interactionId === session.payload.interactionId + && action.payload.publicationId === session.payload.publicationId) completed.add(action.id); + } + } catch { continue; } + } + return [...completed].sort(); + }; +} + +export function completedSlackFormHistoryActionIds(rows: readonly SlackFormHistoryRow[], companyId: string, endpointId: string, coverage?: SlackFormHistoryCoverage): readonly string[] { + return createSlackFormHistoryClassifier(coverage)(rows, companyId, endpointId); +} + +export function completedSlackFormSessionReceipt(session: Pick) { + if (session.kind !== "slack_form_session" || !["processed", "cancelled", "failed"].includes(session.status)) return null; + const binding = parseSlackFormBinding(session.payload.binding), work = parseSlackFormAckWork(session.result?.formAck); + if (session.status === "failed" && work.receipt?.outcome !== "no_effect") return null; + if (binding.companyId !== session.companyId || binding.endpointId !== session.endpointId || work.phase !== "transferred" + || !work.receiptJws || !work.receipt || session.result?.nextAttemptAt !== null) return null; + assertSlackFormReceiptHistory(session, work.receipt); + return work.receipt; +} diff --git a/server/src/services/chat-channels/slack/form-normal-ack.ts b/server/src/services/chat-channels/slack/form-normal-ack.ts new file mode 100644 index 00000000000..8a3432b113e --- /dev/null +++ b/server/src/services/chat-channels/slack/form-normal-ack.ts @@ -0,0 +1,37 @@ +import type { Db } from "@paperclipai/db"; +import type { PaperclipCloudConnector } from "../../paperclip-cloud-connector.js"; +import { stageSlackFormNormalAck, acceptSlackFormAckReceipt } from "./form-ack-work.js"; +import { slackFormConnectorIdentity } from "./cloud-consumer.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; + +export async function acknowledgeSlackFormDeliveries(db: Db, input: { connector: PaperclipCloudConnector; + values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }; companies: ReadonlyMap }): Promise { + const formIds = input.values.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)); + const values = input.values; + if (!formIds.length) await input.connector.acknowledgeEvents(values); + else { + const identity = slackFormConnectorIdentity(input.connector), detailed = Reflect.get(input.connector, "acknowledgeEventsWithReceipts"); + if (!identity || typeof detailed !== "function") throw new SlackCloudReceiptError("form_ack_receipts_unsupported", "attention"); + for (const deliveryId of formIds) { + const companyId = input.companies.get(deliveryId); + if (!companyId) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + await db.transaction(tx => stageSlackFormNormalAck(tx, { companyId, instanceId: identity.instanceId, leaseId: values.leaseId, deliveryIds: [deliveryId] })); + } + const response: unknown = await Reflect.apply(detailed, input.connector, [values]); + if (!response || typeof response !== "object" || Array.isArray(response) || !Array.isArray(Reflect.get(response, "formReceipts"))) + throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const receipts: unknown = Reflect.get(response, "formReceipts"); + if (!Array.isArray(receipts) || receipts.length !== formIds.length) throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const seen = new Set(); + for (const value of receipts) { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + const deliveryId: unknown = Reflect.get(value, "deliveryId"), receiptJws: unknown = Reflect.get(value, "receiptJws"); + if (typeof deliveryId !== "string" || !formIds.includes(deliveryId) || seen.has(deliveryId) || typeof receiptJws !== "string") + throw new SlackCloudReceiptError("form_ack_receipts_invalid", "attention"); + seen.add(deliveryId); + const companyId = input.companies.get(deliveryId); + if (!companyId) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + await db.transaction(tx => acceptSlackFormAckReceipt(tx, { companyId, instanceId: identity.instanceId, leaseId: values.leaseId, receiptJws })); + } + } +} diff --git a/server/src/services/chat-channels/slack/form-open-callback.ts b/server/src/services/chat-channels/slack/form-open-callback.ts new file mode 100644 index 00000000000..173cedc4593 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-open-callback.ts @@ -0,0 +1,43 @@ +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; +import { and, eq } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { sameSlackFormBinding, type SlackFormOpenContext } from "./cloud-forms.js"; +import { stageSlackFormSession, confirmSlackFormSessionUpdate } from "./form-session.js"; + +type Tx = Parameters[0]>[0]; +type Scope = { companyId: string; endpointId: string; conversationId: string; principalId: string }; +export async function stageSlackFormOpenCallback(tx: Tx, input: Scope & { + context: SlackFormOpenContext; resolved: ResolvedChatQuestionFormOpen; lease?: CredentialMutationLeaseGuard; + attemptProviderActionId: string; attemptActionId: string | null; +}): Promise<{ kind: "ready"; attemptActionId: string; resolved: ResolvedChatQuestionFormOpen } | { kind: "duplicate" }> { + const { context, lease } = input; + if (!lease || !context.environment) throw new SlackCloudReceiptError("form_credential_lease_unavailable", "retry"); + const held = await stageSlackFormSession(tx, { operation: context.operation, currentBinding: context.currentBinding, + principalId: input.principalId, resolved: input.resolved, lease, environment: context.environment }); + context.sessionActionId = held.actionId; + if (held.kind !== "update") return { kind: "duplicate" }; + if (input.attemptActionId) return { kind: "ready", attemptActionId: input.attemptActionId, resolved: input.resolved }; + const [attempt] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, input.endpointId), eq(chatActions.conversationId, input.conversationId), + eq(chatActions.principalId, input.principalId), eq(chatActions.kind, "question_form_open_attempt"), + eq(chatActions.providerActionId, input.attemptProviderActionId))).for("update"); + if (!attempt || attempt.status !== "processing") throw new SlackCloudReceiptError("form_open_attempt_unresolved", "attention"); + return { kind: "ready", attemptActionId: attempt.id, resolved: input.resolved }; +} +export async function confirmSlackFormOpenCallback(tx: Tx, input: Omit & { context: SlackFormOpenContext }): Promise { + const { context } = input; + if (!context.sessionActionId || !context.updatedView || !context.operation.view) throw new SlackCloudReceiptError("form_update_receipt_unavailable", "retry"); + const updated = await confirmSlackFormSessionUpdate(tx, { companyId: input.companyId, endpointId: input.endpointId, + principalId: input.principalId, operationId: context.operation.id }, context.operation.view, context.updatedView.hash); + if (!updated) throw new SlackCloudReceiptError("form_update_settlement_changed", "retry"); +} + +export function isRetryableSlackFormUpdate(context: SlackFormOpenContext | undefined, error: unknown): boolean { + return Boolean(context?.sessionActionId && context.updateAttempted === true && !context.updatedView + && context.operation.phase === "confirmed" && context.operation.view && !context.operation.closed && !context.operation.rejection + && sameSlackFormBinding(context.operation.binding, context.currentBinding) + && classifyChatPublicationError(error, 1).kind === "retry"); +} diff --git a/server/src/services/chat-channels/slack/form-opening-proof.ts b/server/src/services/chat-channels/slack/form-opening-proof.ts new file mode 100644 index 00000000000..903fa7dd566 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-opening-proof.ts @@ -0,0 +1,63 @@ +import { createHash } from "node:crypto"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import type { StoredSecretVersionMaterial } from "../../../secrets/types.js"; +import { createLocalEncryptedCipher } from "../../../secrets/local-encrypted-provider.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, sameSlackFormBinding, type SlackFormOperation } from "./cloud-forms.js"; +type Tx = Pick; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return fail("form_opening_proof_invalid"); + return Object.fromEntries(Object.entries(value)); +} +function opening(value: unknown): { rawBody: string; timestamp: string; signature: string } { + const raw = object(value); + if (Object.keys(raw).sort().join() !== "rawBody,signature,timestamp" || typeof raw.rawBody !== "string" || raw.rawBody.length > 1398104 + || !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(raw.rawBody) + || typeof raw.timestamp !== "string" || !/^[0-9]{1,16}$/.test(raw.timestamp) || typeof raw.signature !== "string" || !/^v0=[a-f0-9]{64}$/.test(raw.signature)) + return fail("form_opening_proof_invalid"); + return { rawBody: raw.rawBody, timestamp: raw.timestamp, signature: raw.signature }; +} +const digest = (raw: ReturnType) => createHash("sha256").update(JSON.stringify([raw.rawBody, raw.timestamp, raw.signature])).digest("hex"); +export function readSlackFormOriginalProofs(row: Pick): { opening: ReturnType; terminal: ReturnType | null } { + const saved = object(row.result?.openingProof); + if (row.kind !== "slack_form_session" || saved.version !== 1 || Object.keys(saved).sort().join() !== "material,version") return fail("form_opening_proof_invalid"); + let value: Record; + try { value = object(JSON.parse(createLocalEncryptedCipher("read").decrypt(object(saved.material)))); } + catch { return fail("form_opening_cipher_unavailable"); } + const binding = parseSlackFormBinding(row.payload.binding); + if (value.version !== 1 || value.kind !== "slack_form_original_opening" || value.appReceiptId !== row.id || value.companyId !== row.companyId || value.endpointId !== row.endpointId + || value.operationId !== row.payload.operationId || value.environment !== row.payload.environment + || !sameSlackFormBinding(parseSlackFormBinding(value.binding), binding)) return fail("form_opening_cipher_binding_changed"); + const raw = opening(value.original); + if (digest(raw) !== row.payload.requestDigest) return fail("form_opening_cipher_binding_changed"); + if (value.terminal === undefined) return fail("form_terminal_original_proof_missing"); + return { opening: raw, terminal: value.terminal === null ? null : opening(value.terminal) }; +} +export async function retainSlackFormOpeningProof(tx: Tx, input: { companyId: string; appReceiptId: string; operation: SlackFormOperation; original: unknown }): Promise { + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, input.appReceiptId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, input.operation.binding.endpointId), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row || row.payload.operationId !== input.operation.id || row.payload.requestDigest !== input.operation.requestDigest + || !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), input.operation.binding) || !row.result?.cloudProofAcceptance) + return fail("form_opening_proof_authority_changed"); + const original = opening(input.original); + if (digest(original) !== input.operation.requestDigest) return fail("form_opening_proof_changed"); + const terminal = input.operation.closed ? opening({ rawBody: input.operation.closed.evidence.rawBody, + timestamp: input.operation.closed.evidence.timestamp, signature: input.operation.closed.evidence.signature }) : null; + if (terminal && digest(terminal) !== input.operation.closed?.requestDigest) return fail("form_terminal_original_proof_changed"); + if (row.result.openingProof !== undefined) { + if (!isDeepStrictEqual(readSlackFormOriginalProofs(row), { opening: original, terminal })) return fail("form_opening_proof_changed"); + return; + } + let material: StoredSecretVersionMaterial; + try { material = createLocalEncryptedCipher("write").encrypt(JSON.stringify({ version: 1, kind: "slack_form_original_opening", appReceiptId: row.id, companyId: row.companyId, + endpointId: row.endpointId, operationId: input.operation.id, environment: row.payload.environment, binding: input.operation.binding, original, terminal })); } + catch { return fail("form_opening_cipher_unavailable"); } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const changed = await tx.update(chatActions).set({ result: { ...row.result, openingProof: { version: 1, material } }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (changed.length !== 1) return fail("form_opening_proof_custody_changed"); +} diff --git a/server/src/services/chat-channels/slack/form-owner-rearm.test.ts b/server/src/services/chat-channels/slack/form-owner-rearm.test.ts new file mode 100644 index 00000000000..b010d4bc6cb --- /dev/null +++ b/server/src/services/chat-channels/slack/form-owner-rearm.test.ts @@ -0,0 +1,21 @@ +import { expect, it } from "vitest"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { slackFormAttentionCanRearm } from "./form-owner.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const { eventBindingId, credentialFingerprint, ...identity } = binding; +const enrollment = { ...identity, bindingId: eventBindingId }; +it("allows a recoverable cipher failure only under the exact current configured authority", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding, enrollment })).toBe(true); +}); +it("refuses foreign company and replaced instance during explicit configuration rearm", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, companyId: "foreign" }, enrollment })).toBe(false); + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, instanceId: "foreign" }, enrollment })).toBe(false); +}); +it("does not rewrite stale generations or known authority conflicts into retryable work", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { ...binding, runtimeGeneration: 2 }, enrollment })).toBe(false); + expect(slackFormAttentionCanRearm({ code: "form_owner_authority_unavailable", binding, enrollment })).toBe(false); +}); +it("unknown malformed attention stays blocked without aborting other eligible retries", () => { + expect(slackFormAttentionCanRearm({ code: "form_opening_cipher_unavailable", binding: { malformed: true }, enrollment })).toBe(false); +}); diff --git a/server/src/services/chat-channels/slack/form-owner.ts b/server/src/services/chat-channels/slack/form-owner.ts new file mode 100644 index 00000000000..0f747eeffe5 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-owner.ts @@ -0,0 +1,320 @@ +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; +import { slackFormEnrollmentActionId, slackFormEnrollmentCompatibleActionIds } from "./form-enrollment-identity.js"; +import { acceptSlackFormRetirementLocally, transferSlackFormRetirement } from "./form-retirement-work.js"; +import { reconcileSlackFormNormalAck } from "./form-ack-work.js"; +import { HttpError } from "../../../errors.js"; +import { createPaperclipCloudConnector, paperclipCloudConnectorConfigFromEnv, PaperclipCloudConnectorError } from "../../paperclip-cloud-connector.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatPublications, issueThreadInteractions, toolConnections, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { resolveChatQuestionFormOpen, type ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import type { IssueThreadInteraction } from "@paperclipai/shared"; +import { chatInteractionForm } from "../../chat-interaction-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { slackFormSetupRecord, parseSlackFormBinding, parseSlackFormPublicEnrollment, projectSlackIssuedFormGrant, + type SlackFormPublicEnrollment, type SlackFormBinding } from "./cloud-forms.js"; +import { slackFormConnectorIdentity, type SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +type Tx = Pick; +export type SlackFormProtocol = (input: { subject: string; companyId: string; operation: Record }) => Promise; +function attention(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) return attention("form_owner_response_invalid"); + return value as Record; +} +function matches(binding: SlackFormBinding, enrollment: SlackFormPublicEnrollment) { + const { credentialFingerprint: _, eventBindingId, ...identity } = binding; + return eventBindingId === enrollment.bindingId && isDeepStrictEqual({ ...identity, bindingId: eventBindingId }, enrollment); +} +export function projectSlackFormPublicEnrollment(endpoint: typeof chatEndpoints.$inferSelect): SlackFormPublicEnrollment | null { + const setup = slackFormSetupRecord(endpoint.setup), descriptor = setup.slackCloudEvents; + if (!descriptor || typeof descriptor !== "object" || Array.isArray(descriptor) + || typeof setup.slackCloudInstanceId !== "string") return null; + return parseSlackFormPublicEnrollment({ version: 1, instanceId: setup.slackCloudInstanceId, companyId: endpoint.companyId, + endpointId: endpoint.id, publicId: endpoint.publicId, connectionId: endpoint.connectionId, + runtimeGeneration: Reflect.get(descriptor, "runtimeGeneration"), bindingId: Reflect.get(descriptor, "bindingId"), + teamId: Reflect.get(descriptor, "teamId"), appId: Reflect.get(descriptor, "appId") }); +} +export async function stageSlackFormEnrollment(tx: Tx, input: { enrollment: SlackFormPublicEnrollment; subject: string; environment: string }): Promise { + const enrollment = parseSlackFormPublicEnrollment(input.enrollment); + if (!input.subject || !["development", "test", "staging", "production"].includes(input.environment)) return attention("form_owner_subject_invalid"); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, enrollment.endpointId), eq(chatEndpoints.companyId, enrollment.companyId))).for("update"); + const setup: Record | undefined = endpoint ? slackFormSetupRecord(endpoint.setup) : undefined; + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status) + || endpoint.publicId !== enrollment.publicId || endpoint.connectionId !== enrollment.connectionId + || setup?.runtimeGeneration !== enrollment.runtimeGeneration || setup?.slackCloudInstanceId !== enrollment.instanceId + || setup?.slackCloudSubject !== input.subject || !isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment)) return attention("form_owner_current_enrollment_changed"); + const providerActionId = slackFormEnrollmentActionId(enrollment); + const existingRows = await tx.select().from(chatActions).where(and(eq(chatActions.endpointId, endpoint.id), eq(chatActions.companyId, endpoint.companyId), + inArray(chatActions.providerActionId, [...slackFormEnrollmentCompatibleActionIds(enrollment)]))).for("update"); + if (existingRows.length > 1) return attention("form_owner_identity_ambiguous"); + const [existing] = existingRows; + const payload = { enrollment, subject: input.subject, environment: input.environment }; + if (existing) { + if (existing.kind !== "slack_form_owner" || !Object.keys(payload).every(key => isDeepStrictEqual(existing.payload[key], Reflect.get(payload, key)))) return attention("form_owner_custody_changed"); + return existing.id; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: endpoint.companyId, endpointId: endpoint.id, kind: "slack_form_owner", providerActionId, + payload, status: "issued", result: { phase: "staged", nextAttemptAt: new Date().toISOString() } }).returning({ id: chatActions.id }); + return row.id; +} +export async function reconcileSlackFormEnrollment(db: Db, input: { actionId: string; identity: SlackFormConnectorIdentity; + protocol: SlackFormProtocol; resolveBotToken(endpointId: string): Promise; + onSnapshot?(snapshot: typeof chatActions.$inferSelect): void }): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.kind, "slack_form_owner"))).limit(1); + if (!row || row.result?.phase === "transferred") return; + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), subject = row.payload.subject; + if (enrollment.companyId !== row.companyId || enrollment.endpointId !== row.endpointId || enrollment.instanceId !== input.identity.instanceId + || row.payload.environment !== input.identity.environment || typeof subject !== "string" || !subject) return attention("form_owner_identity_changed"); + if (row.result?.phase === "retirement_accepted" || row.result?.phase === "retirement_gc_pending") return transferSlackFormRetirement(db, row, input.identity, input.protocol, input.onSnapshot); + const [endpoint] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, row.endpointId), eq(chatEndpoints.companyId, row.companyId))).limit(1); + const endpointSetup: Record | undefined = endpoint ? slackFormSetupRecord(endpoint.setup) : undefined; + const live = row.result?.desiredActive !== false && endpoint && ["active", "verifying", "attention"].includes(endpoint.status) && endpointSetup?.runtimeGeneration === enrollment.runtimeGeneration + && endpointSetup?.slackCloudInstanceId === enrollment.instanceId && endpointSetup?.slackCloudSubject === subject && isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment); + let binding: SlackFormBinding; + if (row.payload.binding !== undefined) binding = parseSlackFormBinding(row.payload.binding); + else { + const response = object(await input.protocol({ subject, companyId: row.companyId, operation: live + ? { kind: "enroll", enrollment, botToken: await input.resolveBotToken(row.endpointId) } + : { kind: "owner-receipt", enrollment } })); + if (!live && response.kind === "retired-enrollment") { + if (typeof response.retirementReceiptJws !== "string") throw new SlackCloudReceiptError("form_retirement_receipt_missing", "retry"); + await acceptSlackFormRetirementLocally(db, row, input.identity, response.retirementReceiptJws); + return; + } + binding = parseSlackFormBinding(response.binding); + } + if (!matches(binding, enrollment)) return attention("form_owner_binding_changed"); + if (row.payload.binding === undefined) { + await db.transaction(async tx => { + const [current] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, row.endpointId), eq(chatEndpoints.companyId, row.companyId))).for("update"); + const [intent] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!intent || !isDeepStrictEqual(intent.payload, row.payload) || !isDeepStrictEqual(intent.result, row.result)) throw new SlackCloudReceiptError("form_owner_intent_changed", "retry"); + const currentSetup: Record | undefined = current ? slackFormSetupRecord(current.setup) : undefined; + const currentLive = intent.result?.desiredActive !== false && current && ["active", "verifying", "attention"].includes(current.status) && currentSetup?.runtimeGeneration === enrollment.runtimeGeneration + && currentSetup?.slackCloudInstanceId === enrollment.instanceId && currentSetup?.slackCloudSubject === subject && isDeepStrictEqual(projectSlackFormPublicEnrollment(current), enrollment); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ payload: { ...intent.payload, binding }, result: { phase: currentLive ? "enrolled" : "retired_cleanup", nextAttemptAt: currentLive ? null : new Date().toISOString() }, updatedAt: new Date() }).where(eq(chatActions.id, intent.id)); + if (currentLive) { + await tx.update(chatEndpoints).set({ setup: sql`jsonb_set(${chatEndpoints.setup}, '{slackCloudForms}', ${JSON.stringify(binding)}::jsonb)` }).where(and(eq(chatEndpoints.id, current.id), eq(chatEndpoints.companyId, current.companyId))); + await notifyChatPublicationWork(tx); + } + }); + return; + } + if (!live) { + await input.protocol({ subject, companyId: row.companyId, operation: { kind: "retire", binding } }); + const response = object(await input.protocol({ subject, companyId: row.companyId, operation: { kind: "cleanup", binding } })); + if (response.cleaned !== true) throw new SlackCloudReceiptError("form_owner_retained_resource", "retry"); + if (response.instanceId !== input.identity.instanceId || response.environment !== input.identity.environment + || !isDeepStrictEqual(response.binding, binding)) return attention("form_owner_retirement_receipt_changed"); + if (typeof response.retirementReceiptJws !== "string") throw new SlackCloudReceiptError("form_retirement_receipt_missing", "retry"); + await acceptSlackFormRetirementLocally(db, row, input.identity, response.retirementReceiptJws); + } +} +export async function stageSlackFormGrant(tx: Tx, input: { binding: SlackFormBinding; environment: string; subject: string; resolved: ResolvedChatQuestionFormOpen }) { + const grant = await projectSlackIssuedFormGrant(tx, input.binding, input.resolved); + if (!grant) return attention("form_grant_issued_authority_changed"); + const [owner] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.binding.companyId), eq(chatActions.endpointId, input.binding.endpointId), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(input.binding)}::jsonb`, sql`${chatActions.result}->>'phase' = 'enrolled'`)).for("update"); + if (!owner || owner.payload.environment !== input.environment || owner.payload.subject !== input.subject) return attention("form_grant_owner_changed"); + const providerActionId = `slack-form-grant:${grant.grantId}`; + const [existing] = await tx.select().from(chatActions).where(and(eq(chatActions.endpointId, input.binding.endpointId), eq(chatActions.companyId, input.binding.companyId), eq(chatActions.providerActionId, providerActionId))).for("update"); + if (existing) { if (!isDeepStrictEqual(existing.payload.grant, grant)) return attention("form_grant_custody_changed"); return existing.id; } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: input.binding.companyId, endpointId: input.binding.endpointId, conversationId: grant.conversationId, + kind: "slack_form_grant", providerActionId, payload: { grant, binding: input.binding, subject: input.subject, environment: input.environment }, + status: "issued", result: { phase: "staged", nextAttemptAt: new Date().toISOString() } }).returning({ id: chatActions.id }); + return row.id; +} +export async function publishSlackFormGrant(db: Db, actionId: string, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol) { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.id, actionId), eq(chatActions.kind, "slack_form_grant"))).limit(1); + if (!row || ["published", "transferred"].includes(String(row.result?.phase))) return; + const binding = parseSlackFormBinding(row.payload.binding); + if (binding.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || binding.companyId !== row.companyId + || binding.endpointId !== row.endpointId || typeof row.payload.subject !== "string") return attention("form_grant_identity_changed"); + const response = object(await protocol({ subject: row.payload.subject, companyId: row.companyId, operation: { kind: "grant", grant: row.payload.grant } })); + if (!isDeepStrictEqual(response.grant, row.payload.grant)) return attention("form_grant_receipt_changed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || !isDeepStrictEqual(current.payload, row.payload)) throw new SlackCloudReceiptError("form_grant_intent_changed", "retry"); + if (current.result?.phase === "transferred") return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "processed", result: { phase: "published", nextAttemptAt: null }, updatedAt: new Date() }).where(eq(chatActions.id, row.id)); + }); +} + +export async function stageSlackFormOwnerConfiguration(tx: Tx, endpointId: string, active: boolean, subject: string, identity: SlackFormConnectorIdentity) { + const [endpoint] = await tx.select().from(chatEndpoints).where(eq(chatEndpoints.id, endpointId)).for("update"); + if (!endpoint || endpoint.provider !== "slack") return; + const setup = slackFormSetupRecord(endpoint.setup); + if (active) { + const enrollment = projectSlackFormPublicEnrollment(endpoint); + if (!enrollment || enrollment.instanceId !== identity.instanceId || setup.slackCloudSubject !== subject) return attention("form_owner_event_enrollment_unsettled"); + await stageSlackFormEnrollment(tx, { enrollment, subject, environment: identity.environment }); + await rearmSlackFormAttention(tx, { enrollment, subject, environment: identity.environment }); + const previous = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"))).for("update"); + for (const row of previous) { + const prior = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (prior.companyId !== endpoint.companyId || prior.endpointId !== endpoint.id || prior.instanceId !== identity.instanceId + || row.payload.environment !== identity.environment || typeof row.payload.subject !== "string" || !row.payload.subject + || prior.runtimeGeneration >= enrollment.runtimeGeneration || ["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase))) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "issued", result: { ...row.result, desiredActive: false, + phase: row.payload.binding ? "retired_cleanup" : "staged", nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)); + } + return; + } + const [connection] = await tx.select().from(toolConnections).where(and(eq(toolConnections.id, endpoint.connectionId), eq(toolConnections.companyId, endpoint.companyId))).limit(1); + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"))).for("update"); + for (const row of rows) { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment); + if (enrollment.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || typeof row.payload.subject !== "string" + || enrollment.runtimeGeneration > Number(setup.runtimeGeneration) || ["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase))) continue; + if (["active", "verifying", "attention"].includes(endpoint.status) && connection?.status === "active" && connection.enabled + && enrollment.runtimeGeneration === setup.runtimeGeneration && setup.slackCloudRetiredGeneration !== enrollment.runtimeGeneration + && isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment)) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "issued", result: { ...row.result, desiredActive: false, + phase: row.payload.binding ? "retired_cleanup" : "staged", nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))); + } +} +export async function reconcilePendingSlackFormWork(db: Db, input: { identity: SlackFormConnectorIdentity | null; protocol: SlackFormProtocol; + resolveBotToken(endpointId: string): Promise; connector?(): unknown; report(code: string, actionId: string): void }, now = new Date()): Promise { + if (!input.identity) return null; + const eligible = and(sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`, + sql`${chatActions.payload}->>'environment' = ${input.identity.environment}`, + sql`coalesce(${chatActions.payload}->'binding'->>'instanceId', ${chatActions.payload}->'enrollment'->>'instanceId') = ${input.identity.instanceId}`, + sql`${chatActions.result}->>'nextAttemptAt' is not null`); + const [row] = await db.select().from(chatActions).where(eligible).orderBy(sql`${chatActions.result}->>'nextAttemptAt'`, chatActions.id).limit(1); + if (!row) return null; + const due = typeof row.result?.nextAttemptAt === "string" ? Date.parse(row.result.nextAttemptAt) : Number.NaN; + if (Number.isFinite(due) && due > now.getTime()) return due; + let failureSnapshot = row; + const onSnapshot = (snapshot: typeof chatActions.$inferSelect) => { failureSnapshot = snapshot; }; + try { + if (!Number.isFinite(due)) attention("form_owner_deadline_invalid"); + if (row.kind === "slack_form_owner") await reconcileSlackFormEnrollment(db, { ...input, identity: input.identity, actionId: row.id, onSnapshot }); + else if (row.kind === "slack_form_grant") await publishSlackFormGrant(db, row.id, input.identity, input.protocol); + else await reconcileSlackFormNormalAck(db, { companyId: row.companyId, actionId: row.id, instanceId: input.identity.instanceId, connector: input.connector?.(), protocol: input.protocol, onSnapshot }); + } catch (error) { + const credentialAttention = error instanceof HttpError && [400, 401, 403, 404, 422].includes(error.status); + const permanent = credentialAttention || error instanceof SlackCloudReceiptError && error.disposition !== "retry" + || error instanceof PaperclipCloudConnectorError && ([400, 401, 403, 404].includes(error.status ?? 0) + || error.status === 409 && ["SLACK_FORM_CUSTODY_MISSING", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_RETIREMENT_PROOF_MISSING", "SLACK_FORM_CUSTODY_CHANGED", "SLACK_FORM_CANONICAL_FACT_CHANGED", "SLACK_FORM_CANONICAL_FACT_BUDGET_CHANGED"].includes(error.brokerReason ?? "")); + const code = credentialAttention ? "form_bot_credential_unavailable" : error instanceof SlackCloudReceiptError ? error.code : permanent ? "form_owner_authority_unavailable" : "form_owner_broker_unavailable"; + try { + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId))).for("update"); + if (!current || !isDeepStrictEqual(current, failureSnapshot)) return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const attempts = typeof current.result?.attempts === "number" ? Math.min(current.result.attempts + 1, 1000000) : 1; + await tx.update(chatActions).set({ status: permanent ? "attention" : current.status, result: { ...current.result, attempts, failureCode: code, + failureReason: error instanceof PaperclipCloudConnectorError ? error.brokerReason ?? null : null, + nextAttemptAt: permanent ? null : new Date(Date.now() + Math.min(300000, 5000 * 2 ** Math.min(attempts - 1, 6))).toISOString() }, updatedAt: new Date() }).where(eq(chatActions.id, current.id)); + }); + } catch (writeError) { throw new AggregateError([error, writeError], "Slack form owner failure and durable retry failure"); } + input.report(code, row.id); + } + const [next] = await db.select({ result: chatActions.result }).from(chatActions).where(eligible).orderBy(sql`${chatActions.result}->>'nextAttemptAt'`, chatActions.id).limit(1); + const nextDue = typeof next?.result?.nextAttemptAt === "string" ? Date.parse(next.result.nextAttemptAt) : Number.NaN; + return Number.isFinite(nextDue) ? nextDue : null; +} + +export async function stageSlackFormPublication(tx: Tx, publicationId: string, interaction: IssueThreadInteraction): Promise { + const [currentInteraction] = await tx.select().from(issueThreadInteractions).where(and(eq(issueThreadInteractions.id, interaction.id), + eq(issueThreadInteractions.companyId, interaction.companyId), eq(issueThreadInteractions.issueId, interaction.issueId), eq(issueThreadInteractions.status, "pending"))).limit(1); + if (!currentInteraction || !isDeepStrictEqual(currentInteraction.payload, interaction.payload)) return null; + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, publicationId), eq(chatPublications.companyId, interaction.companyId), eq(chatPublications.issueId, interaction.issueId))).limit(1); + if (!publication || publication.state !== "published" || !publication.providerMessageId) return null; + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, publication.endpointId), eq(chatEndpoints.companyId, publication.companyId))).limit(1); + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status)) return null; + const rawBinding = slackFormSetupRecord(endpoint.setup).slackCloudForms; + if (!rawBinding) return null; + const binding = parseSlackFormBinding(rawBinding), form = chatInteractionForm(interaction, "slack"); + if (!form || form.status !== "pending") return null; + const [owner] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(binding)}::jsonb`, sql`${chatActions.result}->>'phase' = 'enrolled'`)).limit(1); + if (!owner || typeof owner.payload.environment !== "string" || typeof owner.payload.subject !== "string") return attention("form_grant_owner_missing"); + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.conversationId, publication.conversationId), eq(chatActions.kind, "question_form_open"), eq(chatActions.status, "issued"), + sql`${chatActions.payload}->>'publicationId' = ${publication.id}`, sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`)).limit(1); + if (!open) return null; + const resolved = await resolveChatQuestionFormOpen(tx, { companyId: endpoint.companyId, endpointId: endpoint.id, + conversationId: publication.conversationId, interaction: form, openActionId: open.providerActionId }); + if (!resolved) return null; + return stageSlackFormGrant(tx, { binding, environment: owner.payload.environment, subject: owner.payload.subject, resolved }); +} + + +export function createSlackFormConnectorAccess() { + let cached: { config: NonNullable>; connector: ReturnType } | undefined; + const connector = () => { + let config: ReturnType; + try { config = paperclipCloudConnectorConfigFromEnv(); } catch { return null; } + if (!config) { cached = undefined; return null; } + if (!cached || !isDeepStrictEqual(cached.config, config)) cached = { config, connector: createPaperclipCloudConnector({ config }) }; + return cached.connector; + }; + const protocol: SlackFormProtocol = async input => { + const current = connector(), identity = slackFormConnectorIdentity(current); + if (!current || !identity) return attention("form_connector_unavailable"); + const operation = Reflect.get(current, "slackFormOperation"); + if (typeof operation !== "function") return attention("form_connector_unsupported"); + return Reflect.apply(operation, current, [input]); + }; + return { connector, protocol, identity: () => slackFormConnectorIdentity(connector()) }; +} + +const formWorkOwners = new WeakMap Promise>(); +export function registerSlackFormWorkOwner(db: Db, owner: (now: Date) => Promise) { + formWorkOwners.set(db, owner); + return () => { if (formWorkOwners.get(db) === owner) formWorkOwners.delete(db); }; +} +export function reconcileRegisteredSlackFormWork(db: Db, now: Date): Promise { + return formWorkOwners.get(db)?.(now) ?? Promise.resolve(null); +} + +export function slackFormAttentionCanRearm(input: { code: unknown; binding: unknown; enrollment: unknown }): boolean { + const recoverable = ["form_bot_credential_unavailable", "form_opening_cipher_unavailable", "form_submission_cipher_unavailable", + "form_connector_unsupported", "form_ack_receipts_unsupported", "form_ack_acceptance_missing", "form_owner_broker_unavailable"]; + if (typeof input.code !== "string" || !recoverable.includes(input.code)) return false; + try { + const binding = parseSlackFormBinding(input.binding), enrollment = parseSlackFormPublicEnrollment(input.enrollment); + return matches(binding, enrollment); + } catch { return false; } +} +export async function rearmSlackFormAttention(tx: Tx, input: { enrollment: SlackFormPublicEnrollment; subject: string; environment: string }): Promise { + const enrollment = parseSlackFormPublicEnrollment(input.enrollment); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, enrollment.endpointId), eq(chatEndpoints.companyId, enrollment.companyId))).for("update"); + if (!endpoint || endpoint.provider !== "slack" || !isDeepStrictEqual(projectSlackFormPublicEnrollment(endpoint), enrollment) + || slackFormSetupRecord(endpoint.setup).slackCloudSubject !== input.subject) return 0; + const rows = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, enrollment.companyId), eq(chatActions.endpointId, enrollment.endpointId), + eq(chatActions.status, "attention"), sql`${chatActions.kind} in ('slack_form_owner','slack_form_grant','slack_form_session')`)).for("update"); + let count = 0; + for (const row of rows) { + if (row.payload.environment !== input.environment || row.result?.nextAttemptAt !== null) continue; + const binding = row.kind === "slack_form_owner" ? row.payload.binding : row.kind === "slack_form_grant" + ? row.payload.grant && typeof row.payload.grant === "object" ? Reflect.get(row.payload.grant, "binding") : undefined : row.payload.binding; + if (row.payload.subject !== undefined && row.payload.subject !== input.subject) continue; + if (row.kind === "slack_form_owner" && binding === undefined) { + if (row.result?.failureCode !== "form_bot_credential_unavailable") continue; + try { if (!isDeepStrictEqual(parseSlackFormPublicEnrollment(row.payload.enrollment), enrollment)) continue; } catch { continue; } + } else if (!slackFormAttentionCanRearm({ code: row.result?.failureCode, binding, enrollment })) continue; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const phase = row.result?.phase; + const status = row.kind === "slack_form_session" && phase === "submitted" ? "processed" : row.kind === "slack_form_session" && phase === "not_opened" ? "failed" : row.kind === "slack_form_session" && phase === "closed" ? "cancelled" : "issued"; + const updated = await tx.update(chatActions).set({ status, result: { ...row.result, firstFailureCode: row.result?.firstFailureCode ?? row.result?.failureCode, + nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, enrollment.companyId), + eq(chatActions.endpointId, enrollment.endpointId), sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + count += updated.length; + } + return count; +} diff --git a/server/src/services/chat-channels/slack/form-publication.ts b/server/src/services/chat-channels/slack/form-publication.ts new file mode 100644 index 00000000000..d98fc46158a --- /dev/null +++ b/server/src/services/chat-channels/slack/form-publication.ts @@ -0,0 +1,82 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatPublications, toolConnections, type Db } from "@paperclipai/db"; +import type { ChatPublicationPreparationWait } from "../publications/preparation.js"; +import { chatInteractionForm } from "../../chat-interaction-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { parseSlackFormBinding, slackFormSetupRecord } from "./cloud-forms.js"; +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; +import type { EndpointRow } from "../runtime-types.js"; +import { projectSlackFormPublicEnrollment, stageSlackFormPublication } from "./form-owner.js"; + +type Tx = Parameters[0]>[0]; +function object(value: unknown): Record | null { + return value && typeof value === "object" && !Array.isArray(value) ? slackFormSetupRecord(value) : null; +} +function managedFormsExpected(setup: Record): boolean { + return setup.slackCloudIntent !== undefined || setup.slackCloudForms !== undefined + || setup.slackCloudEvents !== undefined || typeof setup.slackCloudInstanceId === "string"; +} +function scheduled(value: unknown): boolean { + return typeof value === "string" && Number.isFinite(Date.parse(value)); +} +function unavailable(): never { + // Inconsistent, stopped and attention custody retains the ordinary failure + // policy and authorized Retry path; only an actual scheduled owner can wait. + throw new SlackCloudReceiptError("form_publication_enrollment_pending", "retry"); +} +const enrollmentWait: ChatPublicationPreparationWait = { + kind: "wait", reason: "form_publication_enrollment_pending", retryAfterMs: 2_000, +}; +export function createSlackFormPublicationStage(db: Db) { + return async (endpoint: EndpointRow, publication: typeof chatPublications.$inferSelect): Promise<((tx: Tx) => Promise) | ChatPublicationPreparationWait | null> => { + if (endpoint.provider !== "slack" || !managedFormsExpected(slackFormSetupRecord(endpoint.setup)) || !publication.issueId + || typeof publication.payload.interactionId !== "string" || publication.companyId !== endpoint.companyId || publication.endpointId !== endpoint.id) return null; + const interaction = (await issueThreadInteractionService(db).listForIssue(publication.issueId)) + .find(value => value.id === publication.payload.interactionId && value.companyId === endpoint.companyId && value.status === "pending"); + if (!interaction || !chatInteractionForm(interaction, "slack")) return null; + const [open] = await db.select({ id: chatActions.id }).from(chatActions).where(and( + eq(chatActions.companyId, endpoint.companyId), eq(chatActions.endpointId, endpoint.id), + eq(chatActions.conversationId, publication.conversationId), eq(chatActions.kind, "question_form_open"), eq(chatActions.status, "issued"), + sql`${chatActions.payload}->>'publicationId' = ${publication.id}`, sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`)).limit(1); + if (!open) return null; + const [current] = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, endpoint.id), eq(chatEndpoints.companyId, endpoint.companyId))).limit(1); + if (!current || current.provider !== "slack" || !["active", "verifying", "attention"].includes(current.status) + || current.connectionId !== endpoint.connectionId || current.publicId !== endpoint.publicId) return unavailable(); + const [connection] = await db.select().from(toolConnections).where(and(eq(toolConnections.id, current.connectionId), eq(toolConnections.companyId, current.companyId))).limit(1); + if (!connection || connection.status !== "active" || !connection.enabled) return unavailable(); + const setup = slackFormSetupRecord(current.setup), generation = setup.runtimeGeneration; + if (generation !== slackFormSetupRecord(endpoint.setup).runtimeGeneration || !Number.isSafeInteger(generation) || Number(generation) < 1 + || Number.isSafeInteger(setup.slackCloudRetiredGeneration) && Number(setup.slackCloudRetiredGeneration) >= Number(generation)) return unavailable(); + const binding = setup.slackCloudForms === undefined ? undefined : parseSlackFormBinding(setup.slackCloudForms); + const mayWait = current.status !== "attention"; + const intent = object(setup.slackCloudIntent); + if (intent) { + // This is the initial events-enrollment responsibility, before onCommitted + // can atomically stage the forms owner. A null deadline means attention. + if (!mayWait || intent.version !== 1 || typeof intent.id !== "string" || !intent.id || intent.active !== true + || intent.companyId !== current.companyId || intent.endpointId !== current.id || intent.publicId !== current.publicId + || intent.connectionId !== current.connectionId || intent.runtimeGeneration !== generation || !/^[A-Za-z0-9_-]{43}$/.test(current.publicId) + || typeof intent.instanceId !== "string" || !intent.instanceId || typeof intent.subject !== "string" || !intent.subject + || typeof intent.teamId !== "string" || !/^T[A-Z0-9]+$/.test(intent.teamId) || intent.teamId !== current.providerAccountId + || typeof intent.appId !== "string" || !/^A[A-Z0-9]+$/.test(intent.appId) + || !scheduled(intent.nextAttemptAt)) return unavailable(); + return enrollmentWait; + } + const enrollment = projectSlackFormPublicEnrollment(current); + if (!enrollment || enrollment.runtimeGeneration !== generation || enrollment.instanceId !== setup.slackCloudInstanceId + || typeof setup.slackCloudSubject !== "string" || !setup.slackCloudSubject) return unavailable(); + const owners = await db.select().from(chatActions).where(and(eq(chatActions.companyId, current.companyId), eq(chatActions.endpointId, current.id), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.payload}->'enrollment' = ${JSON.stringify(enrollment)}::jsonb`)).limit(2); + if (owners.length !== 1) return unavailable(); + const [owner] = owners; + if (!owner || owner.status !== "issued" || owner.result?.desiredActive === false || owner.payload.subject !== setup.slackCloudSubject + || typeof owner.payload.environment !== "string" || !["development", "test", "staging", "production"].includes(owner.payload.environment)) return unavailable(); + if (mayWait && owner.result?.phase === "staged" && owner.payload.binding === undefined && scheduled(owner.result.nextAttemptAt)) return enrollmentWait; + if (owner.result?.phase !== "enrolled" || !binding || !isDeepStrictEqual(owner.payload.binding, binding) + || !isDeepStrictEqual({ version: binding.version, instanceId: binding.instanceId, companyId: binding.companyId, endpointId: binding.endpointId, + publicId: binding.publicId, connectionId: binding.connectionId, runtimeGeneration: binding.runtimeGeneration, + teamId: binding.teamId, appId: binding.appId, bindingId: binding.eventBindingId }, enrollment)) return unavailable(); + return async tx => { await stageSlackFormPublication(tx, publication.id, interaction); }; + }; +} diff --git a/server/src/services/chat-channels/slack/form-retirement-work.ts b/server/src/services/chat-channels/slack/form-retirement-work.ts new file mode 100644 index 00000000000..a5e4758d12c --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retirement-work.ts @@ -0,0 +1,120 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { replaceSlackFormReceiptSignature } from "./form-signature-refresh.js"; +import { parseSlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { verifySlackFormRetirementReceipt } from "./receipt-authority.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { parseSlackFormPublicEnrollment, parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; +import type { SlackFormProtocol } from "./form-owner.js"; +import type { SlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +type Row = typeof chatActions.$inferSelect; +function retry(code: string): never { throw new SlackCloudReceiptError(code, "retry"); } +export function assertSlackFormRetirementWorkAuthority(row: Row, identity: SlackFormConnectorIdentity, receipt: SlackFormRetirementReceipt): void { + const enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), binding = receipt.binding; + if (row.kind !== "slack_form_owner" || row.companyId !== binding.companyId || row.endpointId !== binding.endpointId + || row.payload.subject !== receipt.subject || row.payload.environment !== receipt.environment || receipt.environment !== identity.environment + || binding.instanceId !== identity.instanceId || enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(binding, key) !== Reflect.get(enrollment, key))) + throw new SlackCloudReceiptError("form_retirement_authority_changed", "attention"); + if (row.payload.binding !== undefined && !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), binding)) + throw new SlackCloudReceiptError("form_retirement_fingerprint_changed", "attention"); + if (row.result?.retirementReceipt !== undefined && !isDeepStrictEqual(row.result.retirementReceipt, receipt)) + throw new SlackCloudReceiptError("form_retirement_receipt_changed", "attention"); + if (!isSlackFormEnrollmentActionId(row.providerActionId, enrollment)) throw new SlackCloudReceiptError("form_retirement_owner_changed", "attention"); +} +export function verifyRetainedSlackFormRetirement(compactJws: string) { + try { return verifySlackFormRetirementReceipt({ compactJws }); } + catch (error) { + if (error instanceof Error && ["PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is not configured", "PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS is invalid", + "Cloud runtime identity uses an unknown signing key"].includes(error.message)) return retry("form_retirement_verifier_unavailable"); + throw new SlackCloudReceiptError("form_retirement_receipt_invalid", "attention"); + } +} +export async function acceptSlackFormRetirementLocally(db: Db, row: Row, identity: SlackFormConnectorIdentity, compactJws: string): Promise { + const receipt = verifyRetainedSlackFormRetirement(compactJws); assertSlackFormRetirementWorkAuthority(row, identity, receipt); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_acceptance_changed"); + if (["transferred", "retirement_accepted", "retirement_gc_pending"].includes(String(current.result?.phase))) return; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...current.result, phase: "retirement_accepted", + retirementReceiptJws: compactJws, retirementReceipt: receipt, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(current.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(current.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return retry("form_retirement_acceptance_changed"); + }); +} +export async function transferSlackFormRetirement(db: Db, snapshot: Row, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol, + onSnapshot?: (snapshot: Row) => void): Promise { + let row = snapshot; + if (row.result?.phase === "retirement_gc_pending") return releaseSlackRetirementFact(db, row, identity, protocol); + if (row.result?.phase !== "retirement_accepted" || typeof row.result.retirementReceiptJws !== "string") return retry("form_retirement_acceptance_missing"); + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt); assertSlackFormRetirementWorkAuthority(row, identity, receipt); + const refreshed = await protocol({ subject: receipt.subject, companyId: row.companyId, operation: { kind: "cleanup", binding: receipt.binding, + refreshRetirementReceipt: { receiptId: receipt.receiptId, ownerId: receipt.ownerId } } }); + if (!refreshed || typeof refreshed !== "object" || Array.isArray(refreshed) || Object.keys(refreshed).join() !== "retirementReceiptJws" + || typeof Reflect.get(refreshed, "retirementReceiptJws") !== "string") return retry("form_signature_refresh_unconfirmed"); + const compactJws = Reflect.get(refreshed, "retirementReceiptJws"); + if (typeof compactJws !== "string" || !isDeepStrictEqual(verifyRetainedSlackFormRetirement(compactJws), receipt)) + throw new SlackCloudReceiptError("form_signature_canonical_changed", "attention"); + row = await replaceSlackFormReceiptSignature(db, row, "retirementReceiptJws", compactJws); + onSnapshot?.(row); + const response = await protocol({ subject: receipt.subject, companyId: row.companyId, + operation: { kind: "cleanup", binding: receipt.binding, acceptedRetirementReceipt: compactJws } }); + if (!response || typeof response !== "object" || Array.isArray(response) || Object.keys(response).join() !== "acceptedRetirementDigest" + || Reflect.get(response, "acceptedRetirementDigest") !== receipt.receiptId) return retry("form_retirement_transfer_unconfirmed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_transfer_changed"); + const result = current.result ?? {}; + const retirement = { version: 1, binding: receipt.binding, instanceId: identity.instanceId, environment: identity.environment, + subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...result, phase: "retirement_gc_pending", nextAttemptAt: new Date().toISOString(), retirement }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(current.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(current.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) return retry("form_retirement_transfer_changed"); + const grants = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), + eq(chatActions.kind, "slack_form_grant"), sql`${chatActions.payload}->'binding' = ${JSON.stringify(receipt.binding)}::jsonb`)).for("update"); + for (const grant of grants) { + if (grant.payload.environment !== identity.environment || grant.payload.subject !== receipt.subject) + throw new SlackCloudReceiptError("form_grant_retirement_identity_changed", "attention"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const cancelled = await tx.update(chatActions).set({ status: "cancelled", result: { ...grant.result, phase: "transferred", nextAttemptAt: null, retirement }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, grant.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, grant.status), + sql`${chatActions.payload} = ${JSON.stringify(grant.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(grant.result)}::jsonb`)).returning({ id: chatActions.id }); + if (cancelled.length !== 1) return retry("form_grant_retirement_changed"); + } + }); +} + +export function assertSlackRetirementGCWork(row: Row, identity: SlackFormConnectorIdentity): SlackFormRetirementReceipt { + if (row.result?.phase !== "retirement_gc_pending") throw new SlackCloudReceiptError("form_retirement_gc_missing", "attention"); + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt); + assertSlackFormRetirementWorkAuthority(row, identity, receipt); + const expected = { version: 1, binding: receipt.binding, instanceId: identity.instanceId, environment: identity.environment, + subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + if (!isDeepStrictEqual(row.result.retirement, expected)) throw new SlackCloudReceiptError("form_retirement_gc_authority_changed", "attention"); + return receipt; +} +async function releaseSlackRetirementFact(db: Db, row: Row, identity: SlackFormConnectorIdentity, protocol: SlackFormProtocol): Promise { + const receipt = assertSlackRetirementGCWork(row, identity); + const response = await protocol({ subject: receipt.subject, companyId: row.companyId, + operation: { kind: "cleanup", binding: receipt.binding, releaseRetirementFact: { receiptId: receipt.receiptId } } }); + if (!response || typeof response !== "object" || Array.isArray(response) || Object.keys(response).join() !== "releasedRetirementDigest" + || Reflect.get(response, "releasedRetirementDigest") !== receipt.receiptId) return retry("form_retirement_gc_unconfirmed"); + await db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) return retry("form_retirement_gc_changed"); + const { retirementReceiptJws: _, ...result } = current.result ?? {}; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const rows = await tx.update(chatActions).set({ status: "processed", result: { ...result, phase: "transferred", nextAttemptAt: null }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (rows.length !== 1) return retry("form_retirement_gc_changed"); + }); +} diff --git a/server/src/services/chat-channels/slack/form-retirement.test.ts b/server/src/services/chat-channels/slack/form-retirement.test.ts new file mode 100644 index 00000000000..32face47804 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retirement.test.ts @@ -0,0 +1,88 @@ +import { createHash, generateKeyPairSync, sign } from "node:crypto"; +import { afterEach, expect, it, vi } from "vitest"; +import type { chatActions } from "@paperclipai/db"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormRetirementReceipt, slackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { assertSlackFormRetirementWorkAuthority, verifyRetainedSlackFormRetirement } from "./form-retirement-work.js"; +import { verifySlackFormFinalizedReceipt, verifySlackFormRetirementReceipt } from "./receipt-authority.js"; +import { projectSlackFormRetainedOwner } from "./form-session.js"; + +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "company", endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +const { credentialFingerprint, eventBindingId, ...base } = binding; +const enrollment = { ...base, bindingId: eventBindingId }, identity = { instanceId: "instance", environment: "test" as const }; +const receipt = slackFormRetirementReceipt(binding, "test", "original-actor"); +function row(): typeof chatActions.$inferSelect { + return { id: "11111111-1111-4111-8111-111111111111", companyId: "company", endpointId: "endpoint", deliveryId: null, + conversationId: null, principalId: null, kind: "slack_form_owner", providerActionId: `slack-form-owner:${createHash("sha256").update(JSON.stringify(enrollment)).digest("hex")}`, + payload: { enrollment, binding, environment: "test", subject: "original-actor" }, status: "issued", result: { phase: "retired_cleanup", nextAttemptAt: null }, + createdAt: new Date(0), updatedAt: new Date(0) }; +} +function signed(value: unknown = receipt) { + const pair = generateKeyPairSync("ed25519"), kid = "retirement-unit"; + const env = { PAPERCLIP_CLOUD_STACK_ID: "instance", PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, use: "sig", alg: "EdDSA" }] }) }; + const enc = (input: unknown) => Buffer.from(JSON.stringify(input)).toString("base64url"); + const body = enc({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + enc({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: "instance", receipt: value }); + return { compactJws: body + "." + sign(null, Buffer.from(body), pair.privateKey).toString("base64url"), env }; +} +afterEach(() => vi.unstubAllEnvs()); +it("verifies a real runtime signature without admitting the other receipt kind", () => { + const input = signed(); expect(verifySlackFormRetirementReceipt(input)).toEqual(receipt); + expect(() => verifySlackFormFinalizedReceipt(input)).toThrow(); +}); +it("rejects foreign instance, unknown signing key and tampered retirement payload", () => { + const input = signed(); + expect(() => verifySlackFormRetirementReceipt({ ...input, env: { ...input.env, PAPERCLIP_CLOUD_STACK_ID: "foreign" } })).toThrow(); + expect(() => verifySlackFormRetirementReceipt({ ...input, env: { ...input.env, PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: '{"keys":[]}' } })).toThrow(); + const parts = input.compactJws.split("."); parts[1] = Buffer.from("{}").toString("base64url"); + expect(() => verifySlackFormRetirementReceipt({ ...input, compactJws: parts.join(".") })).toThrow(); +}); +it("fails closed on changed canonical identity, extra keys and oversized original subject", () => { + expect(() => parseSlackFormRetirementReceipt({ ...receipt, subject: "replacement" })).toThrow(); + expect(() => parseSlackFormRetirementReceipt({ ...receipt, extra: true })).toThrow(); + expect(() => slackFormRetirementReceipt(binding, "test", "x".repeat(201))).toThrow(); +}); +it("requires exact local company, immutable original subject, fingerprint and owner action", () => { + const current = row(); expect(() => assertSlackFormRetirementWorkAuthority(current, identity, receipt)).not.toThrow(); + for (const changed of [{ ...current, companyId: "foreign" }, { ...current, payload: { ...current.payload, subject: "replacement" } }, + { ...current, payload: { ...current.payload, binding: { ...binding, credentialFingerprint: "c".repeat(64) } } }, { ...current, providerActionId: "foreign-action" }]) + expect(() => assertSlackFormRetirementWorkAuthority(changed, identity, receipt)).toThrow(); + expect(() => assertSlackFormRetirementWorkAuthority(current, { ...identity, instanceId: "foreign" }, receipt)).toThrow(); +}); +it("key outage remains retryable and invalid signature is visible attention", () => { + const input = signed(); vi.stubEnv("PAPERCLIP_CLOUD_STACK_ID", "instance"); vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", ""); + expect(() => verifyRetainedSlackFormRetirement(input.compactJws)).toThrow(expect.objectContaining({ code: "form_retirement_verifier_unavailable" })); + vi.stubEnv("PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS", input.env.PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS); + const other = signed(); expect(() => verifyRetainedSlackFormRetirement(other.compactJws)).toThrow(expect.objectContaining({ code: "form_retirement_receipt_invalid" })); +}); +it("accepted retirement retains normal polling identity without granting new-use delivery authority", () => { + const current = row(), input = signed(); current.result = { phase: "retirement_accepted", retirementReceipt: receipt, retirementReceiptJws: input.compactJws, nextAttemptAt: new Date(0).toISOString() }; + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toEqual({ subject: "github-bot-events", companyId: current.companyId, binding, requiresDelivery: false }); + const missingBinding = { ...current, payload: { enrollment, environment: "test", subject: "original-actor" } }; + expect(projectSlackFormRetainedOwner(missingBinding, identity.instanceId, identity.environment)).not.toBeNull(); + expect(() => projectSlackFormRetainedOwner({ ...current, result: { ...current.result, retirementReceiptJws: "invalid" } }, identity.instanceId, identity.environment)).toThrow(); +}); +it("only exact committed transfer discharges history and its retained consumer", () => { + const current = row(); current.status = "processed"; + const retirement = { version: 1, binding, instanceId: identity.instanceId, environment: identity.environment, subject: receipt.subject, receiptDigest: receipt.receiptId, receipt }; + current.result = { phase: "transferred", nextAttemptAt: null, retirement }; + expect(slackFormRetirementHistoryReceipt(current)).toEqual(receipt); + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toBeNull(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, companyId: "foreign" })).toThrow(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, result: { ...current.result, retirement: { ...retirement, receiptDigest: "0".repeat(64) } } })).toThrow(); + expect(() => slackFormRetirementHistoryReceipt({ ...current, result: { phase: "transferred", nextAttemptAt: null } })).toThrow(); +}); + +it("preserves exact legacy custody through JSONB ordering for acceptance, polling and history", () => { + const current = row(); + current.payload = { ...current.payload, enrollment: Object.fromEntries(Object.entries(enrollment).sort(([a], [b]) => Buffer.byteLength(a) - Buffer.byteLength(b) || a.localeCompare(b))) }; + expect(() => assertSlackFormRetirementWorkAuthority(current, identity, receipt)).not.toThrow(); + const input = signed(); + current.result = { phase: "retirement_accepted", retirementReceipt: receipt, retirementReceiptJws: input.compactJws, nextAttemptAt: new Date(0).toISOString() }; + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)?.requiresDelivery).toBe(false); + current.status = "processed"; + current.result = { phase: "transferred", nextAttemptAt: null, retirement: { version: 1, binding, instanceId: identity.instanceId, + environment: identity.environment, subject: receipt.subject, receiptDigest: receipt.receiptId, receipt } }; + expect(slackFormRetirementHistoryReceipt(current)).toEqual(receipt); + expect(projectSlackFormRetainedOwner(current, identity.instanceId, identity.environment)).toBeNull(); +}); diff --git a/server/src/services/chat-channels/slack/form-retry.ts b/server/src/services/chat-channels/slack/form-retry.ts new file mode 100644 index 00000000000..d2fcebf36e9 --- /dev/null +++ b/server/src/services/chat-channels/slack/form-retry.ts @@ -0,0 +1,87 @@ +import { parseSlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { assertSlackFormRetirementWorkAuthority, assertSlackRetirementGCWork } from "./form-retirement-work.js"; +import { slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, type Db } from "@paperclipai/db"; +import type { ConnectionManagementContext } from "../../connection-management.js"; +import { logActivity } from "../../activity-log.js"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { badRequest, conflict, forbidden, notFound } from "../../../errors.js"; +import { parseSlackFormBinding, sameSlackFormBinding } from "./cloud-forms.js"; +import { parseSlackFormProofAcceptance } from "./form-session.js"; +import { parseSlackFormAckWork } from "./form-ack-work.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +export async function retrySlackFormAcknowledgement(db: Db, input: { companyId: string; endpointId: string; actionId: string; + identity(): SlackFormConnectorIdentity | null; management: ConnectionManagementContext }): Promise<{ actionId: string; state: "scheduled" | "already_scheduled" | "complete" }> { + if (!/^[a-f0-9-]{36}$/i.test(input.actionId)) throw badRequest("Invalid provider work ID"); + await input.management.reauthorize(); + const identity = input.identity(); + if (input.management.resource.companyId !== input.companyId || !identity || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current company and Cloud management authority are required"); + const result = await db.transaction(async tx => { + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, input.endpointId), eq(chatEndpoints.companyId, input.companyId))).for("update"); + if (!endpoint || endpoint.provider !== "slack") throw notFound("Slack endpoint not found"); + if (input.management.resource.connectionId && input.management.resource.connectionId !== endpoint.connectionId) throw forbidden("Connection authority changed"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, input.actionId), eq(chatActions.companyId, input.companyId), + eq(chatActions.endpointId, endpoint.id), inArray(chatActions.kind, ["slack_form_session", "slack_form_owner"]))).for("update"); + if (!row) throw notFound("Slack form work not found"); + if (row.kind === "slack_form_owner") { + if (row.result?.phase === "transferred") { + const retired = slackFormRetirementHistoryReceipt(row); + if (!retired || retired.binding.instanceId !== identity.instanceId || retired.environment !== identity.environment + || retired.binding.publicId !== endpoint.publicId || retired.binding.connectionId !== endpoint.connectionId) throw conflict("Retirement history authority changed"); + return { actionId: row.id, state: "complete" as const }; + } + if (!["retirement_accepted", "retirement_gc_pending"].includes(String(row.result?.phase)) || typeof row.result?.retirementReceiptJws !== "string") throw conflict("No accepted retirement responsibility is available for retry"); + const receipt = row.result?.phase === "retirement_gc_pending" ? assertSlackRetirementGCWork(row, identity) : parseSlackFormRetirementReceipt(row.result.retirementReceipt); + assertSlackFormRetirementWorkAuthority(row, identity, receipt); + if (receipt.binding.publicId !== endpoint.publicId || receipt.binding.connectionId !== endpoint.connectionId + || ["SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_CUSTODY_CHANGED"].includes(String(row.result.failureReason))) + throw conflict("Retirement responsibility does not match current ownership"); + if (typeof row.result.nextAttemptAt === "string" && Number.isFinite(Date.parse(row.result.nextAttemptAt))) return { actionId: row.id, state: "already_scheduled" as const }; + if (row.status !== "attention" || row.result.nextAttemptAt !== null) throw conflict("Retirement work has no valid retry state"); + await input.management.reauthorize(); + const currentIdentity = input.identity(); + if (!currentIdentity || currentIdentity.instanceId !== identity.instanceId || currentIdentity.environment !== identity.environment + || input.management.resource.companyId !== input.companyId || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current Cloud or management authority changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ status: "issued", result: { ...row.result, + firstFailureCode: row.result.firstFailureCode ?? row.result.failureCode, nextAttemptAt: new Date().toISOString() }, updatedAt: new Date() }) + .where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw conflict("Retirement work changed before retry was committed"); + return { actionId: row.id, state: "scheduled" as const }; + } + const binding = parseSlackFormBinding(row.payload.binding), proof = parseSlackFormProofAcceptance(row.result?.cloudProofAcceptance), work = parseSlackFormAckWork(row.result?.formAck); + if (binding.companyId !== input.companyId || binding.endpointId !== endpoint.id || binding.publicId !== endpoint.publicId || binding.connectionId !== endpoint.connectionId + || binding.instanceId !== identity.instanceId || row.payload.environment !== identity.environment || !sameSlackFormBinding(proof.binding, binding) + || proof.appReceiptId !== row.id || proof.operationId !== row.payload.operationId || work.deliveryId !== `slack_form_${proof.operationId}` + || !["submitted", "closed", "not_opened"].includes(String(row.result?.phase))) throw conflict("Retained form responsibility does not match current Cloud authority"); + if (work.receipt && (work.receipt.appReceiptId !== row.id || work.receipt.operationId !== proof.operationId + || !sameSlackFormBinding(work.receipt.binding, binding) || work.receipt.terminalDigest !== proof.terminalDigest)) throw conflict("Retained form receipt changed"); + if (work.phase === "transferred") { + if (row.status === "attention" || row.result?.nextAttemptAt !== null) throw conflict("Completed provider work has inconsistent retained status"); + return { actionId: row.id, state: "complete" as const }; + } + if (typeof row.result?.nextAttemptAt === "string" && Number.isFinite(Date.parse(row.result.nextAttemptAt))) return { actionId: row.id, state: "already_scheduled" as const }; + if (row.status !== "attention" || row.result?.nextAttemptAt !== null) throw conflict("Provider work has no valid retained retry state"); + await input.management.reauthorize(); + const currentIdentity = input.identity(); + if (!currentIdentity || currentIdentity.instanceId !== identity.instanceId || currentIdentity.environment !== identity.environment + || input.management.resource.companyId !== input.companyId || !input.management.decision.allowed || input.management.decision.action !== "connections:configure") + throw forbidden("Current Cloud or management authority changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const nextAttemptAt = new Date().toISOString(); + const phase = row.result?.phase, status = phase === "submitted" ? "processed" : phase === "not_opened" ? "failed" : "cancelled"; + const updated = await tx.update(chatActions).set({ status, result: { ...row.result, firstFailureCode: row.result?.firstFailureCode ?? row.result?.failureCode, + nextAttemptAt }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, input.companyId), eq(chatActions.endpointId, endpoint.id), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw conflict("Provider work changed before retry was committed"); + return { actionId: row.id, state: "scheduled" as const }; + }); + if (result.state === "scheduled") await logActivity(db, { ...input.management.audit, companyId: input.companyId, action: "chat_provider_work.retry_requested", + entityType: "tool_connection", entityId: input.management.resource.connectionId ?? input.endpointId, details: { endpointId: input.endpointId, actionId: input.actionId } }); + return result; +} diff --git a/server/src/services/chat-channels/slack/form-session.ts b/server/src/services/chat-channels/slack/form-session.ts new file mode 100644 index 00000000000..36c6c322a4b --- /dev/null +++ b/server/src/services/chat-channels/slack/form-session.ts @@ -0,0 +1,448 @@ +import { isSlackFormEnrollmentActionId } from "./form-enrollment-identity.js"; +import { parseSlackFormRetirementReceipt, slackFormRetirementHistoryReceipt } from "./ack-retirement-receipt.js"; +import { createHash } from "node:crypto"; +import type { IssueThreadInteraction } from "@paperclipai/shared"; +import { createLocalEncryptedCipher } from "../../../secrets/local-encrypted-provider.js"; +import type { StoredSecretVersionMaterial } from "../../../secrets/types.js"; +import { isDeepStrictEqual } from "node:util"; +import { and, eq, inArray, sql } from "drizzle-orm"; +import { chatActions, chatEndpoints, chatExternalPrincipals, chatPublications, type Db } from "@paperclipai/db"; +import type { ResolvedChatQuestionFormOpen } from "../../chat-question-forms.js"; +import type { CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; +import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; +import { assertSlackFormTerminalAck, parseSlackFormPublicEnrollment, parseSlackFormOperation, currentSlackFormOwnerBinding, parseSlackFormBinding, slackFormSubmissionEvidence, slackIssuedFormGrantId, projectSlackIssuedFormGrant, sameSlackFormBinding, type SlackFormBinding, type SlackFormOperation, type SlackFormView } from "./cloud-forms.js"; + +type FormTx = Parameters[0]>[0]; +export type SlackFormSessionScope = { companyId: string; endpointId: string; principalId: string; operationId: string }; +function fail(code: string): never { throw new SlackCloudReceiptError(code, "attention"); } +export async function stageSlackFormSession(tx: FormTx, input: { + operation: SlackFormOperation; currentBinding: SlackFormBinding; principalId: string; + resolved: ResolvedChatQuestionFormOpen; lease: CredentialMutationLeaseGuard; environment: string; +}): Promise<{ kind: "update"; actionId: string; view: SlackFormView } | { kind: "closed" | "retained"; actionId: string }> { + const { operation: op, currentBinding: binding, resolved } = input; + await input.lease.assertOwned(tx); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.companyId, binding.companyId), eq(chatEndpoints.id, binding.endpointId))).for("update"); + const setup: Record | undefined = endpoint?.setup && typeof endpoint.setup === "object" && !Array.isArray(endpoint.setup) ? { ...endpoint.setup } : undefined; + if (!endpoint || endpoint.provider !== "slack" || !["active", "verifying", "attention"].includes(endpoint.status) + || endpoint.publicId !== binding.publicId || endpoint.connectionId !== binding.connectionId + || endpoint.providerAccountId !== binding.teamId || setup?.runtimeGeneration !== binding.runtimeGeneration + || !sameSlackFormBinding(binding, op.binding)) fail("form_current_endpoint_changed"); + const localOwner = currentSlackFormOwnerBinding(setup ?? {}); + if (!localOwner || !sameSlackFormBinding(localOwner, binding)) fail("form_current_enrollment_changed"); + const [principal] = await tx.select().from(chatExternalPrincipals).where(and(eq(chatExternalPrincipals.id, input.principalId), + eq(chatExternalPrincipals.companyId, binding.companyId), eq(chatExternalPrincipals.provider, "slack"))).limit(1); + if (!principal || principal.externalId !== op.userId || principal.providerAccountId !== binding.teamId) fail("form_current_principal_changed"); + const grant = await projectSlackIssuedFormGrant(tx, binding, resolved); + if (!grant || grant.grantId !== op.grantId || grant.interactionId !== resolved.interactionId) fail("form_current_issued_authority_changed"); + const providerActionId = `slack-form-session:${op.id}`; + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.providerActionId, providerActionId))).for("update"); + const payload = { version: 1, binding, environment: input.environment, operationId: op.id, grantId: op.grantId, requestDigest: op.requestDigest, + userId: op.userId, publicationId: grant.publicationId, interactionId: grant.interactionId, + openActionRowId: grant.openActionRowId, submitActionRowId: grant.submitActionRowId, + view: op.view ?? op.closed?.view ?? null }; + if (old) { + if (old.kind !== "slack_form_session" || old.principalId !== principal.id || old.conversationId !== grant.conversationId + || !isDeepStrictEqual(old.payload, payload)) fail("form_session_identity_changed"); + if (op.closed) { + await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, operationId: op.id }, op.closed.view); + return { kind: "closed", actionId: old.id }; + } + if (old.status === "processing" && old.result?.phase === "update_attempted" && op.phase === "confirmed" && op.view + && isDeepStrictEqual(old.result.view, op.view)) { + await input.lease.assertOwned(tx); + return { kind: "update", actionId: old.id, view: op.view }; + } + return { kind: old.status === "cancelled" ? "closed" : "retained", actionId: old.id }; + } + if (["reserved", "attempted"].includes(op.phase) && !op.closed) throw new SlackCloudReceiptError("form_receipt_pending", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, + conversationId: grant.conversationId, kind: "slack_form_session", providerActionId, payload, + status: op.closed ? "cancelled" : op.phase === "confirmed" ? "processing" : "attention", + result: op.closed ? { phase: "closed", view: op.closed.view } : op.phase === "confirmed" + ? { phase: "update_attempted", view: op.view } : { phase: "provider_outcome_unresolved" } }) + .onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_session_claim_changed", "retry"); + await input.lease.assertOwned(tx); + if (op.closed) await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: principal.id, operationId: op.id }, op.closed.view); + return op.closed ? { kind: "closed", actionId: row.id } : op.phase === "confirmed" && op.view + ? { kind: "update", actionId: row.id, view: op.view } : { kind: "retained", actionId: row.id }; +} +export async function confirmSlackFormSessionUpdate(tx: FormTx, scope: SlackFormSessionScope, view: SlackFormView, newHash: string): Promise { + if (!newHash || newHash.length > 200) fail("form_updated_hash_invalid"); + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), + eq(chatActions.principalId, scope.principalId), eq(chatActions.providerActionId, `slack-form-session:${scope.operationId}`), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row || row.status !== "processing" || row.result?.phase !== "update_attempted") return false; + const expected = row.result.view; + if (!expected || typeof expected !== "object" || Array.isArray(expected) || Reflect.get(expected, "id") !== view.id + || Reflect.get(expected, "hash") !== view.hash || Reflect.get(expected, "externalId") !== view.externalId) fail("form_updated_view_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { phase: "ready", view: { ...view, hash: newHash } }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, scope.companyId), eq(chatActions.status, "processing"), sql`${chatActions.result}->>'phase' = 'update_attempted'`)).returning({ id: chatActions.id }); + return updated.length === 1; +} +export async function closeSlackFormSession(tx: FormTx, scope: SlackFormSessionScope, view: SlackFormView): Promise { + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), + eq(chatActions.principalId, scope.principalId), eq(chatActions.providerActionId, `slack-form-session:${scope.operationId}`), eq(chatActions.kind, "slack_form_session"))).for("update"); + if (!row) return false; + const current = row.result?.view; + if (!current || typeof current !== "object" || Array.isArray(current) || Reflect.get(current, "id") !== view.id + || Reflect.get(current, "externalId") !== view.externalId + || (Reflect.get(current, "hash") !== view.hash && !(row.status === "processing" && row.result?.phase === "update_attempted"))) fail("form_close_view_changed"); + if (row.status === "cancelled" && row.result?.phase === "closed" && row.result.terminalIntent) return true; + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const identity = holdIdentity(row); + const terminalIntent: SlackFormTerminalIntent = { ...identity, view, outcome: "closed", submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ status: "cancelled", result: { phase: "closed", view, terminalIntent }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, scope.companyId), eq(chatActions.endpointId, scope.endpointId), eq(chatActions.principalId, scope.principalId))); + return true; +} + +export async function slackFormSessionsRequireDelivery(db: Pick): Promise { + const rows = await db.select({ id: chatActions.id }).from(chatActions).where(and(eq(chatActions.kind, "slack_form_session"), + sql`(${chatActions.status} in ('processing','attention') OR (${chatActions.result}->'terminalIntent' is not null AND ${chatActions.result}->'cloudProofAcceptance' is null))`)).limit(1); + return rows.length > 0; +} + +export type SlackFormTerminalIntent = { + version: 1; binding: SlackFormBinding; operationId: string; deliveryId: string; userId: string; grantId: string; + publicationId: string; interactionId: string; view: SlackFormView | null; appReceiptId: string; + outcome: "closed" | "submitted" | "no_effect"; submitReceiptId: string | null; rejectionDigest: string | null; + encryptedEvidence: StoredSecretVersionMaterial | null; +}; +export type SlackFormEvidenceCipher = ReturnType; +const digest = (value: string) => createHash("sha256").update(value).digest("hex"); +function bodyRecord(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) fail("form_terminal_invalid"); + return value as Record; +} +function holdIdentity(row: Pick) { + const p = row.payload, binding = parseSlackFormBinding(p.binding); + if (binding.companyId !== row.companyId || binding.endpointId !== row.endpointId || typeof p.operationId !== "string" + || !/^[a-f0-9]{64}$/.test(p.operationId) || row.providerActionId !== `slack-form-session:${p.operationId}` + || ["userId", "grantId", "publicationId", "interactionId"].some(k => typeof p[k] !== "string" || !(p[k] as string).length)) fail("form_hold_identity_changed"); + return { version: 1 as const, binding, operationId: p.operationId, deliveryId: `slack_form_${p.operationId}`, userId: String(p.userId), + grantId: String(p.grantId), publicationId: String(p.publicationId), interactionId: String(p.interactionId), appReceiptId: row.id }; +} +type TerminalIdentity = Pick; +export function prepareSlackFormSubmissionIntent(identity: TerminalIdentity, view: SlackFormView, submitReceiptId: string, + original: NonNullable>, cipher: SlackFormEvidenceCipher): SlackFormTerminalIntent { + let encryptedEvidence: StoredSecretVersionMaterial; + try { encryptedEvidence = cipher.encrypt(JSON.stringify({ ...identity, kind: "submission", view, submitReceiptId, original })); } + catch { return fail("form_submission_cipher_unavailable"); } + return { ...identity, view, outcome: "submitted", submitReceiptId, rejectionDigest: null, encryptedEvidence }; +} +export function openSlackFormSubmissionIntent(intent: SlackFormTerminalIntent, cipher: SlackFormEvidenceCipher): { + acceptedAt: string; rawBody: string; timestamp: string; signature: string; +} { + let decrypted: Record; + try { decrypted = bodyRecord(JSON.parse(cipher.decrypt(bodyRecord(intent.encryptedEvidence)))); } + catch { return fail("form_submission_cipher_unavailable"); } + if (intent.outcome !== "submitted" || decrypted.version !== 1 || decrypted.kind !== "submission" + || ["appReceiptId", "operationId", "deliveryId", "userId", "grantId", "publicationId", "interactionId", "submitReceiptId"].some(k => decrypted[k] !== Reflect.get(intent, k)) + || !sameSlackFormBinding(parseSlackFormBinding(decrypted.binding), intent.binding) || !isDeepStrictEqual(decrypted.view, intent.view)) fail("form_submission_cipher_binding_changed"); + const original = bodyRecord(decrypted.original); + if (original.endpointId !== intent.binding.endpointId || ["acceptedAt", "rawBody", "timestamp", "signature"].some(k => typeof original[k] !== "string")) + fail("form_submission_cipher_binding_changed"); + return { acceptedAt: String(original.acceptedAt), rawBody: String(original.rawBody), timestamp: String(original.timestamp), signature: String(original.signature) }; +} +export async function stageSlackFormSubmission(tx: FormTx, args: { + companyId: string; endpointId: string; conversationId: string; principalId: string; authorizedUserId: string; runtimeGeneration: number; + submitReceiptId: string; publicationId: string; resolved: IssueThreadInteraction; cipher?: SlackFormEvidenceCipher; +}): Promise { + const original = slackFormSubmissionEvidence(); + if (!original) return null; + if (original.endpointId !== args.endpointId) fail("form_submission_endpoint_changed"); + const payload = bodyRecord(JSON.parse(new URLSearchParams(Buffer.from(original.rawBody, "base64").toString("utf8")).get("payload") ?? "null")); + const view = bodyRecord(payload.view), user = bodyRecord(payload.user), team = bodyRecord(payload.team); + if (payload.type !== "view_submission" || typeof view.external_id !== "string" || !/^pc-form-[a-f0-9]{64}$/.test(view.external_id)) fail("form_submission_identity_changed"); + const operationId = view.external_id.slice("pc-form-".length); + const [hold] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, args.companyId), eq(chatActions.endpointId, args.endpointId), + eq(chatActions.conversationId, args.conversationId), eq(chatActions.principalId, args.principalId), eq(chatActions.kind, "slack_form_session"), + eq(chatActions.providerActionId, `slack-form-session:${operationId}`))).for("update"); + if (!hold) fail("form_submission_hold_missing"); + const identity = holdIdentity(hold), expectedView = bodyRecord(hold.result?.view); + const [endpoint] = await tx.select().from(chatEndpoints).where(and(eq(chatEndpoints.id, args.endpointId), eq(chatEndpoints.companyId, args.companyId))).limit(1); + const setup: Record | undefined = endpoint?.setup && typeof endpoint.setup === "object" && !Array.isArray(endpoint.setup) ? { ...endpoint.setup } : undefined, owner = setup ? currentSlackFormOwnerBinding(setup) : null; + if (!endpoint || !owner || !sameSlackFormBinding(owner, identity.binding) || args.runtimeGeneration !== identity.binding.runtimeGeneration + || setup?.runtimeGeneration !== args.runtimeGeneration || endpoint.connectionId !== identity.binding.connectionId || endpoint.publicId !== identity.binding.publicId) + fail("form_submission_current_generation_changed"); + const [receipt] = await tx.select().from(chatActions).where(and(eq(chatActions.id, args.submitReceiptId), eq(chatActions.companyId, args.companyId), + eq(chatActions.endpointId, args.endpointId), eq(chatActions.conversationId, args.conversationId), eq(chatActions.principalId, args.principalId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.status, "processed"))).limit(1); + if (!receipt || receipt.result?.code !== "question_form_answered" || receipt.result.interactionId !== args.resolved.id + || receipt.payload.publicationId !== args.publicationId || receipt.providerActionId !== view.callback_id || hold.payload.submitActionRowId !== receipt.id + || args.resolved.companyId !== args.companyId || args.resolved.id !== identity.interactionId || args.resolved.status !== "answered" + || args.resolved.resolvedByUserId !== args.authorizedUserId || args.publicationId !== identity.publicationId + || user.id !== identity.userId || team.id !== identity.binding.teamId || payload.api_app_id !== identity.binding.appId || view.app_id !== identity.binding.appId + || view.id !== expectedView.id || view.external_id !== expectedView.externalId || typeof view.hash !== "string" + || (view.hash !== expectedView.hash && hold.result?.phase !== "update_attempted")) fail("form_submission_receipt_changed"); + if (hold.status === "processed" && hold.result?.phase === "submitted") { + const old = bodyRecord(hold.result.terminalIntent); + if (old.submitReceiptId !== receipt.id || old.appReceiptId !== hold.id || old.operationId !== operationId) fail("form_submission_receipt_changed"); + return hold.id; + } + if (hold.status !== "processing" || !["ready", "update_attempted"].includes(String(hold.result?.phase))) fail("form_submission_hold_changed"); + const currentView = { id: String(view.id), hash: String(view.hash), externalId: String(view.external_id) }; + let intent: SlackFormTerminalIntent; + try { intent = prepareSlackFormSubmissionIntent(identity, currentView, receipt.id, original, args.cipher ?? createLocalEncryptedCipher("read")); } + catch { return fail("form_submission_cipher_unavailable"); } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + await tx.update(chatActions).set({ status: "processed", result: { phase: "submitted", view: currentView, terminalIntent: intent }, updatedAt: new Date() }).where(and( + eq(chatActions.id, hold.id), eq(chatActions.companyId, args.companyId), eq(chatActions.status, "processing"))); + return hold.id; +} +export async function acceptSlackFormNoEffect(tx: FormTx, operation: SlackFormOperation, originalActionId: string, environment?: string): Promise { + if (operation.phase !== "rejected" || !operation.rejection || operation.view || operation.closed) fail("form_no_effect_unproven"); + const binding = operation.binding; + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_open"), eq(chatActions.providerActionId, originalActionId))).limit(1); + const [submit] = open ? await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.providerActionId, String(open.payload.formActionId)), eq(chatActions.conversationId, open.conversationId!))).limit(1) : []; + if (!open || !submit || !open.conversationId || open.payload.version !== 1 || submit.payload.version !== 1 + || typeof open.payload.publicationId !== "string" || typeof open.payload.interactionId !== "string" + || submit.payload.publicationId !== open.payload.publicationId || submit.payload.interactionId !== open.payload.interactionId + || slackIssuedFormGrantId(binding, open.id, submit.id, open.payload.publicationId, open.payload.interactionId) !== operation.grantId) + fail("form_no_effect_issued_authority_changed"); + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, open.payload.publicationId), + eq(chatPublications.companyId, binding.companyId), eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.conversationId, open.conversationId))).limit(1); + if (!publication?.providerMessageId) fail("form_no_effect_publication_changed"); + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).for("update"); + if (old) { + const intent = bodyRecord(old.result?.terminalIntent); + if (old.status !== "failed" || old.result?.phase !== "not_opened" || intent.rejectionDigest !== operation.rejection.responseDigest + || intent.operationId !== operation.id || intent.appReceiptId !== old.id) fail("form_no_effect_conflicts_custody"); + return old.id; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, conversationId: open.conversationId, + kind: "slack_form_session", providerActionId: `slack-form-session:${operation.id}`, status: "failed", payload: { version: 1, binding, environment, operationId: operation.id, + grantId: operation.grantId, requestDigest: operation.requestDigest, userId: operation.userId, publicationId: open.payload.publicationId, + interactionId: open.payload.interactionId, openActionRowId: open.id, submitActionRowId: submit.id, view: null }, result: { phase: "not_opened" } }) + .onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_no_effect_claim_changed", "retry"); + const intent: SlackFormTerminalIntent = { version: 1, binding, operationId: operation.id, deliveryId: operation.deliveryId, userId: operation.userId, + grantId: operation.grantId, publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, appReceiptId: row.id, + outcome: "no_effect", submitReceiptId: null, rejectionDigest: operation.rejection.responseDigest, view: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ result: { phase: "not_opened", terminalIntent: intent }, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId))); + const credentialFailure = ["invalid_auth", "token_revoked", "not_authed", "invalid_arguments"].includes(operation.rejection.code); + const lastError = credentialFailure ? "Slack could not open the form. Repair the connection, then open the form again." + : "The Slack form request expired before opening. Open the form again to create a fresh request."; + await tx.update(chatEndpoints).set({ ...(credentialFailure ? { status: "attention" } : {}), lastError, updatedAt: new Date() }).where(and( + eq(chatEndpoints.id, binding.endpointId), eq(chatEndpoints.companyId, binding.companyId), eq(chatEndpoints.connectionId, binding.connectionId), + eq(chatEndpoints.publicId, binding.publicId), inArray(chatEndpoints.status, ["active", "verifying", "attention"]), + sql`${chatEndpoints.setup}->'slackCloudForms' = ${JSON.stringify(binding)}::jsonb`)); + return row.id; +} +export async function readSlackFormTerminalRequest(db: Pick, binding: SlackFormBinding, operationId: string, + cipher?: SlackFormEvidenceCipher): Promise | null> { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operationId}`))).limit(1); + if (!row || !["closed", "submitted", "not_opened"].includes(String(row.result?.phase))) return null; + const identity = holdIdentity(row), intent = bodyRecord(row.result?.terminalIntent); + if (!sameSlackFormBinding(identity.binding, binding) || intent.appReceiptId !== row.id || intent.operationId !== operationId + || intent.grantId !== identity.grantId || intent.publicationId !== identity.publicationId || intent.interactionId !== identity.interactionId + || intent.userId !== identity.userId || !sameSlackFormBinding(parseSlackFormBinding(intent.binding), binding)) fail("form_terminal_intent_changed"); + const view = intent.view === null ? null : bodyRecord(intent.view); + let terminal: Record; + if (intent.outcome === "submitted") { + if (intent.submitReceiptId !== row.payload.submitActionRowId) fail("form_submission_cipher_binding_changed"); + let original: ReturnType; + try { original = openSlackFormSubmissionIntent(intent as unknown as SlackFormTerminalIntent, cipher ?? createLocalEncryptedCipher("read")); } + catch (error) { if (error instanceof SlackCloudReceiptError) throw error; return fail("form_submission_cipher_unavailable"); } + terminal = { kind: "submitted", submitReceiptId: intent.submitReceiptId, evidence: original }; + } else if (intent.outcome === "no_effect") { + if (typeof intent.rejectionDigest !== "string" || !/^[a-f0-9]{64}$/.test(intent.rejectionDigest)) fail("form_terminal_intent_changed"); + terminal = { kind: "no_effect", responseDigest: intent.rejectionDigest }; + } else if (intent.outcome === "closed") terminal = { kind: "closed" }; + else fail("form_terminal_intent_changed"); + return { kind: "terminal-ack", binding, operationId, deliveryId: identity.deliveryId, userId: identity.userId, grantId: identity.grantId, + publicationId: identity.publicationId, interactionId: identity.interactionId, viewId: view?.id ?? null, viewHash: view?.hash ?? null, + externalId: view?.externalId ?? `pc-form-${operationId}`, appReceiptId: row.id, terminal: terminal! }; +} + +export type SlackFormProofAcceptance = { version: 1; binding: SlackFormBinding; operationId: string; appReceiptId: string; + outcome: SlackFormTerminalIntent["outcome"]; terminalDigest: string; acceptedAt: string }; +export function parseSlackFormProofAcceptance(value: unknown): SlackFormProofAcceptance { + const v = bodyRecord(value); + if (Object.keys(v).sort().join() !== ["version", "binding", "operationId", "appReceiptId", "outcome", "terminalDigest", "acceptedAt"].sort().join() + || v.version !== 1 || typeof v.operationId !== "string" || !/^[a-f0-9]{64}$/.test(v.operationId) + || typeof v.appReceiptId !== "string" || !/^[a-f0-9-]{36}$/i.test(v.appReceiptId) + || typeof v.terminalDigest !== "string" || !/^[a-f0-9]{64}$/.test(v.terminalDigest) + || typeof v.acceptedAt !== "string" || !Number.isFinite(Date.parse(v.acceptedAt)) + || v.outcome !== "closed" && v.outcome !== "submitted" && v.outcome !== "no_effect") fail("form_proof_acceptance_invalid"); + return { version: 1, binding: parseSlackFormBinding(v.binding), operationId: v.operationId, appReceiptId: v.appReceiptId, + outcome: v.outcome, terminalDigest: v.terminalDigest, acceptedAt: v.acceptedAt }; +} +export function prepareSlackFormProofAcceptance(intent: SlackFormTerminalIntent, value: unknown, currentInstanceId: string, + cipher?: SlackFormEvidenceCipher): SlackFormProofAcceptance { + const op = parseSlackFormOperation(value); + if (op.binding.instanceId !== currentInstanceId || !sameSlackFormBinding(op.binding, intent.binding) + || op.id !== intent.operationId || op.deliveryId !== intent.deliveryId || op.grantId !== intent.grantId || op.userId !== intent.userId) + fail("form_proof_identity_changed"); + assertSlackFormTerminalAck(op, intent.appReceiptId); + let terminalDigest: string; + if (intent.outcome === "no_effect") { + if (!op.rejection || op.closed || op.view || intent.view || op.rejection.responseDigest !== intent.rejectionDigest) fail("form_proof_outcome_changed"); + terminalDigest = op.rejection.responseDigest; + } else { + if (!op.closed || op.rejection || !intent.view || !isDeepStrictEqual(op.closed.view, intent.view) + || op.closed.kind !== (intent.outcome === "submitted" ? "submitted" : "closed")) fail("form_proof_outcome_changed"); + if (intent.outcome === "submitted") { + if (op.closed.evidence.submitReceiptId !== intent.submitReceiptId) fail("form_proof_submission_changed"); + const original = openSlackFormSubmissionIntent(intent, cipher ?? createLocalEncryptedCipher("read")); + if (digest(JSON.stringify([original.rawBody, original.timestamp, original.signature])) !== op.closed.requestDigest) + fail("form_proof_submission_changed"); + } + terminalDigest = op.closed.requestDigest; + } + if (!op.acknowledgement) fail("form_proof_missing"); + return { version: 1, binding: intent.binding, operationId: op.id, appReceiptId: intent.appReceiptId, outcome: intent.outcome, + terminalDigest, acceptedAt: op.acknowledgement.acceptedAt }; +} +export async function acceptSlackFormTerminalProof(tx: FormTx, input: { intent: SlackFormTerminalIntent; operation: unknown; + currentInstanceId: string; cipher?: SlackFormEvidenceCipher }): Promise { + const { intent } = input; + const [row] = await tx.select().from(chatActions).where(and(eq(chatActions.id, intent.appReceiptId), eq(chatActions.companyId, intent.binding.companyId), + eq(chatActions.endpointId, intent.binding.endpointId), eq(chatActions.kind, "slack_form_session"), + eq(chatActions.providerActionId, `slack-form-session:${intent.operationId}`))).for("update"); + if (!row || !["closed", "submitted", "not_opened"].includes(String(row.result?.phase)) + || !isDeepStrictEqual(row.result?.terminalIntent, intent)) fail("form_proof_intent_changed"); + const identity = holdIdentity(row); + if (identity.appReceiptId !== intent.appReceiptId || !sameSlackFormBinding(identity.binding, intent.binding) + || identity.grantId !== intent.grantId || identity.userId !== intent.userId || identity.publicationId !== intent.publicationId + || identity.interactionId !== intent.interactionId) fail("form_proof_intent_changed"); + const accepted = prepareSlackFormProofAcceptance(intent, input.operation, input.currentInstanceId, input.cipher); + if (row.result?.cloudProofAcceptance !== undefined) { + if (!isDeepStrictEqual(parseSlackFormProofAcceptance(row.result.cloudProofAcceptance), accepted)) fail("form_proof_acceptance_changed"); + return accepted; + } + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const updated = await tx.update(chatActions).set({ result: { ...row.result, cloudProofAcceptance: accepted }, updatedAt: new Date() }).where(and( + eq(chatActions.id, row.id), eq(chatActions.companyId, intent.binding.companyId), + sql`${chatActions.result}->'terminalIntent' = ${JSON.stringify(intent)}::jsonb`)).returning({ id: chatActions.id }); + if (updated.length !== 1) throw new SlackCloudReceiptError("form_proof_acceptance_changed", "retry"); + return accepted; +} +export type SlackFormRetainedRow = Pick; +export function projectSlackFormRetainedOwner(row: SlackFormRetainedRow, currentInstanceId: string, currentEnvironment?: string): { + subject: "github-bot-events"; companyId: string; binding: SlackFormBinding | null; requiresDelivery: boolean; +} | null { + if (["slack_form_owner", "slack_form_grant"].includes(row.kind) && row.result?.phase === "transferred" + && row.result.retirement && typeof row.result.retirement === "object" && Reflect.get(row.result.retirement, "receipt") !== undefined) { + if (!slackFormRetirementHistoryReceipt(row)) fail("form_retained_owner_changed"); + return null; + } + if (row.kind === "slack_form_owner" && (row.result?.phase === "retirement_accepted" || row.result?.phase === "retirement_gc_pending")) { + const receipt = parseSlackFormRetirementReceipt(row.result.retirementReceipt), enrollment = parseSlackFormPublicEnrollment(row.payload.enrollment), binding = receipt.binding; + if (receipt.binding.instanceId !== currentInstanceId || currentEnvironment !== undefined && receipt.environment !== currentEnvironment) return null; + if (row.companyId !== binding.companyId || row.endpointId !== binding.endpointId || row.payload.subject !== receipt.subject + || !isSlackFormEnrollmentActionId(row.providerActionId, enrollment) + || row.payload.environment !== receipt.environment || typeof row.result.retirementReceiptJws !== "string" + || Buffer.byteLength(row.result.retirementReceiptJws, "ascii") > 12000 + || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(row.result.retirementReceiptJws) + || enrollment.bindingId !== binding.eventBindingId + || ["instanceId", "companyId", "endpointId", "publicId", "connectionId", "runtimeGeneration", "teamId", "appId"].some(key => Reflect.get(enrollment, key) !== Reflect.get(binding, key)) + || row.payload.binding !== undefined && !sameSlackFormBinding(parseSlackFormBinding(row.payload.binding), binding)) fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: false }; + } + if (row.kind === "slack_form_owner" && row.payload.binding === undefined) { + const rawEnrollment = row.payload.enrollment; + if (!rawEnrollment || typeof rawEnrollment !== "object" || Array.isArray(rawEnrollment) || Reflect.get(rawEnrollment, "instanceId") !== currentInstanceId) return null; + const enrollment = parseSlackFormPublicEnrollment(rawEnrollment); + if (currentEnvironment !== undefined && row.payload.environment !== currentEnvironment) fail("form_retained_environment_changed"); + if (enrollment.companyId !== row.companyId || enrollment.endpointId !== row.endpointId + || typeof row.payload.subject !== "string" || !row.payload.subject.length + || row.result?.phase !== "staged") fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding: null, requiresDelivery: true }; + } + const raw = row.payload.binding; + if (!raw || typeof raw !== "object" || Array.isArray(raw) || Reflect.get(raw, "instanceId") !== currentInstanceId) return null; + if (currentEnvironment !== undefined && row.payload.environment !== currentEnvironment) fail("form_retained_environment_changed"); + const binding = parseSlackFormBinding(raw); + if (binding.companyId !== row.companyId || binding.endpointId !== row.endpointId) fail("form_retained_owner_changed"); + if (row.kind === "slack_form_session") { + const identity = holdIdentity(row); + if (!sameSlackFormBinding(identity.binding, binding)) fail("form_retained_owner_changed"); + if (row.result?.cloudProofAcceptance !== undefined) { + const accepted = parseSlackFormProofAcceptance(row.result.cloudProofAcceptance), intent = bodyRecord(row.result.terminalIntent); + if (accepted.appReceiptId !== row.id || accepted.operationId !== identity.operationId || intent.appReceiptId !== row.id + || intent.operationId !== identity.operationId || !sameSlackFormBinding(parseSlackFormBinding(accepted.binding), binding) + || !["closed", "submitted", "not_opened"].includes(String(row.result.phase))) fail("form_retained_owner_changed"); + } + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.cloudProofAcceptance === undefined }; + } + if (row.kind === "slack_form_grant") { + const grant = bodyRecord(row.payload.grant); + if (!sameSlackFormBinding(parseSlackFormBinding(grant.binding), binding) || typeof row.payload.subject !== "string" + || !["staged", "published"].includes(String(row.result?.phase))) fail("form_retained_grant_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.phase === "staged" }; + } + if (row.kind !== "slack_form_owner" || typeof row.payload.subject !== "string" || !row.payload.subject.length + || !["staged", "enrolled", "retired_cleanup", "transferred"].includes(String(row.result?.phase))) fail("form_retained_owner_changed"); + return { subject: "github-bot-events", companyId: row.companyId, binding, requiresDelivery: row.result?.phase !== "transferred" }; +} +export async function readSlackFormRetainedOwner(db: Pick, currentInstanceId: string) { + const [row] = await db.select().from(chatActions).where(and(inArray(chatActions.kind, ["slack_form_session", "slack_form_owner"]), + sql`${chatActions.payload}->'binding'->>'instanceId' = ${currentInstanceId}`, + sql`${chatActions.payload}->'binding'->>'companyId' = ${chatActions.companyId}::text`, + sql`${chatActions.payload}->'binding'->>'endpointId' = ${chatActions.endpointId}::text`)) + .orderBy(sql`case when ${chatActions.result}->'cloudProofAcceptance' is null and coalesce(${chatActions.result}->>'phase','') <> 'transferred' then 0 else 1 end`, chatActions.id).limit(1); + return row ? projectSlackFormRetainedOwner(row, currentInstanceId) : null; +} + +export async function acceptSlackFormClosed(tx: FormTx, operation: SlackFormOperation, originalActionId: string, environment: string): Promise { + if (!operation.closed || operation.closed.kind !== "closed" || operation.rejection) fail("form_close_unproven"); + const binding = operation.binding; + const [old] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).for("update"); + if (old) { + if (!sameSlackFormBinding(parseSlackFormBinding(old.payload.binding), binding) || old.payload.userId !== operation.userId) fail("form_close_owner_changed"); + if (old.principalId) { + await closeSlackFormSession(tx, { companyId: binding.companyId, endpointId: binding.endpointId, principalId: old.principalId, operationId: operation.id }, operation.closed.view); + return old.id; + } + if (old.status !== "cancelled" || old.result?.phase !== "closed" || !isDeepStrictEqual(old.result.view, operation.closed.view)) fail("form_close_owner_changed"); + return old.id; + } + const [open] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_open"), eq(chatActions.providerActionId, originalActionId))).limit(1); + const [submit] = open?.conversationId ? await tx.select().from(chatActions).where(and(eq(chatActions.companyId, binding.companyId), eq(chatActions.endpointId, binding.endpointId), + eq(chatActions.kind, "question_form_submit"), eq(chatActions.providerActionId, String(open.payload.formActionId)), eq(chatActions.conversationId, open.conversationId))).limit(1) : []; + if (!open || !submit || !open.conversationId || typeof open.payload.publicationId !== "string" || typeof open.payload.interactionId !== "string" + || submit.payload.publicationId !== open.payload.publicationId || submit.payload.interactionId !== open.payload.interactionId + || slackIssuedFormGrantId(binding, open.id, submit.id, open.payload.publicationId, open.payload.interactionId) !== operation.grantId) fail("form_close_issued_authority_changed"); + const [publication] = await tx.select().from(chatPublications).where(and(eq(chatPublications.id, open.payload.publicationId), eq(chatPublications.companyId, binding.companyId), + eq(chatPublications.endpointId, binding.endpointId), eq(chatPublications.conversationId, open.conversationId))).limit(1); + if (!publication?.providerMessageId) fail("form_close_publication_changed"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const [row] = await tx.insert(chatActions).values({ companyId: binding.companyId, endpointId: binding.endpointId, conversationId: open.conversationId, + kind: "slack_form_session", providerActionId: `slack-form-session:${operation.id}`, status: "cancelled", payload: { version: 1, binding, environment, + operationId: operation.id, grantId: operation.grantId, requestDigest: operation.requestDigest, userId: operation.userId, + publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, openActionRowId: open.id, submitActionRowId: submit.id, view: operation.closed.view }, + result: { phase: "closed", view: operation.closed.view } }).onConflictDoNothing().returning({ id: chatActions.id }); + if (!row) throw new SlackCloudReceiptError("form_close_claim_changed", "retry"); + const intent: SlackFormTerminalIntent = { version: 1, binding, operationId: operation.id, deliveryId: operation.deliveryId, userId: operation.userId, + grantId: operation.grantId, publicationId: open.payload.publicationId, interactionId: open.payload.interactionId, appReceiptId: row.id, + outcome: "closed", view: operation.closed.view, submitReceiptId: null, rejectionDigest: null, encryptedEvidence: null }; + await tx.update(chatActions).set({ result: { phase: "closed", view: operation.closed.view, terminalIntent: intent }, updatedAt: new Date() }).where(eq(chatActions.id, row.id)); + return row.id; +} +export async function readSlackFormTerminalIntent(db: Pick, operation: SlackFormOperation): Promise { + const [row] = await db.select().from(chatActions).where(and(eq(chatActions.companyId, operation.binding.companyId), eq(chatActions.endpointId, operation.binding.endpointId), + eq(chatActions.kind, "slack_form_session"), eq(chatActions.providerActionId, `slack-form-session:${operation.id}`))).limit(1); + if (!row?.result?.terminalIntent) return null; + const raw = bodyRecord(row.result.terminalIntent), identity = holdIdentity(row); + if (!sameSlackFormBinding(identity.binding, operation.binding) || identity.operationId !== operation.id || identity.grantId !== operation.grantId + || identity.userId !== operation.userId || Object.keys(identity).some(k => !isDeepStrictEqual(raw[k], Reflect.get(identity, k)))) fail("form_terminal_intent_changed"); + if (!["closed", "submitted", "no_effect"].includes(String(raw.outcome)) || (raw.submitReceiptId !== null && typeof raw.submitReceiptId !== "string") + || (raw.rejectionDigest !== null && typeof raw.rejectionDigest !== "string")) fail("form_terminal_intent_changed"); + const view = raw.view === null ? null : bodyRecord(raw.view); + if (view && (typeof view.id !== "string" || typeof view.hash !== "string" || typeof view.externalId !== "string")) fail("form_terminal_intent_changed"); + return { ...identity, outcome: raw.outcome as SlackFormTerminalIntent["outcome"], submitReceiptId: raw.submitReceiptId, + rejectionDigest: raw.rejectionDigest, view: view ? { id: String(view.id), hash: String(view.hash), externalId: String(view.externalId) } : null, + encryptedEvidence: raw.encryptedEvidence === null ? null : bodyRecord(raw.encryptedEvidence) }; +} diff --git a/server/src/services/chat-channels/slack/form-signature-refresh.ts b/server/src/services/chat-channels/slack/form-signature-refresh.ts new file mode 100644 index 00000000000..5c47a4936cf --- /dev/null +++ b/server/src/services/chat-channels/slack/form-signature-refresh.ts @@ -0,0 +1,29 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, sql } from "drizzle-orm"; +import { chatActions, type Db } from "@paperclipai/db"; +import { notifyDeliveryWork, DELIVERY_QUEUES } from "../../delivery-work-notifications.js"; +import { SlackCloudReceiptError } from "./cloud-event.js"; + +type Row = typeof chatActions.$inferSelect; +export async function replaceSlackFormReceiptSignature(db: Db, row: Row, field: "retirementReceiptJws" | "formAck", compactJws: string): Promise { + const old = field === "retirementReceiptJws" ? row.result?.retirementReceiptJws + : row.result?.formAck && typeof row.result.formAck === "object" ? Reflect.get(row.result.formAck, "receiptJws") : undefined; + if (typeof old !== "string" || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(compactJws) || Buffer.byteLength(compactJws, "ascii") > 12000) + throw new SlackCloudReceiptError("form_signature_refresh_invalid", "attention"); + if (old === compactJws) return row; + return db.transaction(async tx => { + const [current] = await tx.select().from(chatActions).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), eq(chatActions.endpointId, row.endpointId))).for("update"); + if (!current || current.status !== row.status || current.kind !== row.kind || current.providerActionId !== row.providerActionId + || !isDeepStrictEqual(current.payload, row.payload) || !isDeepStrictEqual(current.result, row.result)) + throw new SlackCloudReceiptError("form_signature_refresh_changed", "retry"); + await notifyDeliveryWork(tx, DELIVERY_QUEUES.connectionEvents); + const result = field === "retirementReceiptJws" + ? sql`jsonb_set(${chatActions.result}, '{retirementReceiptJws}', ${JSON.stringify(compactJws)}::jsonb, false)` + : sql`jsonb_set(${chatActions.result}, '{formAck,receiptJws}', ${JSON.stringify(compactJws)}::jsonb, false)`; + const [updated] = await tx.update(chatActions).set({ result, updatedAt: new Date() }).where(and(eq(chatActions.id, row.id), eq(chatActions.companyId, row.companyId), + eq(chatActions.endpointId, row.endpointId), eq(chatActions.status, row.status), eq(chatActions.kind, row.kind), eq(chatActions.providerActionId, row.providerActionId), + sql`${chatActions.payload} = ${JSON.stringify(row.payload)}::jsonb`, sql`${chatActions.result} = ${JSON.stringify(row.result)}::jsonb`)).returning(); + if (!updated) throw new SlackCloudReceiptError("form_signature_refresh_changed", "retry"); + return updated; + }); +} diff --git a/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts b/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts new file mode 100644 index 00000000000..ef7f9e3f13c --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-history-owner-proof.test.ts @@ -0,0 +1,30 @@ +import { expect, it } from "vitest"; +import { PgDialect } from "drizzle-orm/pg-core"; +import { idleChatHistoryChecks, idleChatHistoryChecksForOwner } from "../idle-history-proof.js"; +it("absence retains the exact existing chat history query objects", () => { + expect(idleChatHistoryChecksForOwner()).toBe(idleChatHistoryChecks); + expect(idleChatHistoryChecksForOwner([])).toBe(idleChatHistoryChecks); +}); +it("the same validated bound action identity appears in action and delivery queries with normal graph checks", () => { + const id = "11111111-1111-4111-8111-111111111111", checks = idleChatHistoryChecksForOwner([id, id]); + for (const reason of ["chat_actions", "chat_deliveries"]) { + const check = checks.find(check => check.reason === reason); if (!check || typeof check.query === "string") throw new Error("Parameterized owner query missing"); + const rendered = new PgDialect().sqlToQuery(check.query); + expect(rendered.params).toEqual([[id]]); + expect(rendered.sql).not.toContain(id); + expect(rendered.sql).toContain("e.company_id = a.company_id"); + expect(rendered.sql).toContain("d.company_id = a.company_id"); + expect(rendered.sql).toContain("unnest($1::uuid[])"); + } + expect(() => idleChatHistoryChecksForOwner(["foreign' OR true"])).toThrow(); + +}); + +it("retained completion identity counts do not become SQL literal or parameter ceilings",()=>{ + const ids=Array.from({length:65536},(_,index)=>`11111111-1111-4111-8111-${String(index).padStart(12,"0")}`); + const check=idleChatHistoryChecksForOwner(ids).find(value=>value.reason==="chat_actions"); + if(!check||typeof check.query==="string")throw new Error("Parameterized owner query absent"); + const rendered=new PgDialect().sqlToQuery(check.query); + expect(rendered.params).toEqual([ids]);expect(rendered.sql).not.toContain(ids[0]); + expect(rendered.sql).toContain("unnest($1::uuid[])"); +}); diff --git a/server/src/services/chat-channels/slack/idle-incoming.test.ts b/server/src/services/chat-channels/slack/idle-incoming.test.ts new file mode 100644 index 00000000000..777ce540fde --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-incoming.test.ts @@ -0,0 +1,64 @@ +import { slackFormRetirementReceipt } from "./ack-retirement-receipt.js"; +import { generateKeyPairSync, randomUUID, sign } from "node:crypto"; +import { expect, it } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { CAPABILITIES } from "../provider-catalog.js"; +import { verifyCloudControlAssertion, CLOUD_CONTROL_AUDIENCE, CLOUD_CONTROL_JWS_TYPE } from "../../cloud-runtime-identity.js"; +import type { IdleCapabilities, VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { parseSlackFormBinding } from "./cloud-forms.js"; +import { slackConfiguredBindingsFingerprint } from "./configured-coverage.js"; +import { readSlackIdleIncomingProof, projectSlackIdleIncomingRows, type SlackIdleEndpoint, type SlackIdleConnection } from "./idle-incoming.js"; +import type { SlackFormHistoryRow } from "./form-idle-history.js"; +const binding = parseSlackFormBinding({ version: 1, instanceId: "instance", companyId: "11111111-1111-4111-8111-111111111111", endpointId: "22222222-2222-4222-8222-222222222222", publicId: "p".repeat(43), connectionId: "33333333-3333-4333-8333-333333333333", runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "a".repeat(64), credentialFingerprint: "b".repeat(64) }); +function verifiedPartial(): VerifiedIdleCapabilities { + const key = generateKeyPairSync("ed25519"), kid = "slack-idle-unit", now = Math.floor(Date.now() / 1000), generation = "c".repeat(64); + const caps: IdleCapabilities = { incomingWake: { version: 1, ownerId: randomUUID(), generation, routes: ["slack-broker-events-v1"], + slackBroker: { version: 1, instanceId: binding.instanceId, generation, bindingsFingerprint: slackConfiguredBindingsFingerprint([binding]), entries: [{ binding, lanes: ["events", "forms", "modal_close", "modal_submit"] }] } } }; + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const body = encode({ alg: "EdDSA", typ: CLOUD_CONTROL_JWS_TYPE, kid }) + "." + encode({ v: 1, iss: "paperclip-cloud", aud: CLOUD_CONTROL_AUDIENCE, + sub: binding.instanceId, action: "task-drain:read", requestId: randomUUID(), iat: now, exp: now + 60, idleCapabilities: caps }); + const claims = verifyCloudControlAssertion({ compactJws: body + "." + sign(null, Buffer.from(body), key.privateKey).toString("base64url"), expectedAction: "task-drain:read", + env: { PAPERCLIP_CLOUD_STACK_ID: binding.instanceId, PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...key.publicKey.export({ format: "jwk" }), kid, use: "sig", alg: "EdDSA" }] }) } }); + if (!claims.idleCapabilities) throw new Error("Actual signed idle capability absent"); + return claims.idleCapabilities as VerifiedIdleCapabilities; +} +function rows() { + const { credentialFingerprint: _, eventBindingId, ...base } = binding, enrollment = { ...base, bindingId: eventBindingId }; + const endpoint: SlackIdleEndpoint = { id: binding.endpointId, companyId: binding.companyId, connectionId: binding.connectionId, publicId: binding.publicId, + provider: "slack", status: "active", capabilities: CAPABILITIES.slack, setup: { step: "complete", runtimeGeneration: 1, slackCloudInstanceId: binding.instanceId, + slackCloudSubject: "actor", slackCloudForms: binding, slackCloudEvents: { runtimeGeneration: 1, bindingId: binding.eventBindingId, teamId: binding.teamId, appId: binding.appId } } }; + const connection: SlackIdleConnection = { id: binding.connectionId, companyId: binding.companyId, status: "active", enabled: true, transport: "chat_sdk", connectionPurpose: "channel" }; + const owner: SlackFormHistoryRow = { id: "44444444-4444-4444-8444-444444444444", companyId: binding.companyId, endpointId: binding.endpointId, kind: "slack_form_owner", principalId: null, conversationId: null, + status: "issued", providerActionId: "slack-form-owner:" + "d".repeat(64), payload: { enrollment, binding, subject: "actor", environment: "test" }, result: { phase: "enrolled", nextAttemptAt: null } }; + return { endpoints: [{ endpoint, connection }], actions: [owner] }; +} +it("absent signed Slack proof performs zero query and budget operations", async () => { + let calls = 0; + const tx: Pick = { select: () => { calls++; throw new Error("No absent-feature query permitted"); } }; + expect(await readSlackIdleIncomingProof(tx, { receiverBound: true, beforeRead: async () => { calls++; }, remaining: () => { calls++; return 1; }, registeredOptions: () => false })).toBeUndefined(); + expect(calls).toBe(0); + expect(await readSlackIdleIncomingProof(tx, { capabilities: verifiedPartial(), identity: { instanceId: binding.instanceId, environment: "test" }, receiverBound: false, + beforeRead: async () => { calls++; }, remaining: () => { calls++; return 1; }, registeredOptions: () => false })).toBeUndefined(); + expect(calls).toBe(0); +}); +it("genuinely signed forms-events-only coverage keeps default commands and general actions blocked", () => { + const result = projectSlackIdleIncomingRows({ ...rows(), capabilities: verifiedPartial(), identity: { instanceId: binding.instanceId, environment: "test" }, registeredOptions: () => false }); + expect(result).toEqual({ endpointIds: [], connectionIds: [], completedActionIds: [] }); +}); +it("foreign company and duplicate current owners never become current Slack completion authority", () => { + const current = rows(), capabilities = verifiedPartial(), common = { capabilities, identity: { instanceId: binding.instanceId, environment: "test" as const }, registeredOptions: () => false }; + expect(projectSlackIdleIncomingRows({ ...current, ...common, endpoints: [{ ...current.endpoints[0]!, connection: { ...current.endpoints[0]!.connection, companyId: "foreign" } }] })).toBeUndefined(); + expect(projectSlackIdleIncomingRows({ ...current, ...common, actions: Array.from({ length: 1001 }, () => current.actions[0]!) })).toBeUndefined(); +}); + +it("retained completed receipts exceed one history page without enlarging the current endpoint cap", () => { + const current=rows(),receipt=slackFormRetirementReceipt(binding,"test","actor"); + const history:SlackFormHistoryRow[]=Array.from({length:1005},()=>({id:randomUUID(),companyId:binding.companyId,endpointId:binding.endpointId, + kind:"slack_form_grant",principalId:null,conversationId:null,status:"cancelled",providerActionId:"slack-form-grant:"+randomUUID(), + payload:{binding,subject:"actor",environment:"test"},result:{phase:"transferred",nextAttemptAt:null, + retirement:{version:1,binding,instanceId:binding.instanceId,environment:"test",subject:"actor",receiptDigest:receipt.receiptId,receipt}}})); + const common={capabilities:verifiedPartial(),identity:{instanceId:binding.instanceId,environment:"test" as const},registeredOptions:()=>false}; + const result=projectSlackIdleIncomingRows({...current,...common,actions:[...current.actions,...history]}); + expect(result?.completedActionIds).toEqual(history.map(row=>row.id).sort()); + expect(projectSlackIdleIncomingRows({...current,...common,endpoints:Array.from({length:1001},()=>current.endpoints[0]!)})).toBeUndefined(); +}); diff --git a/server/src/services/chat-channels/slack/idle-incoming.ts b/server/src/services/chat-channels/slack/idle-incoming.ts new file mode 100644 index 00000000000..262416c7e1d --- /dev/null +++ b/server/src/services/chat-channels/slack/idle-incoming.ts @@ -0,0 +1,175 @@ +import { isDeepStrictEqual } from "node:util"; +import { and, eq, gt, inArray, sql } from "drizzle-orm"; +import { alias } from "drizzle-orm/pg-core"; +import { chatActions, chatEndpoints, toolConnections, type Db } from "@paperclipai/db"; +import type { VerifiedIdleCapabilities } from "../../idle-incoming-wake-capability.js"; +import { parseSlackFormBinding, parseSlackFormPublicEnrollment, slackFormSetupRecord, sameSlackFormBinding } from "./cloud-forms.js"; +import { matchSlackConfiguredCoverage, type SlackConfiguredRequirement } from "./configured-coverage.js"; +import { createSlackFormHistoryClassifier, type SlackFormHistoryRow } from "./form-idle-history.js"; +import type { SlackFormConnectorIdentity } from "./cloud-consumer.js"; + +export type SlackIdleEndpoint = Pick & { setup: unknown; capabilities: unknown }; +export type SlackIdleConnection = Pick; +export type SlackIdleIncomingProjection = Readonly<{ endpointIds: readonly string[]; connectionIds: readonly string[]; completedActionIds: readonly string[] }>; +export const slackHasRegisteredOptionsAtCurrentBuild = (_endpointId: string): boolean => false; +function object(value: unknown): Record { + if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid Slack idle object"); + return Object.fromEntries(Object.entries(value)); +} +function flag(value: Record, key: string): boolean { + const v = value[key]; if (typeof v !== "boolean") throw new Error("Unknown Slack configured capability"); return v; +} +type SlackIncomingRowsInput = { + endpoints: readonly { endpoint: SlackIdleEndpoint; connection: SlackIdleConnection }[]; actions: readonly SlackFormHistoryRow[]; + capabilities: VerifiedIdleCapabilities; identity: SlackFormConnectorIdentity; registeredOptions(endpointId: string): boolean; check?(): void; +}; +function slackIncomingContext(input: SlackIncomingRowsInput) { + if (input.endpoints.length > 1000 || !input.capabilities.incomingWake?.routes.includes("slack-broker-events-v1")) return; + const requirements: SlackConfiguredRequirement[] = [], endpointIds = new Set(), currentOwners = new Map(); + const groups = groupHistory(input.actions); + try { + for (const { endpoint, connection } of input.endpoints) { + input.check?.(); + if (endpoint.provider !== "slack" || endpoint.companyId !== connection.companyId || endpoint.connectionId !== connection.id || connection.transport !== "chat_sdk" || connection.connectionPurpose !== "channel" || endpointIds.has(endpoint.id)) return; + endpointIds.add(endpoint.id); + if (endpoint.status !== "active" || connection.status !== "active" || !connection.enabled) continue; + const setup = slackFormSetupRecord(endpoint.setup), binding = parseSlackFormBinding(setup.slackCloudForms); + const capabilities = object(endpoint.capabilities), registeredOptions = input.registeredOptions(endpoint.id); + if (typeof registeredOptions !== "boolean" || setup.step !== "complete" || setup.slackCloudInstanceId !== binding.instanceId || setup.runtimeGeneration !== binding.runtimeGeneration + || binding.instanceId !== input.identity.instanceId || binding.companyId !== endpoint.companyId || binding.endpointId !== endpoint.id + || binding.connectionId !== connection.id || binding.publicId !== endpoint.publicId || typeof setup.slackCloudSubject !== "string") return; + const descriptor = object(setup.slackCloudEvents), enrollment = parseSlackFormPublicEnrollment({ version: 1, instanceId: input.identity.instanceId, + companyId: endpoint.companyId, endpointId: endpoint.id, publicId: endpoint.publicId, connectionId: endpoint.connectionId, + runtimeGeneration: descriptor.runtimeGeneration, bindingId: descriptor.bindingId, teamId: descriptor.teamId, appId: descriptor.appId }); + const { credentialFingerprint: _, eventBindingId, ...base } = binding; + if (!isDeepStrictEqual({ ...base, bindingId: eventBindingId }, enrollment)) return; + const owners = (groups.get(`${endpoint.companyId}:${endpoint.id}`) ?? []).filter(row => row.companyId === endpoint.companyId && row.endpointId === endpoint.id && row.kind === "slack_form_owner" + && /^slack-form-owner:[a-f0-9]{64}$/.test(row.providerActionId) && row.result?.phase === "enrolled" && row.result.nextAttemptAt === null && row.result.desiredActive !== false && ["issued", "processed"].includes(row.status) + && row.payload.environment === input.identity.environment && row.payload.subject === setup.slackCloudSubject + && isDeepStrictEqual(row.payload.binding, binding) && isDeepStrictEqual(row.payload.enrollment, enrollment)); + if (owners.length !== 1) return; + currentOwners.set(owners[0]!.id, owners[0]!); + requirements.push({ binding, capabilities: { actions: flag(capabilities, "actions"), modals: flag(capabilities, "modals"), slashCommands: flag(capabilities, "slashCommands") }, registeredOptions }); + } + const expected = { ...input.identity, generation: input.capabilities.incomingWake.generation }; + const matched = matchSlackConfiguredCoverage(requirements, input.capabilities, expected); + return { requirements, expected, matched, currentOwners }; + } catch { return; } +} + +function groupHistory(rows: readonly SlackFormHistoryRow[]) { + const groups = new Map(); + for (const row of rows) { + const key = `${row.companyId}:${row.endpointId}`, group = groups.get(key); + if (group) group.push(row); else groups.set(key, [row]); + } + return groups; +} +function projection(context: NonNullable>, completed: Set): SlackIdleIncomingProjection { + return Object.freeze({ endpointIds: Object.freeze([...(context.matched?.endpointIds ?? [])]), + connectionIds: Object.freeze([...(context.matched?.connectionIds ?? [])]), completedActionIds: Object.freeze([...completed].sort()) }); +} +export function projectSlackIdleIncomingRows(input: SlackIncomingRowsInput): SlackIdleIncomingProjection | undefined { + try { + const context = slackIncomingContext(input); + if (!context) return; + const classify = createSlackFormHistoryClassifier({ requirements: context.requirements, verified: input.capabilities, expected: context.expected, currentOwners: context.currentOwners }); + const groups = groupHistory(input.actions), completed = new Set(); + for (const { endpoint } of input.endpoints) { + input.check?.(); + for (const id of classify(groups.get(`${endpoint.companyId}:${endpoint.id}`) ?? [], endpoint.companyId, endpoint.id, input.check)) { + if (!uuid.test(id)) return; completed.add(id); + } + } + return projection(context, completed); + } catch { return; } +} +const uuid = /^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$/i; +const pageSize = 1000; +const historyOpen = alias(chatActions, "slack_idle_history_open"), historySubmit = alias(chatActions, "slack_idle_history_submit"); +const relatedId = (key: "openActionRowId" | "submitActionRowId") => sql`CASE + WHEN ${chatActions.payload}->>${key} ~* '^[a-f0-9]{8}-(?:[a-f0-9]{4}-){3}[a-f0-9]{12}$' + THEN (${chatActions.payload}->>${key})::uuid ELSE NULL END`; + +export async function readSlackIdleIncomingProof(tx: Pick, input: { + capabilities?: VerifiedIdleCapabilities; identity?: SlackFormConnectorIdentity; receiverBound: boolean; + beforeRead(): Promise; remaining(): number; registeredOptions(endpointId: string): boolean; +}): Promise { + if (!input.capabilities?.incomingWake?.routes.includes("slack-broker-events-v1") || !input.identity || !input.receiverBound) return; + const broker = input.capabilities.incomingWake.slackBroker; + if (!broker || broker.instanceId !== input.identity.instanceId || broker.generation !== input.capabilities.incomingWake.generation) return; + await input.beforeRead(); + const endpoints = await tx.select({ endpoint: chatEndpoints, connection: toolConnections }).from(chatEndpoints).innerJoin(toolConnections, + and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, chatEndpoints.companyId))) + .where(and(eq(chatEndpoints.provider, "slack"), eq(toolConnections.transport, "chat_sdk"), eq(toolConnections.connectionPurpose, "channel"), + sql`exists (select 1 from companies where id=${chatEndpoints.companyId} and status <> 'archived')`)).orderBy(chatEndpoints.id).limit(1001); + input.remaining(); if (endpoints.length > 1000) return; + if (!endpoints.length && broker.entries.length) return; + if (!endpoints.length) return Object.freeze({ endpointIds: [], connectionIds: [], completedActionIds: [] }); + // Retain only bounded current-owner candidates. Old enrolled generations do + // not become current coverage, and duplicate exact owners on ANY page fail. + const endpointsById = new Map(endpoints.map(value => [value.endpoint.id, value])); + const currentOwners = new Map(); + let ownerCursor: string | undefined; + for (;;) { + await input.beforeRead(); + const page = await tx.select({ action: chatActions }).from(chatActions).innerJoin(chatEndpoints, + and(eq(chatEndpoints.id, chatActions.endpointId), eq(chatEndpoints.companyId, chatActions.companyId))) + .where(and(eq(chatEndpoints.provider, "slack"), inArray(chatActions.endpointId, endpoints.map(value => value.endpoint.id)), + eq(chatActions.kind, "slack_form_owner"), sql`${chatActions.result}->>'phase'='enrolled'`, + ownerCursor ? gt(chatActions.id, ownerCursor) : undefined)) + .orderBy(chatActions.id).limit(pageSize); + input.remaining(); + for (const { action } of page) { + input.remaining(); + const scope = endpointsById.get(action.endpointId); + if (!scope || action.companyId !== scope.endpoint.companyId || scope.endpoint.status !== "active" + || scope.connection.status !== "active" || !scope.connection.enabled) continue; + try { + const setup = slackFormSetupRecord(scope.endpoint.setup), binding = parseSlackFormBinding(setup.slackCloudForms); + if (!/^slack-form-owner:[a-f0-9]{64}$/.test(action.providerActionId) || !["issued", "processed"].includes(action.status) + || action.result?.nextAttemptAt !== null || action.result.desiredActive === false || action.payload.environment !== input.identity.environment + || action.payload.subject !== setup.slackCloudSubject || !sameSlackFormBinding(parseSlackFormBinding(action.payload.binding), binding)) continue; + const enrollment = parseSlackFormPublicEnrollment(action.payload.enrollment); + const { credentialFingerprint: _, eventBindingId, ...base } = binding; + if (!isDeepStrictEqual({ ...base, bindingId: eventBindingId }, enrollment)) continue; + const owners = currentOwners.get(action.endpointId) ?? []; + owners.push(action); if (owners.length > 1) return; + currentOwners.set(action.endpointId, owners); + } catch { continue; } + } + if (page.length < pageSize) break; + ownerCursor = page.at(-1)!.action.id; + } + const common = { endpoints, capabilities: input.capabilities, identity: input.identity, + registeredOptions: input.registeredOptions, check: () => { input.remaining(); } }; + const context = slackIncomingContext({ ...common, actions: [...currentOwners.values()].flat() }); + if (!context) return; + const classify = createSlackFormHistoryClassifier({ requirements: context.requirements, verified: input.capabilities, expected: context.expected, currentOwners: context.currentOwners }); + const completed = new Set(); let cursor: string | undefined; + for (;;) { + await input.beforeRead(); + // Indexed primary-key joins supply only the two referenced actions. A + // session/page boundary never requires an all-history JSON accumulator or + // one extra read per historical session. + const page = await tx.select({ action: chatActions, open: historyOpen, submit: historySubmit }).from(chatActions) + .innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatActions.endpointId), eq(chatEndpoints.companyId, chatActions.companyId))) + .leftJoin(historyOpen, and(eq(chatActions.kind, "slack_form_session"), eq(historyOpen.kind, "question_form_open"), eq(historyOpen.id, relatedId("openActionRowId")), eq(historyOpen.companyId, chatActions.companyId), eq(historyOpen.endpointId, chatActions.endpointId))) + .leftJoin(historySubmit, and(eq(chatActions.kind, "slack_form_session"), eq(historySubmit.kind, "question_form_submit"), eq(historySubmit.id, relatedId("submitActionRowId")), eq(historySubmit.companyId, chatActions.companyId), eq(historySubmit.endpointId, chatActions.endpointId))) + .where(and(eq(chatEndpoints.provider, "slack"), inArray(chatActions.endpointId, endpoints.map(value => value.endpoint.id)), + inArray(chatActions.kind, ["slack_form_owner", "slack_form_grant", "slack_form_session", "question_form_open", "question_form_submit"]), + cursor ? gt(chatActions.id, cursor) : undefined)).orderBy(chatActions.id).limit(pageSize); + input.remaining(); + const groups = groupHistory(page.flatMap(row => [row.action, ...(row.open ? [row.open] : []), ...(row.submit ? [row.submit] : [])])); + for (const rows of groups.values()) { + input.remaining(); const first = rows[0]!; + for (const id of classify(rows, first.companyId, first.endpointId, () => { input.remaining(); })) { + if (!uuid.test(id)) return; completed.add(id); + } + } + if (page.length < pageSize) break; + cursor = page.at(-1)!.action.id; + } + input.remaining(); + return projection(context, completed); +} diff --git a/server/src/services/chat-channels/slack/receipt-authority.ts b/server/src/services/chat-channels/slack/receipt-authority.ts new file mode 100644 index 00000000000..52e1c4db625 --- /dev/null +++ b/server/src/services/chat-channels/slack/receipt-authority.ts @@ -0,0 +1,11 @@ +import { verifyCloudConnectorReceipt } from "../../cloud-runtime-identity.js"; +import { parseSlackFormFinalizedReceipt, type SlackFormFinalizedReceipt } from "./ack-receipt.js"; +import { parseSlackFormRetirementReceipt, type SlackFormRetirementReceipt } from "./ack-retirement-receipt.js"; + +type ReceiptVerificationInput = { compactJws: string; env?: NodeJS.ProcessEnv }; +export function verifySlackFormFinalizedReceipt(input: ReceiptVerificationInput): SlackFormFinalizedReceipt { + return verifyCloudConnectorReceipt(input, parseSlackFormFinalizedReceipt); +} +export function verifySlackFormRetirementReceipt(input: ReceiptVerificationInput): SlackFormRetirementReceipt { + return verifyCloudConnectorReceipt(input, parseSlackFormRetirementReceipt); +} diff --git a/server/src/services/chat-publication-errors.test.ts b/server/src/services/chat-publication-errors.test.ts index a2337931264..56abb236e21 100644 --- a/server/src/services/chat-publication-errors.test.ts +++ b/server/src/services/chat-publication-errors.test.ts @@ -475,4 +475,17 @@ describe("chat publication error classification", () => { }); }, ); + it("retries an unknown preparation failure only with a local before-invocation fact", () => { + const error = new Error("form_publication_enrollment_pending"); + expect(classifyChatPublicationError(error, 1, "not_started")).toEqual({ + kind: "retry", retryAfterMs: 2000, reason: error.message, + }); + expect(classifyChatPublicationError(error, 1)).toEqual({ kind: "delivery_unknown", reason: error.message }); + expect(classifyChatPublicationError(error, 1, "possible")).toEqual({ kind: "delivery_unknown", reason: error.message }); + }); + it("preserves definite provider dispositions with before-invocation context", () => { + expect(classifyChatPublicationError(Object.assign(new Error("rejected"), { status: 400 }), 1, "not_started")).toMatchObject({ kind: "failed" }); + expect(classifyChatPublicationError(Object.assign(new Error("slow down"), { status: 429 }), 1, "not_started")).toMatchObject({ kind: "retry", providerRateLimit: true }); + }); + }); diff --git a/server/src/services/chat-publication-errors.ts b/server/src/services/chat-publication-errors.ts index 24847917fa0..04f4c1c23d8 100644 --- a/server/src/services/chat-publication-errors.ts +++ b/server/src/services/chat-publication-errors.ts @@ -145,6 +145,7 @@ function structuredProviderErrors(error: unknown): StructuredProviderError[] { export function classifyChatPublicationError( error: unknown, attempt: number, + providerInvocation: "possible" | "not_started" = "possible", ): ChatPublicationErrorDisposition { const values = structuredProviderErrors(error); const firstNumber = (select: (value: StructuredProviderError) => unknown) => @@ -399,5 +400,8 @@ export function classifyChatPublicationError( // Adapter NetworkError and ordinary fetch/transport errors are ambiguous: // the request may have reached the provider even when no response reached // Paperclip. An operator must inspect the native conversation before replay. + if (providerInvocation === "not_started") { + return { kind: "retry", retryAfterMs: Math.min(60_000, 2 ** Math.max(0, attempt) * 1_000), reason }; + } return { kind: "delivery_unknown", reason }; } diff --git a/server/src/services/cloud-runtime-identity.ts b/server/src/services/cloud-runtime-identity.ts index b1f37dbea87..8d2280a599a 100644 --- a/server/src/services/cloud-runtime-identity.ts +++ b/server/src/services/cloud-runtime-identity.ts @@ -1,5 +1,6 @@ +import { DEFAULT_JSON_BODY_LIMIT_BYTES } from "../http/body-limits.js"; import { parseIdleCapabilities, type IdleCapabilities } from "./idle-incoming-wake-capability.js"; -import { createPublicKey, timingSafeEqual, verify, type JsonWebKey } from "node:crypto"; +import { createHash, createPublicKey, timingSafeEqual, verify, type JsonWebKey } from "node:crypto"; import { eq } from "drizzle-orm"; import type { Db } from "@paperclipai/db"; import { instanceSettings } from "@paperclipai/db"; @@ -449,6 +450,7 @@ export const CLOUD_CONTROL_JWS_TYPE = "paperclip-cloud-control+jwt"; export const CLOUD_CONTROL_ACTIONS = [ "ephemeral-enrollment:prepare", "task-drain:read", + "task-drain:read-body", "task-drain:start", "task-drain:stop", "lifecycle:read", @@ -496,6 +498,7 @@ export type CloudControlClaims = { action: CloudControlAction; requestId: string; idleCapabilities?: IdleCapabilities; + bodySha256?: string; iat: number; exp: number; }; @@ -509,6 +512,7 @@ export type CloudControlClaims = { */ export function verifyCloudControlAssertion(input: { compactJws: string; + rawBody?: Uint8Array; expectedAction: CloudControlAction; env?: NodeJS.ProcessEnv; now?: Date; @@ -577,9 +581,27 @@ export function verifyCloudControlAssertion(input: { ) { throw new Error("Cloud control assertion is expired or has an invalid lifetime"); } - if (payload.idleCapabilities !== undefined) { - if (payload.action !== "task-drain:read") throw new Error("Idle capabilities require task-drain:read"); - payload.idleCapabilities = parseIdleCapabilities(payload.idleCapabilities); + if (payload.action === "task-drain:read-body") { + if (payload.idleCapabilities !== undefined || typeof payload.bodySha256 !== "string" || !/^[a-f0-9]{64}$/.test(payload.bodySha256) + || !input.rawBody || input.rawBody.byteLength === 0 || input.rawBody.byteLength > DEFAULT_JSON_BODY_LIMIT_BYTES) { + throw new Error("Invalid Cloud idle status body binding"); + } + const raw = Buffer.from(input.rawBody); + if (createHash("sha256").update(raw).digest("hex") !== payload.bodySha256) throw new Error("Cloud idle status body digest changed"); + const text = new TextDecoder("utf-8", { fatal: true }).decode(raw); + let body: unknown; + try { body = JSON.parse(text); } + catch { throw new Error("Invalid Cloud idle status body JSON"); } + if (!body || typeof body !== "object" || Array.isArray(body) || Object.keys(body).join() !== "idleCapabilities") { + throw new Error("Invalid Cloud idle status body"); + } + payload.idleCapabilities = parseIdleCapabilities(Reflect.get(body, "idleCapabilities")); + } else { + if (payload.bodySha256 !== undefined) throw new Error("Cloud body binding requires task-drain:read-body"); + if (payload.idleCapabilities !== undefined) { + if (payload.action !== "task-drain:read") throw new Error("Idle capabilities require task-drain:read"); + payload.idleCapabilities = parseIdleCapabilities(payload.idleCapabilities); + } } // Consumed LAST, only after every other check passed: a rejected // assertion must not burn its request id, or an attacker could deny a @@ -589,3 +611,24 @@ export function verifyCloudControlAssertion(input: { } return payload as CloudControlClaims; } + +export function verifyCloudConnectorReceipt(input: { compactJws: string; env?: NodeJS.ProcessEnv }, parseReceipt: (value: unknown) => T): T { + const env = input.env ?? process.env; + if (Buffer.byteLength(input.compactJws, "ascii") > 12_000 || !/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(input.compactJws)) + throw new Error("Invalid connector ACK receipt encoding"); + const [headerPart, payloadPart, signaturePart] = input.compactJws.split("."); + const header = decodeJsonPart(headerPart, "connector receipt header"); + if (Object.keys(header).sort().join() !== "alg,kid,typ" || header.alg !== "EdDSA" + || header.typ !== "paperclip-connector-ack-receipt+jwt" || typeof header.kid !== "string" || !header.kid) + throw new Error("Invalid connector ACK receipt header"); + const key = cloudRuntimePublicKeyForKid(env, header.kid); + if (!verify(null, Buffer.from(`${headerPart}.${payloadPart}`, "ascii"), key, Buffer.from(signaturePart, "base64url"))) + throw new Error("Invalid connector ACK receipt signature"); + const claims = decodeJsonPart(payloadPart, "connector receipt claims"); + if (Object.keys(claims).sort().join() !== "aud,iss,receipt,sub,v" || claims.v !== 1 || claims.iss !== CLOUD_RUNTIME_IDENTITY_ISSUER + || claims.aud !== "paperclip-connector-ack-receipt/v1" || claims.sub !== nonEmpty(env.PAPERCLIP_CLOUD_STACK_ID)) + throw new Error("Invalid connector ACK receipt claims"); + const receipt = parseReceipt(claims.receipt); + if (receipt.binding.instanceId !== claims.sub) throw new Error("Connector ACK receipt instance changed"); + return receipt; +} diff --git a/server/src/services/github-connection-events.ts b/server/src/services/github-connection-events.ts index e92c27d9953..e095c52c1a0 100644 --- a/server/src/services/github-connection-events.ts +++ b/server/src/services/github-connection-events.ts @@ -1,3 +1,7 @@ +import { parseSlackFormOperation } from "./chat-channels/slack/cloud-forms.js"; +import { acknowledgeSlackFormDeliveries } from "./chat-channels/slack/form-normal-ack.js"; +import { readSlackFormRetainedConsumer } from "./chat-channels/slack/cloud-consumer.js"; +import { reconcileRegisteredSlackFormWork } from "./chat-channels/slack/form-owner.js"; import { SlackCloudReceiptError } from "./chat-channels/slack/cloud-event.js"; import { reconcilePendingSlackCloudEnrollments } from "./chat-channels/slack/cloud-enrollment.js"; import { notifyExternalObjectRefresh } from "./external-object-refresh-notifications.js"; @@ -522,6 +526,11 @@ export function githubConnectionEventService( logger.warn({ reason: "enrollment_reconcile_unavailable" }, "Slack enrollment responsibility retained"); } } + let formsAt: number | null = null; + try { formsAt = await reconcileRegisteredSlackFormWork(db, now()); } + catch { formsAt = now().getTime() + 5_000; } + const next = Math.min(nextEnrollmentAt ?? Number.POSITIVE_INFINITY, formsAt ?? Number.POSITIVE_INFINITY); + nextEnrollmentAt = Number.isFinite(next) ? next : null; } } @@ -531,7 +540,25 @@ export function githubConnectionEventService( const slackBots = await db.select().from(chatEndpoints).where(and(eq(chatEndpoints.provider, "slack"), sql`${chatEndpoints.setup}->'slackCloudEvents' is not null`, sql`${chatEndpoints.status} in ('active','verifying','attention')`, sql`exists (select 1 from companies where id = ${chatEndpoints.companyId} and status <> 'archived')`)); const bots = [...githubBots, ...slackBots]; let identity: { subject: string; companyId: string } | null = bindings[0] ?? (bots[0] ? { subject: "github-bot-events", companyId: bots[0].companyId } : null); - const hasConsumer = bindings.length > 0 || bots.length > 0; + let hasConsumer = bindings.length > 0 || bots.length > 0; + if (!identity) { + try { + const retained = await readSlackFormRetainedConsumer(db, connector); + identity = retained.owner; + hasConsumer ||= retained.owner !== null; + const time = now().getTime(); + if (retained.attention.length && time >= nextEnrollmentDiagnosticAt) { + nextEnrollmentDiagnosticAt = time + 300_000; + logger.warn({ reason: "form_retained_owner_invalid", count: retained.attention.length }, "Slack form custody requires attention"); + } + } catch { + const time = now().getTime(); + if (time >= nextEnrollmentDiagnosticAt) { + nextEnrollmentDiagnosticAt = time + 300_000; + logger.warn({ reason: "form_retained_owner_unavailable" }, "Slack form custody requires attention"); + } + } + } if (!identity) { const [history] = await db.select({ grant: connectionGrants, connection: toolConnections }).from(connectionGrants) .innerJoin(toolConnections, and(eq(toolConnections.id, connectionGrants.connectionId), eq(toolConnections.companyId, connectionGrants.companyId))) @@ -567,9 +594,18 @@ export function githubConnectionEventService( failed: lease.failedEventIds?.length ?? 0, }; const acknowledge: string[] = []; + const formCompanies = new Map(); for (const event of lease.events) { if (event.provider === "slack") { - try { await dispatchSlackCloudEvent(db, event); acknowledge.push(event.id); result.processed += 1; } + try { + await dispatchSlackCloudEvent(db, event); + if (/^slack_form_[a-f0-9]{64}$/.test(event.id)) { + const packet = event.payload.slackFormOpen; + if (!packet || typeof packet !== "object" || Array.isArray(packet)) throw new SlackCloudReceiptError("form_ack_company_missing", "attention"); + formCompanies.set(event.id, parseSlackFormOperation(Reflect.get(packet, "operation")).binding.companyId); + } + acknowledge.push(event.id); result.processed += 1; + } catch (error) { const code = error instanceof SlackCloudReceiptError ? error.code : "owner_or_admission_unavailable"; const disposition = error instanceof SlackCloudReceiptError ? error.disposition : "retry"; @@ -610,12 +646,8 @@ export function githubConnectionEventService( } } if (acknowledge.length > 0) { - await connector.acknowledgeEvents({ - subject: first.subject, - companyId: first.companyId, - leaseId: lease.leaseId, - deliveryIds: acknowledge, - }); + await acknowledgeSlackFormDeliveries(db, { connector, + values: { subject: first.subject, companyId: first.companyId, leaseId: lease.leaseId, deliveryIds: acknowledge }, companies: formCompanies }); } for (const deliveryId of lease.failedEventIds ?? []) { logger.warn({ deliveryId, reason: "inner_event_authentication_failed" }, diff --git a/server/src/services/idle-incoming-wake-capability.ts b/server/src/services/idle-incoming-wake-capability.ts index a0f26fff61c..00c086adf36 100644 --- a/server/src/services/idle-incoming-wake-capability.ts +++ b/server/src/services/idle-incoming-wake-capability.ts @@ -1,12 +1,14 @@ +import { parseSlackConfiguredCoverage, type SlackConfiguredCoverage } from "./chat-channels/slack/configured-coverage.js"; /** Closed Cloud-owned ingress catalog. No credential material belongs in this proof. */ export interface IdleIncomingWakeCapability { version: 1; ownerId: string; generation: string; - routes: ("tenant-api-http-v2" | "github-broker-events-v1" | "mcp-oauth-http-v1")[]; + routes: ("tenant-api-http-v2" | "github-broker-events-v1" | "slack-broker-events-v1" | "mcp-oauth-http-v1")[]; apiKeyProof?: { version: 1; challenge: string }; mcpOAuthProof?: { version: 1; challenge: string }; githubBroker?: { instanceId: string; bindingsFingerprint: string }; + slackBroker?: SlackConfiguredCoverage; } export interface IdleCapabilities { incomingWake?: IdleIncomingWakeCapability; @@ -40,6 +42,7 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { "generation", "routes", "githubBroker", + "slackBroker", "apiKeyProof", "mcpOAuthProof", ]) || @@ -51,9 +54,9 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { !digest(v.generation) || !Array.isArray(v.routes) || v.routes.length < 1 || - v.routes.length > 3 || + v.routes.length > 4 || v.routes.some( - (r) => !["tenant-api-http-v2", "github-broker-events-v1", "mcp-oauth-http-v1"].includes(r), + (r) => !["tenant-api-http-v2", "github-broker-events-v1", "slack-broker-events-v1", "mcp-oauth-http-v1"].includes(r), ) || new Set(v.routes).size !== v.routes.length || [...v.routes].sort().join() !== v.routes.join() @@ -82,10 +85,18 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities { } else if (v.githubBroker !== undefined) throw new Error("Unexpected GitHub wake binding"); } + if (value.incomingWake && typeof value.incomingWake === "object" && !Array.isArray(value.incomingWake)) { + const incoming = Object.fromEntries(Object.entries(value.incomingWake)); + if (Array.isArray(incoming.routes) && incoming.routes.includes("slack-broker-events-v1")) { + const proof = parseSlackConfiguredCoverage(incoming.slackBroker); + if (proof.generation !== incoming.generation) throw new Error("Slack capability generation changed"); + } else if (incoming.slackBroker !== undefined) throw new Error("Unexpected Slack wake binding"); + } // Return a detached immutable snapshot; request callers cannot mutate a verified claim. const parsed = JSON.parse(JSON.stringify(value)) as IdleCapabilities; if (parsed.incomingWake) { Object.freeze(parsed.incomingWake.routes); + if (parsed.incomingWake.slackBroker) parsed.incomingWake.slackBroker = parseSlackConfiguredCoverage(parsed.incomingWake.slackBroker); if (parsed.incomingWake.apiKeyProof) Object.freeze(parsed.incomingWake.apiKeyProof); if (parsed.incomingWake.mcpOAuthProof) Object.freeze(parsed.incomingWake.mcpOAuthProof); if (parsed.incomingWake.githubBroker) diff --git a/server/src/services/idle-incoming-wake.ts b/server/src/services/idle-incoming-wake.ts index f611552c93c..977179ecb05 100644 --- a/server/src/services/idle-incoming-wake.ts +++ b/server/src/services/idle-incoming-wake.ts @@ -1,3 +1,5 @@ +import { idleChatHistoryChecksForOwner } from "./chat-channels/idle-history-proof.js"; +import { readSlackIdleIncomingProof, slackHasRegisteredOptionsAtCurrentBuild } from "./chat-channels/slack/idle-incoming.js"; import { readIdleMcpSleepProof, type IdleMcpSleepProof } from "./public-mcp/idle-sleep-proof.js"; import { idleMcpWorkQuery } from "./public-mcp/idle-work-query.js"; import { idleExternalAgentIncomingWorkQuery } from "../modules/external-agents/index.js"; @@ -22,6 +24,7 @@ import { hasConnectionEventWakeReceiver } from "./connection-event-wake.js"; export interface IdleIncomingWakeProof { capability: IdleIncomingWakeCapability; apiWakeAdmission?: ApiWakeAdmission; + completedActionIds?: readonly string[]; mcp?: IdleMcpSleepProof; connectionIds: string[]; endpointIds: string[]; @@ -102,6 +105,18 @@ export async function readIdleIncomingWakeProof( expiresAt: row.expiresAt === null ? null : new Date(row.expiresAt).toISOString() })), }); } + if (capability.routes.includes("slack-broker-events-v1")) { + const config = paperclipCloudConnectorConfigFromEnv(); + const slack = await readSlackIdleIncomingProof(tx, { capabilities: input.capabilities, + identity: config ? { instanceId: config.instanceId, environment: config.environment } : undefined, + receiverBound: hasConnectionEventWakeReceiver(input.receiverOwner), beforeRead, remaining, + registeredOptions: slackHasRegisteredOptionsAtCurrentBuild }); + if (slack) { + proof.endpointIds.push(...slack.endpointIds); + proof.connectionIds.push(...slack.connectionIds); + if (slack.completedActionIds.length) proof.completedActionIds = slack.completedActionIds; + } + } if (capability.routes.includes("mcp-oauth-http-v1")) { const stackId = process.env.PAPERCLIP_CLOUD_STACK_ID; if (!stackId || !capability.mcpOAuthProof) throw new Error("MCP wake challenge is missing"); @@ -306,6 +321,11 @@ export function incomingWakeWorkQuery( continuityConnectionIds.includes(id), ) ?? []), ]; + if ((reason === "chat_actions" || reason === "chat_deliveries") && proof?.completedActionIds?.length) { + const check = idleChatHistoryChecksForOwner(proof.completedActionIds).find(value => value.reason === reason); + if (!check) throw new Error("Chat history owner query missing"); + return typeof check.query === "string" ? sql.raw(check.query) : check.query; + } if (reason === "agent_api_keys") return proof?.apiWakeAdmission !== undefined ? sql`SELECT 1 WHERE false` diff --git a/server/src/services/paperclip-cloud-connector.test.ts b/server/src/services/paperclip-cloud-connector.test.ts index 4d7c594153b..e8e3767cd1f 100644 --- a/server/src/services/paperclip-cloud-connector.test.ts +++ b/server/src/services/paperclip-cloud-connector.test.ts @@ -5,6 +5,7 @@ import { generateKeyPairSync, hkdfSync, randomBytes, + sign, type KeyObject, } from "node:crypto"; import { SLACK_MANAGER_SCOPES } from "@paperclipai/shared"; @@ -22,6 +23,9 @@ import { type PaperclipCloudConnectorConfig, } from "./paperclip-cloud-connector.js"; +import { parseSlackFormFinalizedReceipt } from "./chat-channels/slack/ack-receipt.js"; +import { verifySlackFormFinalizedReceipt } from "./chat-channels/slack/receipt-authority.js"; + const instanceId = "inst_test"; const companyId = "company_test"; const subject = "user_test"; @@ -860,3 +864,56 @@ describe("sealed Slack event connector transport", () => { expect(JSON.parse(String(init.body))).toMatchObject({ enrollment: JSON.stringify(enrollment) }); }); }); + + +describe("Slack form normal connector terminal reply contracts", () => { + it("returns current signed refresh and GC-only replies through the actual signed connector", async () => { + const keys = config(), pair = generateKeyPairSync("ed25519"), kid = "connector-terminal-reply"; + const canonical = parseSlackFormFinalizedReceipt({ version: 1, kind: "slack_form_finalized", environment: keys.config.environment, + binding: { version: 1, instanceId, companyId, endpointId: "endpoint", publicId: "p".repeat(43), connectionId: "connection", + runtimeGeneration: 1, teamId: "TTEST", appId: "ATEST", eventBindingId: "slack-event:" + "b".repeat(64), credentialFingerprint: "c".repeat(64) }, + operationId: "a".repeat(64), deliveryId: "slack_form_" + "a".repeat(64), appReceiptId: "11111111-1111-4111-8111-111111111111", + grantId: "grant", publicationId: "publication", interactionId: "interaction", userId: "UTEST", viewDigest: null, + terminalDigest: "d".repeat(64), outcome: "no_effect", finalizedAt: "2026-11-07T00:00:00.000Z" }); + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + const unsigned = encode({ alg: "EdDSA", typ: "paperclip-connector-ack-receipt+jwt", kid }) + "." + + encode({ v: 1, iss: "paperclip-cloud", aud: "paperclip-connector-ack-receipt/v1", sub: instanceId, receipt: canonical }); + const compactJws = unsigned + "." + sign(null, Buffer.from(unsigned), pair.privateKey).toString("base64url"); + const operations: Record[] = []; + let reply: unknown = { ackReceiptJws: compactJws }; + const request = vi.fn(async (_url, init) => { + const body = JSON.parse(String(init?.body)); + const parts = String(body.request).split("."); expect(parts).toHaveLength(3); + const claims = JSON.parse(Buffer.from(parts[1]!, "base64url").toString("utf8")); + expect(claims.sub).toBe(subject); expect(claims.cid).toBe(companyId); + expect(claims.sh).toBe(createHash("sha256").update(body.operation).digest("base64url")); + operations.push(JSON.parse(body.operation)); + return Response.json({ slackForm: reply }); + }); + const connector = createPaperclipCloudConnector({ config: keys.config, request }); + const refresh = { kind: "receipt", binding: { instanceId }, operationId: canonical.operationId, refreshAckReceipt: true }; + await expect(connector.slackFormOperation({ subject, companyId, operation: refresh })).resolves.toEqual({ ackReceiptJws: compactJws }); + const digest = "d".repeat(64); + reply = { releasedAckDigest: digest }; + const release = { kind: "receipt", binding: { instanceId }, operationId: canonical.operationId, releaseAckFact: true }; + await expect(connector.slackFormOperation({ subject, companyId, operation: release })).resolves.toEqual({ releasedAckDigest: digest }); + expect(operations).toEqual([refresh, release]); expect(request).toHaveBeenCalledTimes(2); + expect(verifySlackFormFinalizedReceipt({ compactJws, env: { PAPERCLIP_CLOUD_STACK_ID: instanceId, + PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS: JSON.stringify({ keys: [{ ...pair.publicKey.export({ format: "jwk" }), kid, alg: "EdDSA", use: "sig" }] }) } })).toEqual(canonical); + }); + it("does not confuse refresh, release, acceptance or sealed operation responses", async () => { + const keys = config(); let reply: unknown; + const connector = createPaperclipCloudConnector({ config: keys.config, request: vi.fn(async () => Response.json({ slackForm: reply })) }); + const call = (operation: Record) => connector.slackFormOperation({ subject, companyId, operation: { kind: "receipt", ...operation } }); + for (const invalid of [{ releasedAckDigest: "d".repeat(64) }, { ackReceiptJws: "missing.signature" }, + { ackReceiptJws: "a.b.c", extra: true }, { acceptedReceiptDigest: "d".repeat(64), needsTerminalOriginal: false }, { sealed: {} }]) { + reply = invalid; await expect(call({ refreshAckReceipt: true })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + } + for (const invalid of [{ ackReceiptJws: "a.b.c" }, { releasedAckDigest: "invalid" }, { releasedAckDigest: "d".repeat(64), extra: true }]) { + reply = invalid; await expect(call({ releaseAckFact: true })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + } + reply = { releasedAckDigest: "d".repeat(64) }; + await expect(call({ acceptedAckReceipt: "a.b.c" })).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + await expect(call({})).rejects.toBeInstanceOf(PaperclipCloudConnectorError); + }); +}); diff --git a/server/src/services/paperclip-cloud-connector.ts b/server/src/services/paperclip-cloud-connector.ts index 368d8708f8a..b1f9f087133 100644 --- a/server/src/services/paperclip-cloud-connector.ts +++ b/server/src/services/paperclip-cloud-connector.ts @@ -36,7 +36,7 @@ export const GMAIL_CONNECTOR_SCOPES = [ export { GOOGLE_WORKSPACE_CONNECTOR_PROFILES }; export type PaperclipCloudConnectorEnvironment = "development" | "test" | "staging" | "production"; -export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events" | "mcp-client-mirror" | "mcp-revocation-index"; +export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events" | "slack-forms" | "mcp-client-mirror" | "mcp-revocation-index"; export type PaperclipCloudConnectorProfileId = GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId | "slack.manager" | "github.bot"; export type PaperclipCloudConnectorProvider = "google" | "github" | "asana" | "slack"; @@ -87,7 +87,7 @@ export type SealedConnectorEvents = { }>; }; -type SealedEnvelopeProfileId = PaperclipCloudConnectorProfileId | "slack.events.v1"; +type SealedEnvelopeProfileId = PaperclipCloudConnectorProfileId | "slack.events.v1" | "slack.form-open.v1" | "slack.callbacks.v1"; type SealedEnvelope = { v: 1; @@ -117,8 +117,10 @@ type ConnectorResponse = { leaseId?: unknown; events?: unknown; acknowledged?: unknown; + formReceipts?: unknown; githubApps?: { version: number }; slackEvents?: { version: number }; + slackForm?: unknown; enrollment?: unknown; retirement?: unknown; eventWakeVersion?: unknown; @@ -128,6 +130,7 @@ const ENDPOINTS: Record = { "mcp-client-mirror": "/v1/connector/mcp-client-mirror", "mcp-revocation-index": "/v1/connector/mcp-revocation-index", "slack-events": "/v1/connector/slack-events", + "slack-forms": "/v1/connector/slack-forms", "github-app": "/v1/connector/github-apps", status: "/v1/connector/instance-status", session: "/v1/connector/sessions", @@ -159,6 +162,8 @@ const BROKER_REJECTION_REASONS = new Set([ "SLACK_ENROLLMENT_GENERATION_CHANGED", "SLACK_ENROLLMENT_IDENTITY_CHANGED", "SLACK_ENROLLMENT_CHANGED", "SLACK_ENROLLMENT_UNSETTLED", "SLACK_ENROLLMENT_DISABLED", "SLACK_COMPANY_MISMATCH", "SLACK_EVENTS_UNAVAILABLE", + "SLACK_FORM_SUBJECT_CHANGED", "SLACK_FORM_INSTANCE_CHANGED", "SLACK_FORM_ENROLLMENT_CHANGED", "SLACK_FORM_CREDENTIAL_CHANGED", + "SLACK_FORM_RETIREMENT_CHANGED", "SLACK_FORM_RETIREMENT_PROOF_MISSING", "SLACK_FORM_CUSTODY_CHANGED", "SLACK_FORM_RETIREMENT_SIGNER_UNAVAILABLE", "SLACK_FORM_CANONICAL_FACT_CHANGED", "SLACK_FORM_CANONICAL_FACT_BUDGET_CHANGED", "SLACK_FORM_CANONICAL_FACT_PENDING", ]); const UNKNOWN_BROKER_REASON = "UNKNOWN_BROKER_ERROR"; @@ -293,7 +298,7 @@ export function createPaperclipCloudConnector(input: { async function call( operation: PaperclipCloudConnectorOperation, claims: { subject: string; companyId: string; profile?: PaperclipCloudConnectorProfileId; scopes?: string[]; returnUri?: string; returnState?: string; claimId?: string; redemptionId?: string; eventWakeVersion?: 0 | 1 }, - secret?: { field: "refreshToken" | "token" | "binding" | "acknowledgement" | "enrollment"; value: string }, + secret?: { field: "refreshToken" | "token" | "binding" | "acknowledgement" | "enrollment" | "operation"; value: string }, ): Promise { const endpoint = new URL(ENDPOINTS[operation], `${config.baseUrl}/`).toString(); const issuedAt = Math.floor(now() / 1000); @@ -398,6 +403,7 @@ export function createPaperclipCloudConnector(input: { } return { + instanceIdentity: () => Object.freeze({ instanceId: config.instanceId, environment: config.environment }), async syncMcpClientMirror(serialized: string): Promise { const response = await call("mcp-client-mirror", {subject:"instance-client-metadata",companyId:"instance-client-metadata"}, {field:"binding",value:serialized}); return (response as ConnectorResponse & {mcpClientMirror?:unknown}).mcpClientMirror; @@ -411,6 +417,50 @@ export function createPaperclipCloudConnector(input: { const response = await call("status", { subject: "instance-capabilities", companyId: "instance-capabilities" }); return response.active === true && response.slackEvents?.version === 1; }, + async slackFormOperation(values: { subject: string; companyId: string; operation: Record }) { + const response = await call("slack-forms", values, { field: "operation", value: JSON.stringify(values.operation) }); + if (!isRecord(response.slackForm)) throw badEnvelope(); + if (values.operation.kind === "receipt" && values.operation.refreshAckReceipt !== undefined) { + if (values.operation.acceptedAckReceipt !== undefined || values.operation.releaseAckFact !== undefined + || Object.keys(response.slackForm).join() !== "ackReceiptJws" || typeof response.slackForm.ackReceiptJws !== "string" + || Buffer.byteLength(response.slackForm.ackReceiptJws, "ascii") > 12000 + || !/^[A-Za-z0-9_-]+[.][A-Za-z0-9_-]+[.][A-Za-z0-9_-]+$/.test(response.slackForm.ackReceiptJws)) throw badEnvelope(); + return response.slackForm; + } + if (values.operation.kind === "receipt" && values.operation.releaseAckFact !== undefined) { + if (values.operation.acceptedAckReceipt !== undefined || Object.keys(response.slackForm).join() !== "releasedAckDigest" + || typeof response.slackForm.releasedAckDigest !== "string" || !/^[a-f0-9]{64}$/.test(response.slackForm.releasedAckDigest)) throw badEnvelope(); + return response.slackForm; + } + if (values.operation.kind === "receipt" && values.operation.acceptedAckReceipt !== undefined) { + if (Object.keys(response.slackForm).sort().join() !== "acceptedReceiptDigest,needsTerminalOriginal" || typeof response.slackForm.acceptedReceiptDigest !== "string" + || !/^[a-f0-9]{64}$/.test(response.slackForm.acceptedReceiptDigest) || typeof response.slackForm.needsTerminalOriginal !== "boolean") throw badEnvelope(); + return response.slackForm; + } + if (!["receipt", "terminal-ack"].includes(String(values.operation.kind))) return response.slackForm; + const envelope = parseEnvelope(response.slackForm.sealed, "events", "slack", "slack.form-open.v1"); + const receipt = decryptEnvelope(envelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(receipt) || receipt.version !== 1 || receipt.instanceId !== config.instanceId || receipt.environment !== config.environment) throw badEnvelope(); + if (!isRecord(receipt.operation)) throw badEnvelope(); + if (receipt.operation.closed !== null) { + if (!isRecord(receipt.operation.closed)) throw badEnvelope(); + const closeEnvelope = parseEnvelope(receipt.operation.closed.evidence, "events", "slack", "slack.form-open.v1"); + const evidence = decryptEnvelope(closeEnvelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(evidence) || evidence.version !== 1 || !["close", "submission"].includes(String(evidence.kind)) || evidence.instanceId !== config.instanceId + || evidence.environment !== config.environment || evidence.operationId !== receipt.operation.id) throw badEnvelope(); + receipt.operation.closed = { ...receipt.operation.closed, evidence }; + } + if (receipt.operation.rejection !== null) { + if (!isRecord(receipt.operation.rejection)) throw badEnvelope(); + const rejectionEnvelope = parseEnvelope(receipt.operation.rejection.evidence, "events", "slack", "slack.form-open.v1"); + const evidence = decryptEnvelope(rejectionEnvelope, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(evidence) || evidence.version !== 1 || evidence.kind !== "rejection" || evidence.instanceId !== config.instanceId + || evidence.environment !== config.environment || evidence.operationId !== receipt.operation.id + || evidence.responseDigest !== receipt.operation.rejection.responseDigest) throw badEnvelope(); + receipt.operation.rejection = { ...receipt.operation.rejection, evidence }; + } + return receipt; + }, async slackEventEnrollment(values: { subject: string; companyId: string; enrollment: Record }) { const response = await call("slack-events", values, { field: "enrollment", value: JSON.stringify(values.enrollment) }); return values.enrollment.active === false ? response.retirement : response.enrollment; @@ -586,6 +636,24 @@ export function createPaperclipCloudConnector(input: { event.payload = { githubApp: delivery }; } if (event.provider === "slack") { + if (event.payload.slackCallback !== undefined) { + const packet = event.payload.slackCallback; + if (!isRecord(packet) || typeof packet.bindingId !== "string" || packet.sealed === undefined) throw badEnvelope(); + const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.callbacks.v1"); + const delivery = decryptEnvelope(sealed, sealKey, config.instanceId, config.environment, "slack", "slack.callbacks.v1", []); + if (!isRecord(delivery) || delivery.version !== 1 || delivery.instanceId !== config.instanceId || delivery.environment !== config.environment + || delivery.bindingId !== packet.bindingId || delivery.acceptedAt !== event.createdAt || delivery.lane !== event.event + || !["command", "action"].includes(String(delivery.lane))) throw badEnvelope(); + event.payload = { slackCallback: delivery }; + } else if (event.payload.slackFormOpen !== undefined) { + const packet = event.payload.slackFormOpen; + if (!isRecord(packet) || typeof packet.operationId !== "string" || packet.bindingId !== `slack-form-open:${packet.operationId}`) throw badEnvelope(); + const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.form-open.v1"); + const delivery = decryptEnvelope(sealed, sealKey, config.instanceId, config.environment, "slack", "slack.form-open.v1", []); + if (!isRecord(delivery) || delivery.version !== 1 || delivery.instanceId !== config.instanceId || delivery.environment !== config.environment + || !isRecord(delivery.operation) || delivery.operation.id !== packet.operationId || delivery.acceptedAt !== event.createdAt) throw badEnvelope(); + event.payload = { slackFormOpen: delivery }; + } else { const packet = event.payload.slackEvent; if (!isRecord(packet) || typeof packet.bindingId !== "string" || packet.sealed === undefined) throw badEnvelope(); const sealed = parseEnvelope(packet.sealed, "events", "slack", "slack.events.v1"); @@ -594,6 +662,7 @@ export function createPaperclipCloudConnector(input: { || delivery.environment !== config.environment || delivery.bindingId !== packet.bindingId || delivery.acceptedAt !== event.createdAt) throw badEnvelope(); event.payload = { slackEvent: delivery }; + } } events.push(event); } catch { @@ -602,6 +671,20 @@ export function createPaperclipCloudConnector(input: { } return { leaseId: opened.leaseId, events, ...(failedEventIds.length ? { failedEventIds } : {}) }; }, + async acknowledgeEventsWithReceipts(values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }): Promise<{ acknowledged: number; formReceipts: { deliveryId: string; receiptJws: string }[] }> { + const response = await call("event-ack", values, { field: "acknowledgement", value: JSON.stringify({ leaseId: values.leaseId, deliveryIds: values.deliveryIds }) }); + if (typeof response.acknowledged !== "number" || !Number.isSafeInteger(response.acknowledged) || response.acknowledged < 0 + || !Array.isArray(response.formReceipts) || response.formReceipts.length > 50) throw badEnvelope(); + const seen = new Set(); + const formReceipts = response.formReceipts.map((value: unknown) => { + if (!isRecord(value) || Object.keys(value).sort().join() !== "deliveryId,receiptJws" || typeof value.deliveryId !== "string" + || !/^slack_form_[a-f0-9]{64}$/.test(value.deliveryId) || !values.deliveryIds.includes(value.deliveryId) || seen.has(value.deliveryId) + || typeof value.receiptJws !== "string" || Buffer.byteLength(value.receiptJws, "ascii") > 12000) throw badEnvelope(); + seen.add(value.deliveryId); return { deliveryId: value.deliveryId, receiptJws: value.receiptJws }; + }); + if (values.deliveryIds.filter(id => /^slack_form_[a-f0-9]{64}$/.test(id)).some(id => !seen.has(id))) throw badEnvelope(); + return { acknowledged: response.acknowledged, formReceipts }; + }, async acknowledgeEvents(values: { subject: string; companyId: string; leaseId: string; deliveryIds: string[] }): Promise { const acknowledgement = JSON.stringify({ leaseId: values.leaseId, deliveryIds: values.deliveryIds }); const response = await call("event-ack", values, { field: "acknowledgement", value: acknowledgement }); @@ -633,7 +716,10 @@ function parseCloudHandoff(value: unknown): { kind: "paperclip_cloud"; session: return { kind: "paperclip_cloud", session }; } -export type PaperclipCloudConnector = ReturnType; +export type PaperclipCloudConnector = Omit, "instanceIdentity" | "slackFormOperation"> & { + instanceIdentity?: () => Readonly<{ instanceId: string; environment: PaperclipCloudConnectorEnvironment }>; + slackFormOperation?: ReturnType["slackFormOperation"]; +}; export type PaperclipCloudGoogleWorkspaceConnector = PaperclipCloudConnector; /** Accept persisted Paperclip ID-era records while all new records use the Cloud strategy. */ diff --git a/tests/runner-e2e/source-sdk-evidence.test.ts b/tests/runner-e2e/source-sdk-evidence.test.ts index 188d370a163..99803867d83 100644 --- a/tests/runner-e2e/source-sdk-evidence.test.ts +++ b/tests/runner-e2e/source-sdk-evidence.test.ts @@ -5,6 +5,8 @@ import os from "node:os"; import path from "node:path"; import { promisify } from "node:util"; import { existsSync } from "node:fs"; +import { prepareReviewedSdkReference } from "./source-sdk-reviewed-fixture.js"; +import pins from "./source-sdk-public-inputs.json" with { type: "json" }; import { afterEach, expect, it } from "vitest"; import { stagePrivateDiagnostics } from "./private-diagnostics.js"; import { prepareSourceServerDependencies } from "./source-startup.js"; @@ -110,11 +112,24 @@ it("retains declarations and source maps under names the private result sealer a for (const file of receipt.files) expect(hash(await readFile(path.join(uploadDir, SOURCE_SDK_DIAGNOSTICS, file.retainedPath)))).toBe(file.sha256); await expect(sealManualResultEvidence(context, [], results)).resolves.not.toThrow(); }); -// Needs the built SDK (scripts/ensure-plugin-build-deps.mjs) and the pinned public source. +// Build a genuine OWN normal producer-certified reviewed reference. The live +// CI lock can be regenerated for unrelated PR patches and must not be relabeled reviewed. const builtSdk = ["packages/shared/dist", "packages/plugins/sdk/dist"].every(target => existsSync(new URL(`../../${target}/.paperclip-build-complete`, import.meta.url))); -it.skipIf(!builtSdk)("seals the real built SDK upload, keeping proven key-shaped declarations out", async () => { - const repositoryRoot = await realpath(path.resolve(import.meta.dirname, "../..")); +it("seals a normal producer-certified reviewed-reference SDK, keeping key-shaped declarations out", async () => { + const originalRoot=await realpath(path.resolve(import.meta.dirname,"../..")); + const repositoryRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-reviewed-build-")));roots.push(repositoryRoot); + await prepareReviewedSdkReference(originalRoot,repositoryRoot); + // Existing normal Shared then SDK producer, exact original configs/options. + // Fresh dist directories and OWN cache/Shared resolution; no donor outputs. + expect(existsSync(path.join(repositoryRoot,"packages/shared/dist"))).toBe(false); + expect(existsSync(path.join(repositoryRoot,"packages/plugins/sdk/dist"))).toBe(false); + await prepareSourceServerDependencies(repositoryRoot,{...process.env},false); + for(const directory of ["packages/shared/dist","packages/plugins/sdk/dist"]){ + expect(await realpath(path.join(repositoryRoot,directory))).toBe(path.join(repositoryRoot,directory)); + expect(JSON.parse(await readFile(path.join(repositoryRoot,directory,".paperclip-build-complete"),"utf8"))) + .toMatchObject({sources:expect.stringMatching(/^[a-f0-9]{64}$/),outputs:expect.stringMatching(/^[a-f0-9]{64}$/)}); + } const root = await realpath(await mkdtemp(path.join(os.tmpdir(), "source-sdk-real-unit-"))); roots.push(root); const temporaryRoot = path.join(root, "attempt"), results = path.join(root, "results"), uploadDir = path.join(results, "campaign", "attempt-1"); await mkdir(path.join(temporaryRoot, "artifacts-private"), { recursive: true, mode: 0o700 }); await mkdir(path.join(temporaryRoot, "selected-private-inputs"), { recursive: true, mode: 0o700 }); @@ -126,3 +141,35 @@ it.skipIf(!builtSdk)("seals the real built SDK upload, keeping proven key-shaped const context = { root, protocol: MANUAL_PRIVATE_PROTOCOL, private: true, workflowSha: "a".repeat(40), targetSha: "b".repeat(40), runId: "123", attempt: "1", executionIds: [MANUAL_PRIVATE_EXECUTIONS[0]!] } as ManualPrivateHandoff; await expect(sealManualResultEvidence(context, secrets, results)).resolves.not.toThrow(); }, 120_000); + +it("withholds real unreviewed-build SDK bytes instead of authenticating the CI-resolved lock",async()=>{ + const originalRoot=await realpath(path.resolve(import.meta.dirname,"../..")); + let repositoryRoot=originalRoot,restoreLock:(()=>Promise)|undefined; + const actual=await readFile(path.join(originalRoot,"pnpm-lock.yaml")); + if(hash(actual)===pins.lockSha256){ + // On a reviewed local checkout, make the same refusal observable in an + // OWN genuine-built reference by invalidating only its owning lock input. + repositoryRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-unreviewed-build-")));roots.push(repositoryRoot); + await prepareReviewedSdkReference(originalRoot,repositoryRoot); + await prepareSourceServerDependencies(repositoryRoot,{...process.env},false); + const file=path.join(repositoryRoot,"pnpm-lock.yaml"),reviewed=await readFile(file); + await writeFile(file,Buffer.concat([reviewed,Buffer.from("\n# unreviewed unit input mutation\n")])); + restoreLock=()=>writeFile(file,reviewed); + }else if(!builtSdk)throw new Error("Unreviewed CI build requires its actual normal producer outputs"); + const root=await realpath(await mkdtemp(path.join(os.tmpdir(),"source-sdk-unreviewed-refusal-")));roots.push(root); + const temporaryRoot=path.join(root,"attempt"); + await mkdir(path.join(temporaryRoot,"selected-private-inputs"),{recursive:true,mode:0o700}); + stagePrivateDiagnostics(temporaryRoot,[executionId],["synthetic-unit-provider-credential-0123456789"]); + try{ + await expect(retainPreparedSourceSdk({repositoryRoot,temporaryRoot,executionIds:[executionId],sourceRevision,installed:false})) + .rejects.toThrow("Source SDK evidence privacy check failed; bytes withheld"); + const file=path.join(temporaryRoot,"artifacts-private/cases/plugins.install-plugin/snapshots/source-sdk-privacy-observations.json"); + const observation=JSON.parse(await readFile(file,"utf8")); + expect(observation.complete).toBe(false);expect(observation.audit.publicSource).toBeNull(); + expect(observation.audit.bounded).toBe(true);expect(observation.audit.suppressedFindings).toBe(0); + expect(observation.audit.observations).toEqual(expect.arrayContaining([expect.objectContaining({ + pathSha256:hash("packages/shared/dist/validators/agent.d.ts"),detector:3,classification:"unresolved_context"})])); + await expect(stat(path.join(temporaryRoot,"artifacts-private/source-sdk-build/receipt.json"))).rejects.toThrow(); + await expect(stat(path.join(temporaryRoot,"artifacts-private/source-sdk-build/packages"))).rejects.toThrow(); + }finally{await restoreLock?.();} +},120_000); diff --git a/tests/runner-e2e/source-sdk-public-type.test.ts b/tests/runner-e2e/source-sdk-public-type.test.ts index 552d5fdc7f6..7678df7e7e6 100644 --- a/tests/runner-e2e/source-sdk-public-type.test.ts +++ b/tests/runner-e2e/source-sdk-public-type.test.ts @@ -3,6 +3,7 @@ import { copyFile, mkdir, mkdtemp, readFile, realpath, rm, symlink, writeFile } import path from "node:path"; import os from "node:os"; import { beforeAll, afterAll, expect, it } from "vitest"; +import { prepareReviewedSdkReference } from "./source-sdk-reviewed-fixture.js"; import pins from "./source-sdk-public-inputs.json" with { type: "json" }; import { createSdkArtifactScanner, sdkPublicTypeSyntax, verifySdkPublicSource } from "./source-sdk-public-type.js"; import { findSecretLeakMatch, normalizedSecrets } from "./redaction.js"; @@ -12,16 +13,18 @@ const blob = (v: string | Buffer) => createHash("sha1").update(`blob ${Buffer.by const originalRoot = path.resolve(import.meta.dirname, "../.."), file = "packages/shared/dist/validators/synthetic-unit.d.ts", inputFile = file.replace("/dist/", "/src/").replace(/\.d\.ts$/, ".ts"); const source = 'import {z} from "zod";export const schema=z.object({daytona_api_key:z.string().min(1)});'; const declaration = 'import {z} from "zod";export declare const schema:{daytona_api_key:z.ZodOptional};'; -let root: string; let fixtureLockSha256: string; +let root: string; let reviewedRoot: string; let fixtureLockSha256: string; const fixtureInputs: Record = {}; beforeAll(async () => { root = await realpath(await mkdtemp(path.join(os.tmpdir(), "sdk-public-type-synthetic-"))); - for (const [relative] of Object.entries(pins.inputs)) { const dest = path.join(root, relative); await mkdir(path.dirname(dest), { recursive: true }); await copyFile(path.join(originalRoot, relative), dest); fixtureInputs[relative] = blob(await readFile(dest)); } - await copyFile(path.join(originalRoot, "pnpm-lock.yaml"), path.join(root, "pnpm-lock.yaml")); fixtureLockSha256 = hash(await readFile(path.join(root, "pnpm-lock.yaml"))); - await symlink(await realpath(path.join(originalRoot, "node_modules")), path.join(root, "node_modules")); - await symlink(await realpath(path.join(originalRoot, "packages/shared/node_modules")), path.join(root, "packages/shared/node_modules")); + await prepareReviewedSdkReference(originalRoot,root); + reviewedRoot=await realpath(await mkdtemp(path.join(os.tmpdir(),"sdk-public-reviewed-reference-"))); + await prepareReviewedSdkReference(originalRoot,reviewedRoot); + for(const relative of Object.keys(pins.inputs))fixtureInputs[relative]=blob(await readFile(path.join(root,relative))); + fixtureLockSha256=hash(await readFile(path.join(root,"pnpm-lock.yaml"))); + }); -afterAll(async () => { if (root) await rm(root, { recursive: true, force: true }); }); +afterAll(async () => { for(const directory of [root,reviewedRoot])if(directory)await rm(directory,{recursive:true,force:true}); }); async function fixture(sourceText = source) { await mkdir(path.dirname(path.join(root, inputFile)), { recursive: true }); await writeFile(path.join(root, inputFile), sourceText); return { ...pins, lockSha256: fixtureLockSha256, inputs: { ...fixtureInputs, [inputFile]: blob(sourceText) } }; @@ -29,13 +32,7 @@ async function fixture(sourceText = source) { it("default public proof accepts the reviewed build lock and still rejects a changed lock", async () => { const directory = await realpath(await mkdtemp(path.join(os.tmpdir(), "sdk-reviewed-runtime-"))); try { - for (const relative of Object.keys(pins.inputs)) { - const destination = path.join(directory, relative); - await mkdir(path.dirname(destination), { recursive: true }); - await copyFile(path.join(originalRoot, relative), destination); - } - await symlink(await realpath(path.join(originalRoot, "node_modules")), path.join(directory, "node_modules")); - await symlink(await realpath(path.join(originalRoot, "packages/shared/node_modules")), path.join(directory, "packages/shared/node_modules")); + await prepareReviewedSdkReference(originalRoot,directory); const reviewed = JSON.parse(await readFile(path.join(originalRoot, "tests/runner-e2e/report-lock.json"), "utf8")); const lock = await readFile(path.join(directory, "pnpm-lock.yaml")); expect(hash(lock)).toBe(reviewed.outputSha256); @@ -56,7 +53,7 @@ it("classifies the shipped agent env schema declaration and keeps a literal at t // Plugin cells retain packages/shared/dist/validators/agent.d.ts, whose // `OPENROUTER_API_KEY: z.ZodOptional` matches the OpenRouter key // shape. Stale default pins left it unresolved and blocked web-server startup. - const repository = await realpath(originalRoot), agentFile = "packages/shared/dist/validators/agent.d.ts"; + const repository = reviewedRoot, agentFile = "packages/shared/dist/validators/agent.d.ts"; const shipped = 'import { z } from "zod";\nexport declare const env: z.ZodObject<{\n OPENROUTER_API_KEY: z.ZodOptional;\n}>;\n'; const scanner = createSdkArtifactScanner(repository, []); expect(await scanner.scan(agentFile, Buffer.from(shipped))).toBe(true); expect((await scanner.audit()).observations.map(row => row.classification)).toEqual(["public_schema_type"]); diff --git a/tests/runner-e2e/source-sdk-reviewed-fixture.ts b/tests/runner-e2e/source-sdk-reviewed-fixture.ts new file mode 100644 index 00000000000..cfdbaafdccf --- /dev/null +++ b/tests/runner-e2e/source-sdk-reviewed-fixture.ts @@ -0,0 +1,72 @@ +// Test-only reviewed SDK reference. Actual CI builds may use an unreviewed +// resolved lock; neither this fixture nor production admits that as public proof. +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { copyFile, mkdir, readFile, readdir, realpath, symlink, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; +import path from "node:path"; +import pins from "./source-sdk-public-inputs.json" with { type: "json" }; +import { prepareReportLock } from "./prepare-report-lock.mjs"; +import { verifySdkPublicSource } from "./source-sdk-public-type.js"; +const sha=(bytes:Buffer)=>createHash("sha256").update(bytes).digest("hex"); +const blob=(bytes:Buffer)=>createHash("sha1").update(`blob ${bytes.length}\0`).update(bytes).digest("hex"); +const gitFile=(root:string,file:string)=>execFileSync("git",["show",`HEAD:${file}`],{cwd:root,maxBuffer:8*1024*1024,stdio:["ignore","pipe","pipe"]}); + +// Create owned directories, link only read-only installed third-party inputs. +// Never link workspace compiled exports or donor node_modules/.cache: the +// normal producer's lock and every Shared/SDK emit stay in this fixture root. +async function thirdPartyLinks(from:string,to:string,installedStore:string){ + const link=async(source:string,destination:string)=>{ + const target=await realpath(source); + // A local workspace may have an unscoped name too. Admit only installed + // third-party store entries, never another checkout/package's exports. + if(target!==installedStore&&!target.startsWith(installedStore+path.sep))return; + await symlink(target,destination); + }; + await mkdir(to,{recursive:true}); + for(const entry of await readdir(from,{withFileTypes:true})){ + if([".cache",".bin","@paperclipai"].includes(entry.name))continue; + const source=path.join(from,entry.name),destination=path.join(to,entry.name); + if(entry.name.startsWith("@")){ + await mkdir(destination,{recursive:true}); + for(const child of await readdir(source))await link(path.join(source,child),path.join(destination,child)); + }else await link(source,destination); + } +} +export async function prepareReviewedSdkReference(original:string,directory:string){ + if(await realpath(directory)!==directory||(await readdir(directory)).length!==0)throw new Error("Reviewed SDK reference needs a fresh owned directory"); + const policyFile="tests/runner-e2e/report-lock.json",patchFile="tests/runner-e2e/report-lock.patch"; + const policyBytes=gitFile(original,policyFile),policy=JSON.parse(policyBytes.toString("utf8")); + if(!(await readFile(path.join(original,policyFile))).equals(policyBytes)||policy.outputSha256!==pins.lockSha256) + throw new Error("Reviewed SDK reference policy does not match public proof"); + const committed=gitFile(original,"pnpm-lock.yaml"); + if(![policy.inputSha256,policy.outputSha256].includes(sha(committed)))throw new Error("Committed SDK reference lock is not reviewed"); + for(const [file,expected]of Object.entries(pins.inputs)){ + const bytes=file==="pnpm-lock.yaml"?committed:await readFile(path.join(original,file)); + if(file!=="pnpm-lock.yaml"&&blob(bytes)!==expected)throw new Error("Reviewed SDK reference source changed"); + const target=path.join(directory,file);await mkdir(path.dirname(target),{recursive:true}); + if(file==="pnpm-lock.yaml")await writeFile(target,bytes);else await copyFile(path.join(original,file),target); + } + for(const file of [policyFile,patchFile]){ + const bytes=gitFile(original,file); + if(!(await readFile(path.join(original,file))).equals(bytes))throw new Error("Reviewed SDK lock policy changed"); + const target=path.join(directory,file);await mkdir(path.dirname(target),{recursive:true}); + await writeFile(target,bytes); + } + // Same existing authenticated input->output correction. No hash refresh, + // changed-lock admission, install, resolver or compiler-option substitution. + await prepareReportLock(directory); + if(sha(await readFile(path.join(directory,"pnpm-lock.yaml")))!==policy.outputSha256)throw new Error("Reviewed SDK reference lock changed"); + const installedStore=await realpath(path.join(original,"node_modules/.pnpm")); + await thirdPartyLinks(path.join(original,"node_modules"),path.join(directory,"node_modules"),installedStore); + for(const name of ["shared","plugins/sdk"])await thirdPartyLinks(path.join(original,"packages",name,"node_modules"),path.join(directory,"packages",name,"node_modules"),installedStore); + for(const modules of ["node_modules","packages/plugins/sdk/node_modules"]){ + const scope=path.join(directory,modules,"@paperclipai");await mkdir(scope,{recursive:true}); + await symlink(path.join(directory,"packages/shared"),path.join(scope,"shared")); + } + const require=createRequire(path.join(directory,"packages/plugins/sdk/package.json")); + const sharedEntry=await realpath(require.resolve("@paperclipai/shared")); + if(!sharedEntry.startsWith(path.join(directory,"packages/shared")+path.sep))throw new Error("SDK reference resolved a donor workspace"); + // Authenticate original compiler/parser/Zod/source pins BEFORE normal build. + await verifySdkPublicSource(directory); +}