diff --git a/doc/plans/2026-10-12-chat-channel-workflow-refactor.md b/doc/plans/2026-10-12-chat-channel-workflow-refactor.md new file mode 100644 index 00000000000..0bb245a9afb --- /dev/null +++ b/doc/plans/2026-10-12-chat-channel-workflow-refactor.md @@ -0,0 +1,159 @@ +# Chat channel workflow refactor + +The user authorized six hours of sustained work starting at 2026-10-13 02:14 UTC, +with a target of fewer than 8,000 lines in `server/src/services/chat-channels.ts`. +The starting commit is `7eaa9cfa9ce04398a5eb0211a86aacaaa01959dd`: 19,153 lines, +762,442 bytes. At kickoff, the preceding ten PRs were reviewed and green, but not yet merged. +The user authorizes publishing and merging reviewed changes to paperclipai/paperclip. + +The previous plan prioritized too many small helper cuts. This plan prioritizes +complete workflows, provider ownership, and meaningful reductions per PR. Shared +transactions and admission policy belong in shared workflow modules. Provider +transport, envelope checks, setup, and native interactions belong in provider +folders. Do not put shared code in one provider's folder or use a mutable context +bag to move the entire service unchanged. + +| Priority | Responsibility and destination | Estimated net reduction | Contract and state | Risk and verification | +| --- | --- | ---: | --- | --- | +| 1 | Native interaction callbacks and continuation: `interactions/`, `telegram/confirmation-actions.ts`, `discord/question-forms.ts` | 2,900 | Current actor/runtime/destination readers; durable token and interaction transactions; deferred wakeups. | Preserve token replay, audit order, provider acknowledgement, post-commit recovery, and live fault barriers. Existing public service, HTTP and real-database interaction tests before and after. | +| 2 | Endpoint configuration and provider setup: provider `setup.ts`, shared credential persistence/configuration | 1,600–2,200 | Credential lease and current endpoint; provider registration outcome; stable native identity. | Preserve fail-closed persistence, webhook/receiver ownership, partial setup cleanup, and credential rotation races. Existing setup and runtime tests. | +| 3 | Message admission and provider policy: shared intake workflow, GitHub message policy, provider envelope projections | 2,400–3,000 | Delivery claim and conversation/task admission; explicit provider policy results; per-service message provenance. | Preserve row locks, deduplication, author authority, task generations, attachments, and GitHub automatic/explicit admission. Existing real-database ingress and replay tests. | +| 4 | Durable delivery and lifecycle: shared queue, provider lifecycle inputs | 1,700–2,000 | Conversation leases, replay decode, ordered drains, mutation reconciliation; shared live receipt state. | Preserve stale lease exclusion, event order, deletion/edit races, deferred startup and shutdown. Existing concurrency and lifecycle tests. | +| 5 | Outbound publication: provider delivery modules and shared publication worker | 2,700–3,400 | Current-source authorization; exact transport parts; attempts, unknown outcomes, completion receipts and cleanup. | Preserve no duplicate sends, ambiguous acceptance, credential fencing, file recovery, concurrency and current task-control completion. Existing assembled publication and file tests. | +| 6, if needed | Operator recovery and endpoint projections | 800–1,200 | Explicit replay/resolution and company-scoped board calls. | Preserve retry audit, current policy and authorization. Existing operator-resolution integration tests. | + +The estimates are not a PR quota. Inspect every full execution path before moving +it. Increase size only where a cohesive responsibility has a narrow contract. +After every reviewed slice, remap the remaining file and report the measured size. + +For each slice: run a behavioral baseline before editing production; retain +existing bodies and SQL; compare normalized AST and statement order; run server +typecheck, module boundaries and public-path tests. Review all intentional callback +and dispatch substitutions. Preserve live option reads with getters or callbacks. +Keep per-service state per service and shared cache/lease ownership shared. + +Open one PR per cohesive extraction with the full repository template. Verify its +current-head review and all CI gates before opening the next extraction. Build each +on its immediate predecessor. Run repository-wide typecheck, tests and build once +at the final handoff; investigate and disclose any failures or unreached coverage. +The earlier batch recorded local timing/host cleanup failures; green CI and passing +focused retries do not erase that evidence. No timeout or assertion weakening. + +Public application fixtures prove assembled service/HTTP/database behavior. They +do not prove live third-party execution. Use only owned fixture data and processes. + +## First extraction + +The native interaction workflow reduces the service from 19,153 to 16,117 lines +and from 762,442 to 656,146 bytes. Telegram owns confirmation resolution and +settlement. Discord owns correction draft retention and accepted-answer cleanup. +Shared callbacks retain token/source/actor checks and atomic question resolution. +Shared continuation work retains durable wakeup recovery. SDK state persistence +has a direct module import and retains the public compatibility export. + +The same 62 public-path interaction cases pass before and after. Seven supporting +suites pass all 70 cases. Server typecheck and module boundaries pass. AST checks +cover nine moved declarations, three provider bodies, and retained service order. +Fault barriers remain live getters. No assertion or timeout changed. + +## Second extraction + +Endpoint configuration and provider setup reduce the service to 14,241 lines and +580,227 bytes. Shared modules own endpoint records, credential identity/persistence, +and the credential-leased configuration state machine. Channel folders own Telegram +registration, GitHub repair, Speko tool installation, Photon configuration and +inspection, and Slack setup projection. Registration dependencies remain live. + +All 1,278 chat cases passed on the first PR before this move. The final setup wiring +passes 268 selected integration cases, server typecheck, and module boundaries. +AST checks cover 25 declarations and nine provider bodies. One final test attempt +could not start PostgreSQL because this Mac exhausted IPC slots; after reclaiming +one unused zero-attachment segment with a dead creator, the same suite passed. + +## Third extraction + +The publication workflow reduces the service to 11,198 lines and 462,108 bytes. +Authorization, durable preparation, transport, failures, attempt ownership, task +control completion, and bounded endpoint scheduling now have named modules. +Photon owns prompt and reply delivery. Discord and Telegram own durable Markdown +attachment preparation. Shared native splitting remains shared across providers. + +The same 301 public-path publication cases pass before and after. Eight supporting +suites pass all 185 cases. Server typecheck, module boundaries, and AST checks pass. +The AST audit covers 26 declarations, three provider bodies, and retained service +order. Shared endpoint lanes and Teams cooldowns retain their original maps; +shutdown and publication wake listeners stay live. Early callback registration +uses typed forwarding functions instead of reading factories before initialization. + +## Fourth extraction + +Ordered delivery and lifecycle reconciliation reduce the service to 9,287 lines +and 388,734 bytes. The delivery ledger owns ordering and runtime-fence projections. +The conversation queue owns exact leases, reconstruction and ordered replay. +The lifecycle workflow owns edit/delete admission and reconciliation. Teams owns +authenticated route recording. GitHub owns restoration of durable message policy +into the same per-service provenance maps. + +The same 225 lifecycle, ordering, lease, replay and recovery cases pass before and +after. Typecheck and module boundaries pass. AST checks cover 26 declarations, the +GitHub restoration body, and retained service order. Live maps and lease hooks retain +their original lifetime. The publication predecessor's packaging retry passed unchanged. This fourth +slice is published as PR #16096. + + +## Fifth extraction + +Inbound authority, envelope preparation, durable admission and resource admission +reduce the service to 7,899 lines and 335,607 bytes. The shared admission module +keeps endpoint locking, runtime and credential fences, gateway lease renewal, +deduplication and delivery insertion in one transaction. Teams, Telegram and +Photon own their destination rules and use that same transaction. The main +message path now calls preparation, durable admission and resource admission +before the retained task workflow. + +The same 278 selected public-path cases pass before and after the first move. +All 1,278 existing chat integration cases pass on the final provider dispatch +in 309.47 seconds. Server typecheck and module boundaries pass. One full-suite attempt +could not allocate PostgreSQL shared memory before any test ran. Only a verified +unused 56-byte segment with a dead creator and zero attachments was reclaimed +before retrying. No active database was stopped. + +AST checks cover 21 moved declarations, envelope and resource phase bodies, +three provider destination bodies, and the complete admission transaction after +restoring the explicit provider calls. The service still owns the per-call +gateway ownership handoff. The envelope retains its per-call recovery map and +the original authenticated runtime. No assertion or timeout changed. + +## Stack landing + +The preceding ten PRs reached master through #16058. The five workflow PRs are +#16080, #16085, #16089, #16096 and #16105. Each extraction has individual behavioral +evidence and a 5/5 review. The four child PRs merged into their predecessors after +each combined revision passed its own checks and review. Every consolidation +preserved the tested tip tree exactly. + +The final master integration leaves the service at 7,917 lines and 337,195 bytes. +The 18-line increase over the extraction tip preserves upstream execution-default +selection and authorization. Endpoint serialization and credential-leased updates +retain that policy in the extracted endpoint module. Task creation and failed-run +recovery retain their upstream behavior. All 107 extracted declarations match +current master after reversing the documented binding substitutions. The endpoint, +task-creation and recovery functions also pass exact normalized AST comparisons. + +Repository-wide typecheck and build pass on this integration. The first full local +test command passed 18,391 server tests and skipped 105, but one unchanged tool-access +test hit its 15-second timeout. That case passed alone. The UI and CLI groups passed +8,706 tests. Shared and adapter groups passed 4,751 tests and skipped 20. A database +group could not initialize one PostgreSQL fixture. This Mac has a 32-segment shared +memory limit and concurrent fixture databases. The unchanged database cases are +being rerun one file at a time before the remaining chat and serialized route lanes. +Keep the original failed runs in the verification record. No assertion or timeout +was changed. The root PR remains open for fresh review and CI against master. + +The service now has 67 direct functions. The largest retained paths are message +task handling (1,449 lines), webhook dispatch (685), operator action resolution +(488), and publication resolution (343). Provider setup, transport preparation, +native interactions and destination admission live in their channel folders. +Shared admission, delivery, lifecycle and publication transactions have named +workflow modules. The retained message path calls those phases and keeps task +mutation and per-call ownership together. diff --git a/server/src/services/chat-channels.ts b/server/src/services/chat-channels.ts index ea9742c0bcf..f4fd375d29e 100644 --- a/server/src/services/chat-channels.ts +++ b/server/src/services/chat-channels.ts @@ -1,22 +1,52 @@ +import { PROVIDER_LABELS, CAPABILITIES } from "./chat-channels/provider-catalog.js"; +import { createMessageResourceAdmission } from "./chat-channels/inbound/resource-admission.js"; +import type { DiscordGatewayOwnership } from "./chat-channels/runtime-types.js"; +import { createInboundAdmission } from "./chat-channels/inbound/admission.js"; +import { createInboundEnvelope } from "./chat-channels/inbound/envelope.js"; +import { sameTeamsThreadIdentity, durableExternalThreadIdentity, teamsThreadRootMessageId, canonicalProviderResourceId, nonDirectDestinationAllowed, linearControlCommand, actionThreadMatchesConversation } from "./chat-channels/inbound/surfaces.js"; +import { createChatInboundAuthority } from "./chat-channels/inbound/authorization.js"; +import { createTeamsRoutes } from "./chat-channels/microsoft-teams/routes.js"; +import type { LiveInboundMessage } from "./chat-channels/runtime-types.js"; +import { DELIVERY_PROCESSING_STALE_MS, normalizedDeliveryThreadId, normalizedLifecycleEffect, deliveryReady, pendingInboundWakeupCondition, lifecycleRuntimeFence, normalizedLifecycleWithRuntimeFence } from "./chat-channels/inbound/delivery-ledger.js"; +import { createChatMessageLifecycle } from "./chat-channels/inbound/lifecycle.js"; +import { createChatConversationDeliveryQueue } from "./chat-channels/inbound/conversation-queue.js"; +import { publicationTransportOrderKey } from "./chat-channels/publications/order.js"; +import { createChatPublicationTaskControl } from "./chat-channels/publications/task-control.js"; +import { createChatPublicationTransport } from "./chat-channels/publications/transport.js"; +import { createChatPublicationAuthorization } from "./chat-channels/publications/authorization.js"; +import { createChatPublicationFailures } from "./chat-channels/publications/failures.js"; +import { createChatPublicationPreparation } from "./chat-channels/publications/preparation.js"; +import { createChatPublicationWorker } from "./chat-channels/publications/worker.js"; +import { createChatPublicationQueue } from "./chat-channels/publications/queue.js"; +import { isExplicitOperatorPublication } from "./chat-channels/publications/policy.js"; + +import { createChatCredentials } from "./chat-channels/credentials.js"; +import { createChatEndpoints } from "./chat-channels/endpoints.js"; +import { createChatConfiguration } from "./chat-channels/configuration.js"; +import { createPhotonSetupInspection } from "./chat-channels/imessage-photon/setup.js"; +import { createChatSdkStatePersistence } from "./chat-channels/sdk-state.js"; +export { createChatSdkStatePersistence } from "./chat-channels/sdk-state.js"; +import { createChatInteractionCallbacks } from "./chat-channels/interactions/callbacks.js"; +import { createChatInteractionContinuations } from "./chat-channels/interactions/continuations.js"; import { createGitHubInstallationLifecycle } from "./chat-channels/github/installation-lifecycle.js"; import { createGitHubAppSetup } from "./chat-channels/github/app-setup.js"; import { createSlackTaskAuthorityRegistration } from "./chat-channels/slack/task-authority.js"; import { createGitHubPublicationPolicy } from "./chat-channels/github/publication-policy.js"; import { createTelegramAttachmentFailure } from "./chat-channels/telegram/attachment-failure.js"; -import { createSlackCommands, slackCommandForAgent } from "./chat-channels/slack/commands.js"; +import { createSlackCommands } from "./chat-channels/slack/commands.js"; import { createTelegramCommands, telegramGuidanceCommand } from "./chat-channels/telegram/commands.js"; import { createTeamsFileDelivery } from "./chat-channels/microsoft-teams/file-delivery.js"; import { createGitHubSetupSecret } from "./chat-channels/github/setup-secret.js"; -import { prepareSlackInventory, createSlackInventory } from "./chat-channels/slack/inventory.js"; -import { prepareGitHubInventory } from "./chat-channels/github/inventory.js"; -import { prepareDiscordInventory } from "./chat-channels/discord/inventory.js"; -import type { VerifiedProviderIdentity } from "./chat-channels/runtime-types.js"; -import { verifySlackCredentials } from "./chat-channels/slack/credentials.js"; -import { verifyDiscordCredentials } from "./chat-channels/discord/credentials.js"; -import { verifyTelegramCredentials } from "./chat-channels/telegram/credentials.js"; -import { verifyGitHubCredentials, REQUIRED_GITHUB_EVENTS, UNAVOIDABLE_GITHUB_EVENTS } from "./chat-channels/github/credentials.js"; -import { verifySpekoCredentials } from "./chat-channels/speko/credentials.js"; -import { verifyTeamsCredentials } from "./chat-channels/microsoft-teams/credentials.js"; +import { createSlackInventory } from "./chat-channels/slack/inventory.js"; + + + + + + +import { REQUIRED_GITHUB_EVENTS, UNAVOIDABLE_GITHUB_EVENTS } from "./chat-channels/github/credentials.js"; + + import { createPhotonCredentialChecks } from "./chat-channels/imessage-photon/credentials.js"; import { slackTestMessageService } from "./connectors/slack/test-message.js"; import type { SlackSetupOptions } from "./connectors/slack/setup/registration.js"; @@ -24,19 +54,19 @@ import { isProvisionedSlackSetup } from "@paperclipai/shared"; import { createSetupProbes } from "./chat-channels/setup-probes.js"; import { createChatActivity, publicationSummary } from "./chat-channels/activity.js"; import { fileTransferProjectionColumns } from "./chat-channels/microsoft-teams/file-transfer-projection.js"; -import { createProviderEffects, providerEffectPayload, type ProviderEffectTarget, type ProviderEffectPayload } from "./chat-channels/provider-effects.js"; +import { createProviderEffects, providerEffectPayload, type ProviderEffectTarget } from "./chat-channels/provider-effects.js"; import { createTelegramCallbackNoticeAuthority } from "./chat-channels/telegram/callback-notices.js"; -import { createOutboundAttachments, isOutboundAttachmentHydrationError, isOutboundAttachmentValidationError } from "./chat-channels/outbound-attachments.js"; +import { createOutboundAttachments } from "./chat-channels/outbound-attachments.js"; export { hydrateOutboundAttachment } from "./chat-channels/outbound-attachments.js"; -import { discordMarkdownAttachment } from "./chat-channels/discord/messages.js"; -import { telegramMarkdownAttachment } from "./chat-channels/telegram/messages.js"; + + import { createInboundAttachments } from "./chat-channels/inbound-attachments.js"; import { createInboundReactions } from "./chat-channels/reactions.js"; -import { slackThreadChannelId } from "./chat-channels/slack/messages.js"; + import { createReceiptReactions } from "./chat-channels/receipt-reactions.js"; import { createDiscordRootThreads } from "./chat-channels/discord/root-threads.js"; import { createDiscordGatewayEvents } from "./chat-channels/discord/gateway-events.js"; -import { createSlackCallbackVerification, slackRequestSignatureIsValid, slackRequestWorkspaceId, canonicalCallbackUrl, slackCallbackMatchesPublicUrl, inspectSlackCallback } from "./chat-channels/slack/callback-verification.js"; +import { createSlackCallbackVerification, slackRequestSignatureIsValid, slackRequestWorkspaceId, slackCallbackMatchesPublicUrl, inspectSlackCallback } from "./chat-channels/slack/callback-verification.js"; import { createTeamsFileAuthority } from "./chat-channels/microsoft-teams/file-transfer-authority.js"; import { createPhotonEvents } from "./chat-channels/imessage-photon/events.js"; import { createTelegramPublicationDrafts } from "./chat-channels/telegram/publication-drafts.js"; @@ -47,40 +77,30 @@ import { createSlackTaskStart, slackSlashTaskRecoveryPayload, slackTaskStarterTh import { createGitHubWebhookIngress, type GitHubIngressStageResult } from "./chat-channels/github/webhook-ingress.js"; import { createSlackSessionStop } from "./chat-channels/slack/session-stop.js"; import { createTelegramMaintenance } from "./chat-channels/telegram/maintenance.js"; -import type { EndpointRow, DbTransaction, DbOrTransaction, RuntimeContext, LifecycleRuntimeFence, InboundRuntimeContext, SlackCallbackSurface, InternalSetupState } from "./chat-channels/runtime-types.js"; +import type { EndpointRow, DbTransaction, DbOrTransaction, RuntimeContext, LifecycleRuntimeFence, InboundRuntimeContext, InternalSetupState } from "./chat-channels/runtime-types.js"; import { createChatFailedRunRetry } from "./chat-channels/failed-run-retry.js"; import { createChatPublicationReplacement } from "./chat-channels/publication-replacement.js"; import { createChatIdentityLinks } from "./chat-channels/identity-links.js"; import { createChatResourceReconciliation } from "./chat-channels/resource-reconciliation.js"; -import { baseTeamsConversationId } from "./chat-channels/microsoft-teams/resources.js"; + import { slackResourceLabelIsFallback } from "./chat-channels/slack/resources.js"; -import { telegramResourceLabelIsFallback } from "./chat-channels/telegram/resources.js"; + import { MAX_INBOUND_TEXT, stableExternalPrincipalId, - type LifecycleActor, - lifecycleActorFromAuthor, } from "./chat-channels/message-lifecycle.js"; import { telegramLifecycleEventFromPayload, isTelegramEditedMessageRaw, isTelegramMigrationRaw, - telegramMessageSequence, telegramZeroMessageId, telegramDeliveryHasZeroMessageId, - telegramMessageSentAt, } from "./chat-channels/telegram/messages.js"; -import { - slackLifecycleFilesDigest, - slackMessageSentAt, -} from "./chat-channels/slack/messages.js"; -import { - discordLifecycleFilesDigest, -} from "./chat-channels/discord/messages.js"; + + import { microsoftTeamsLifecycleEventFromPayload, - teamsActivitySentAt, } from "./chat-channels/microsoft-teams/messages.js"; import { type GitHubLifecycleEvent, @@ -89,8 +109,8 @@ import { import { dueChatAction, nextChatActionAt, SLACK_COMMAND_POST_STALE_MS, SLACK_COMMAND_EXPLICIT_RETRY_STALE_MS } from "./chat-action-work.js"; import type { ScheduledChatAction } from "./chat-work-notifications.js"; import { notifyChatDeliveryWork, notifyChatPublicationWork, notifyChatEndpointWork, notifyChatActionWork, notifyChatVerificationWork } from "./chat-work-notifications.js"; -import { syncSpekoVoiceTools } from "./voice/speko-agent-tools.js"; -import { configureSpekoSessionTools } from "./voice/speko-tool-setup.js"; + + import { voiceSessionService } from "./voice/voice-session-service.js"; import { createSpekoProvider } from "./voice/speko-provider.js"; import { currentVoiceCredentialFingerprint, voiceSessionStore } from "./voice/voice-session-store.js"; @@ -98,14 +118,12 @@ import { createChatRuntime, type ChatRuntime } from "./chat-runtime.js"; import type { ChatEndpointRuntime } from "./voice/voice-runtime.js"; import { parseMarkdown } from "chat"; import { githubPublicAttachmentsFromMessage } from "./chat-github-attachments.js"; -import { - stageReceiptReactionRemovals, -} from "./chat-receipt-reactions.js"; + import { fastResponseService, enqueueFastResponse, fastResponseSourceCurrent, fastResponseTurnQueued } from "./fast-responses.js"; import { fastResponseRequests } from "@paperclipai/db"; -import { chatCredentialMutationLease, CREDENTIAL_MUTATION_LEASE_TTL_MS, type CredentialMutationLeaseGuard } from "./chat-credential-mutation-lease.js"; +import { chatCredentialMutationLease } from "./chat-credential-mutation-lease.js"; import type { AgentAvatarRequest } from "./agent-avatars.js"; -import { slackChatRegistrationService, slackRegistrationProjection } from "./chat-slack-registration.js"; +import { slackChatRegistrationService } from "./chat-slack-registration.js"; import { chatSlackRegistrations } from "@paperclipai/db"; import { canActorReadIssuePrivacy, canPublishIssueToChatAudience } from "./authorization.js"; @@ -113,15 +131,15 @@ import { withSlackBoardLease } from "./slack-board-lease.js"; import { mirrorSlackBoardComment, slackBoardReplyBindings } from "./slack-board-messages.js"; import { authorizeSlackBoardPublication } from "./slack-board-authority.js"; import { assertSlackBoardWorkAllowed } from "./slack-board-resume.js"; -import { slackExplicitPublicationDuplicate, unresolvedSlackPublicationCondition } from "./connectors/slack-publication.js"; + import { rememberVerifiedSlackSearchEvent } from "./connectors/slack-search-context.js"; -import { slackPublicationAllowed } from "./connectors/slack-access.js"; + import { buildChatCommunicationGuidance } from "./chat-communication-guidance.js"; -import { assertChatExecutionDefaultsAccess, assertChatTaskCreationAccess, resolveChatExecutionDefaults } from "./chat-execution-defaults.js"; +import { assertChatTaskCreationAccess, resolveChatExecutionDefaults } from "./chat-execution-defaults.js"; function githubPolicyRecord(value: unknown): Record { return value && typeof value === "object" && !Array.isArray(value) ? value as Record : {}; } import { githubReviewCheckService } from "./chat-github-checks.js"; @@ -131,20 +149,19 @@ import { githubAutomaticAdmission, githubPreviousAssessment, githubExplicitMentionEvent, - githubMessageAddressesAnotherBot, } from "./chat-github-events.js"; import { githubReviewPrompt, githubManualMessagePrompt, GITHUB_CONFIGURED_SKILL_GUIDANCE } from "./chat-github-review-policy.js"; -import { chatGitHubConfigurations, chatGitHubReviews } from "@paperclipai/db"; +import { chatGitHubReviews } from "@paperclipai/db"; import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubAutomaticEventContext, GitHubReviewPolicy } from "@paperclipai/shared"; import { githubChatReviewService } from "./chat-github-reviews.js"; import { githubResponseCommentService } from "./chat-github-response-comments.js"; -import { githubChatPrincipalAccess } from "./chat-github-access.js"; -import { githubAppJwt } from "./chat-github-client.js"; + + import { resumeExternalConversation } from "./slack-conversation-state.js"; -import { settleExternalConversation } from "./slack-conversation-lifecycle.js"; + import { runtimeCanonicalOrigin } from "./cloud-runtime-identity.js"; import { writePhotonCheckpoint } from "./photon/receiver.js"; @@ -152,18 +169,17 @@ import { PhotonState } from "./photon/state.js"; -import { nativePhotonInteraction, publishPhotonPrompt, type PhotonPromptReceipt, type PhotonInteractionBinding } from "./photon/interactions.js"; -import { PhotonError, photonSharedIdentity, photonSharedScope } from "./photon/cloud.js"; -import { PhotonChatAdapter, photonReplyReference } from "./photon/adapter.js"; -import { photonChannelConfigurationSchema } from "@paperclipai/shared"; -import { defaultSlackAppConfiguration } from "@paperclipai/shared"; + +import { photonReplyReference } from "./photon/adapter.js"; + + + import { createHash, createHmac, - randomBytes, randomUUID, timingSafeEqual, } from "node:crypto"; @@ -172,30 +188,22 @@ import { asc, desc, eq, - gt, inArray, - isNotNull, isNull, - like, - lt, lte, ne, notExists, notInArray, or, sql, - type SQL, } from "drizzle-orm"; -import { alias, type AnyPgColumn } from "drizzle-orm/pg-core"; +import { alias } from "drizzle-orm/pg-core"; import { recordChatWebhookReceipt, recordChatWebhookStage, } from "./chat-webhook-diagnostics.js"; import { - canonicalTeamsThreadId, - externalThreadIdentityCondition, readChatControlChronology, - teamsConversationId, } from "./chat-control-chronology.js"; import { retryChatControlAdmission } from "./chat-control-admission-retry.js"; import type { Db } from "@paperclipai/db"; @@ -217,48 +225,33 @@ import { chatIdentityLinks, chatMessageLinks, chatPublications, - chatSdkState, chatTeamsFileTransfers, companyMemberships, - companySecretBindings, heartbeatRuns, issueComments, issueAttachments, issueThreadInteractions, issues, - toolApplications, toolConnections, } from "@paperclipai/db"; import type { - ChatAdapterCapabilities, - ChatEndpointCallbackSurfaces, ChatEventKind, - ChatEndpoint, ChatEndpointSetupState, ChatProvider, ChatPublicationBatchStatus, - ConfigureChatEndpointInput, - CreateChatEndpointInput, ExternalChannelBindingSummary, SafeChatPublicationPayload, ToolCredentialSecretRef, - UpdateChatEndpointInput, } from "@paperclipai/shared"; import { - isAgentStatusInvokable, isUuidLike, LOW_TRUST_REVIEW_PRESET, LOW_TRUST_REVIEW_PRESET_VERSION, LOW_TRUST_REVIEW_RAW_OUTPUT_DISPOSITION, } from "@paperclipai/shared"; -import { - MAX_ATTACHMENT_BYTES, - normalizeContentType, -} from "../attachment-types.js"; -import { isUniqueViolation } from "../db-errors.js"; -import { - deriveTeamsPersonalRecipient, -} from "./chat-teams-personal-recipient.js"; + + + import { type TeamsFileTransferOptions, } from "./chat-teams-file-transfers.js"; @@ -269,18 +262,11 @@ import { projectChatPublicationBatch, } from "./chat-publication-batches.js"; -import { telegramAttachmentForUpload } from "./chat-telegram-photo.js"; -import { - readTelegramCallbackProvenance, - telegramCallbackThreadId, - TELEGRAM_PRIVATE_ACTION_UNAVAILABLE, -} from "./chat-telegram-ephemeral.js"; -import { normalizeTelegramRichMessage } from "./chat-telegram-rich-intake.js"; -import { - isTelegramDraftStopped, - type TelegramDraftControl, -} from "./chat-telegram-draft-stop.js"; + + + + import { parseChatWebhookPublicBaseUrl } from "../chat-webhook-public-url.js"; @@ -299,20 +285,12 @@ import type { StorageService } from "../storage/types.js"; import { type ChatSdkCallbackEvent, type ChatSdkMessageCallbackEvent, - type ChatSdkMessageUpdatedCallbackEvent, type ChatSdkProvider, type ChatSdkRuntime, type DiscordGatewayCallbackEvent, - type ResolvedChatSdkProviderConfig, - type SlackFileUploadAcceptedReceipt, } from "./chat-sdk-runtime.js"; -import type { - ChatSdkStateCompareAndSetInput, - ChatSdkStateDeleteInput, - ChatSdkStatePersistence, - ChatSdkStateScope, -} from "./chat-sdk-state.js"; + import { logActivity, publishActivity, @@ -324,59 +302,34 @@ import { } from "./issue-assignment-wakeup.js"; import { issueService } from "./issues.js"; import { - authorizeNativeChatReviewPresentation, NativeChatReviewPresentationContentionError, } from "./native-runtime/native-chat-review-presentation.js"; import { isExternalChatWaitAuthorizationContention } from "./native-runtime/chat-attachment-reuse.js"; import { projectSafeChatPublication } from "./chat-publication-projection.js"; -import { safeChatTaskUrl } from "./chat-task-url.js"; -import { - resyncGitHubAppWebhook, -} from "./chat-github-webhook-config.js"; + + import { githubAttachmentLimitOmissions, - restoreGitHubAttachmentLimitOmissions, } from "./chat-github-attachments.js"; import { getExternalChannelBindingSummary } from "./chat-channel-binding.js"; -import { type ChatProviderInventoryResult } from "./chat-provider-inventory.js"; -import { - discordMarkdownRequiresAttachment, -} from "./chat-discord.js"; -import { - renderPublicationTransportText, - splitNativePublicationText, -} from "./chat-publication-text-parts.js"; + + + import { readRegisteredDiscordCommandRegistration, } from "./chat-discord-command-registration-store.js"; -import { - deleteDiscordQuestionFormCorrection, - discordQuestionFormCorrectionModal, - discordQuestionFormDenialResponse, - discordQuestionFormThreadId, - isDiscordQuestionFormCorrectionId, - loadDiscordQuestionFormCorrection, - retainDiscordQuestionFormCorrection, -} from "./chat-discord-question-forms.js"; + import { parseSlackSessionStop, } from "./chat-slack-sessions.js"; -import { - stageSlackSessionSync, -} from "./chat-slack-session-outbox.js"; + import { parseChatProviderLifecycle, type ChatProviderLifecycleEffect, } from "./chat-provider-lifecycle.js"; -import { - enqueueTerminalIssueInteractionChatPublications, - nativeChatQuestion, - nativeTelegramConfirmation, - TELEGRAM_CALLBACK_DATA_LIMIT_BYTES, - telegramChatSdkCallbackData, -} from "./chat-interaction-publications.js"; -import { chatProviderConversationUrl } from "./chat-provider-links.js"; -import { classifyChatPublicationError } from "./chat-publication-errors.js"; + + + import { inboundWakePublicationKey, @@ -384,177 +337,21 @@ import { parseInboundWakePublicationKey, resolveInboundWakeReceipt, } from "./chat-inbound-wakeup-publications.js"; -import { - normalizeMicrosoftTeamsCredentialIds, -} from "./chat-teams-credentials.js"; -import { - prepareTeamsInlineImage, -} from "./chat-teams-inline-images.js"; -import { - shouldStreamSafePublicationText, - splitTelegramPublicationText, - streamSafePublicationText, - telegramMarkdownRequiresAttachment, -} from "./chat-publication-stream.js"; -import { issueThreadInteractionService } from "./issue-thread-interactions.js"; + + + + import { questionResponseDeliveryService, type QuestionResponseDeliveryServiceOptions, } from "./question-response-delivery.js"; -import { - chatQuestionFormDenialResponse, - chatQuestionFormValidationResponse, - claimChatQuestionFormSubmission, - completeChatQuestionFormSubmission, - isChatQuestionFormOpenActionId, - loadChatQuestionFormSubmissionToken, - resolveChatQuestionFormOpen, - validateChatQuestionFormSubmission, -} from "./chat-question-forms.js"; + import { secretService } from "./secrets.js"; import type { - ActionEvent, - AdapterPostableMessage, - Attachment, - Author, - FileUpload, Message, - MessageDeletedEvent, - ModalResponse, - ModalSubmitEvent, SlashCommandEvent, Thread, } from "chat"; -import { Actions, Button, Card, CardText, LinkButton } from "chat"; - -const PROVIDER_LABELS: Record = { - speko: "Speko", - "imessage-photon": "iMessage Photon", - agentmail: "AgentMail", - slack: "Slack", - github: "GitHub", - discord: "Discord", - "microsoft-teams": "Microsoft Teams", - telegram: "Telegram", -}; - -type PublicationOrderTable = { - createdAt: AnyPgColumn; - id: AnyPgColumn; - payload: AnyPgColumn; -}; - -function publicationTransportOrderKey(table: PublicationOrderTable) { - return sql`coalesce(${table.payload}->'transportPart'->>'orderKey', ${table.id}::text)`; -} - -function publicationPrecedes( - earlier: PublicationOrderTable, - later: PublicationOrderTable, -) { - return or( - lt(earlier.createdAt, later.createdAt), - and( - eq(earlier.createdAt, later.createdAt), - lt( - publicationTransportOrderKey(earlier), - publicationTransportOrderKey(later), - ), - ), - ); -} - -const CAPABILITIES: Record = { - speko: { threads: false, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: false, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false }, - "imessage-photon": { threads: false, directMessages: true, nativeStreaming: false, messageEdits: true, messageDeletes: false, reactions: false, files: true, cards: false, actions: true, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false }, - agentmail: { threads: true, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: true, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: true }, - slack: { - threads: true, - directMessages: true, - nativeStreaming: true, - messageEdits: true, - messageDeletes: true, - reactions: true, - files: true, - cards: true, - actions: true, - modals: true, - slashCommands: true, - ephemeralMessages: true, - proactiveDirectMessages: false, - }, - github: { - threads: true, - directMessages: false, - nativeStreaming: false, - messageEdits: true, - messageDeletes: true, - reactions: true, - files: false, - cards: true, - actions: false, - modals: false, - slashCommands: false, - ephemeralMessages: false, - proactiveDirectMessages: false, - }, - discord: { - threads: true, - directMessages: true, - nativeStreaming: false, - messageEdits: true, - messageDeletes: true, - reactions: true, - files: true, - cards: true, - actions: true, - modals: true, - // Automatic registration upgrades these only after a durable provider - // receipt. A draft or an uncertain/conflicting registration stays false. - slashCommands: false, - ephemeralMessages: false, - proactiveDirectMessages: false, - }, - "microsoft-teams": { - threads: true, - directMessages: true, - // Production webhook processing is deferred into Paperclip's durable - // ingress queue. The Teams adapter's request-scoped DM streamer is gone by - // the time agent output is published, so advertise the durable behavior we - // can actually provide. editMessage still lets one run coalesce its - // queued, working, and final states in place. - nativeStreaming: false, - // Paperclip supplements the pinned adapter's missing inbound Bot Framework - // messageUpdate/messageDelete dispatch after provider authentication. - messageEdits: true, - messageDeletes: true, - reactions: true, - files: true, - cards: true, - actions: true, - modals: true, - slashCommands: false, - ephemeralMessages: true, - proactiveDirectMessages: false, - }, - telegram: { - threads: true, - directMessages: true, - nativeStreaming: true, - messageEdits: true, - // Telegram's Bot API does not emit an update when a user deletes a - // message, so this capability cannot be offered truthfully. - messageDeletes: false, - reactions: true, - files: true, - cards: true, - actions: true, - modals: false, - slashCommands: false, - ephemeralMessages: false, - proactiveDirectMessages: false, - }, -}; const REQUIRED_CREDENTIALS: Record< Exclude, @@ -590,7 +387,6 @@ const SUPPLIED_CREDENTIAL_KEYS: Record = { }; const MAX_ERROR_TEXT = 2_000; -const DELIVERY_PROCESSING_STALE_MS = 60_000; // A Discord root mention is durably staged before the adapter creates its // provider thread. Give the bounded provider retry loop ample time to finish; // if the process disappears, the delivery worker verifies that thread over @@ -650,133 +446,9 @@ function credentialFingerprint(refs: ToolCredentialSecretRef[]): string { ); return createHash("sha256").update(JSON.stringify(stable)).digest("hex"); } - -const ATTACHMENT_FAILURE_NOTICE_PREFIX = "attachment-failure-notice:"; - -function attachmentFailureNoticeFence( - idempotencyKey: string, -): LifecycleRuntimeFence | null { - const match = - /^attachment-failure-notice:[0-9a-f-]{36}:(\d+):([0-9a-f]{64})$/i.exec( - idempotencyKey, - ); - if (!match) return null; - const generation = Number(match[1]); - if (!Number.isSafeInteger(generation) || generation < 0) return null; - return { generation, credentialFingerprint: match[2]!.toLowerCase() }; -} - -function definiteAttachmentProviderRejection(error: unknown): boolean { - const pending: Array<{ depth: number; value: unknown }> = [ - { depth: 0, value: error }, - ]; - const seen = new Set(); - const statuses: number[] = []; - const names: string[] = []; - const codes: string[] = []; - while (pending.length > 0) { - const current = pending.shift(); - if ( - !current || - current.depth > 4 || - !current.value || - typeof current.value !== "object" || - seen.has(current.value) - ) { - continue; - } - seen.add(current.value); - const record = current.value as { - cause?: unknown; - code?: unknown; - details?: { code?: unknown; providerStatus?: unknown }; - innerHttpError?: { statusCode?: unknown }; - name?: unknown; - original?: unknown; - originalError?: unknown; - response?: { status?: unknown }; - status?: unknown; - statusCode?: unknown; - }; - for (const status of [ - record.status, - record.statusCode, - record.response?.status, - record.details?.providerStatus, - record.innerHttpError?.statusCode, - ]) { - if (typeof status === "number" && Number.isInteger(status)) { - statuses.push(status); - } - } - if (typeof record.name === "string") names.push(record.name); - for (const code of [record.code, record.details?.code]) { - if (typeof code === "string") codes.push(code); - } - for (const nested of [ - record.cause, - record.original, - record.originalError, - ]) { - pending.push({ depth: current.depth + 1, value: nested }); - } - } - if ( - names.some((name) => - ["ValidationError", "NotImplementedError"].includes(name), - ) || - codes.some((code) => - [ - "CHAT_ADAPTER_COMPATIBILITY_ERROR", - "CHAT_PROVIDER_PRETRANSPORT_REJECTED", - "NOT_IMPLEMENTED", - "VALIDATION_ERROR", - ].includes(code), - ) - ) { - return false; - } - // Slack's Web API can reject a file operation with `ok: false` in an HTTP - // 200 response. The SDK's structured platform-error code is still a - // definitive provider rejection; the main disposition classifier has - // already excluded auth, destination, and rate-limit outcomes. - if (codes.includes("slack_webapi_platform_error")) return true; - return statuses.some( - (status) => - status >= 400 && - status < 500 && - ![401, 403, 404, 410, 429].includes(status), - ); -} - -function attachmentFailureKind( - payload: SafeChatPublicationPayload, -): - | { kind: "generated_response"; provider: "discord" | "telegram" } - | { kind: "selected_file"; provider: null } - | null { - if (payload.transportPart?.mode === "discord_markdown_attachment") { - return { kind: "generated_response", provider: "discord" }; - } - if (payload.transportPart?.mode === "telegram_markdown_attachment") { - return { kind: "generated_response", provider: "telegram" }; - } - if (payload.attachmentIds?.length) { - return { kind: "selected_file", provider: null }; - } - return null; -} type ConversationRow = typeof chatConversations.$inferSelect; type DeliveryRow = typeof chatDeliveries.$inferSelect; -type LiveInboundMessage = { - endpoint: EndpointRow; - message: Message; - receiptReactionSupported: boolean; - providerUrl: string | null; - runtimeContext?: InboundRuntimeContext; - thread: Thread; - trigger: ChatSdkMessageCallbackEvent["trigger"]; -}; + export interface ChatChannelServiceOptions { /** @@ -903,165 +575,6 @@ export interface ChatChannelServiceOptions { storage?: StorageService; } -type DiscordGatewayOwnership = { - companyId: string; - context: RuntimeContext; - endpointId: string; - expiresAt: Date; - leaseKey: typeof DISCORD_GATEWAY_LEASE_KEY | "photon_receiver_runtime"; - renewTimer: ReturnType | null; - renewal: Promise | null; - stopPromise: Promise | null; - stopping: boolean; - token: string; -}; - -function iso(value: Date | null | undefined): string | null { - return value ? value.toISOString() : null; -} - -function providerResourceType( - provider: ChatProvider, - surfaceKind: ChatSurfaceKind, -): string { - if (surfaceKind === "direct_message") return "direct_message"; - if (provider === "github") return "repository"; - if (provider === "imessage-photon") return "group_chat"; - if (provider === "discord") return "channel"; - if (provider === "microsoft-teams") - return surfaceKind === "linear_group" ? "group_chat" : "channel"; - // Telegram topics are task/thread boundaries inside one group resource. The - // operator grants access to the chat once rather than having to rediscover - // and enable every topic independently. - if (provider === "telegram") return "chat"; - return "channel"; -} - -type ChatSurfaceKind = "direct_message" | "linear_group" | "native_thread"; - - -function sameTeamsThreadIdentity(left: string, right: string): boolean { - const leftConversationId = teamsConversationId(left); - const rightConversationId = teamsConversationId(right); - return ( - leftConversationId !== null && - rightConversationId !== null && - leftConversationId === rightConversationId - ); -} - - -function durableExternalThreadIdentity(threadId: string): string { - return canonicalTeamsThreadId(threadId) ?? threadId; -} - - -function microsoftTeamsServiceUrl(raw: unknown): string | null { - if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; - const value = (raw as { serviceUrl?: unknown }).serviceUrl; - return typeof value === "string" && value ? value : null; -} - -function teamsThreadRootMessageId(threadId: string): string | null { - const conversationId = teamsConversationId(threadId); - if (!conversationId) return null; - return /;messageid=([^;]+)/i.exec(conversationId)?.[1] ?? null; -} - -function canonicalProviderResourceId( - provider: ChatProvider, - thread: Pick, -): string { - if (provider === "slack") return thread.channelId.replace(/^slack:/, ""); - if (provider === "github") - return thread.channelId.replace(/^github:/, "").toLowerCase(); - if (provider === "discord") { - const parts = thread.channelId.split(":"); - return parts[2] ?? thread.channelId; - } - if (provider === "microsoft-teams") { - const conversationId = - teamsConversationId(thread.channelId) ?? teamsConversationId(thread.id); - return conversationId - ? baseTeamsConversationId(conversationId) - : baseTeamsConversationId(thread.channelId); - } - if (provider === "telegram") - return thread.channelId.replace(/^telegram:/, ""); - return thread.channelId; -} - -function providerResourceLabelFromThread( - provider: ChatProvider, - thread: Pick, -): { label: string; fallback: boolean } { - const providerResourceId = canonicalProviderResourceId(provider, thread); - const candidate = thread.channel.name?.trim(); - if (!candidate) return { label: providerResourceId, fallback: true }; - if ( - (provider === "slack" && - slackResourceLabelIsFallback(providerResourceId, candidate)) || - (provider === "telegram" && - telegramResourceLabelIsFallback(providerResourceId, candidate)) - ) { - return { label: providerResourceId, fallback: true }; - } - return { label: candidate, fallback: false }; -} - -/** - * Native channel threads keep one issue forever. Linear surfaces instead - * advance through Paperclip task generations because their provider id is - * reused after a task completes or the user explicitly starts a new task. - */ -function chatSurfaceKind( - provider: ChatProvider, - thread: Thread, -): ChatSurfaceKind { - if (thread.isDM) return "direct_message"; - if (provider === "imessage-photon") return "linear_group"; - if (provider === "telegram") { - return /^telegram:[^:]+:[^:]+$/.test(thread.id) - ? "native_thread" - : "linear_group"; - } - if (provider === "microsoft-teams") { - const conversationId = teamsConversationId(thread.id); - return conversationId?.includes(";messageid=") - ? "native_thread" - : "linear_group"; - } - return "native_thread"; -} - -/** - * Teams group chats are admitted as a class: installation at the provider and - * the endpoint's group-chat toggle are the two gates the operator can - * actually control. Teams channels, and every other non-DM destination, - * retain the explicit per-resource enablement gate shown in Settings. - */ -function nonDirectDestinationAllowed( - endpoint: EndpointRow, - resource: ResourceRow | null | undefined, -): boolean { - if (!resource || resource.availability !== "available") return false; - if ( - endpoint.provider === "microsoft-teams" && - resource.type === "group_chat" - ) { - return endpoint.allowGroupChats; - } - return resource.enabled; -} - -function linearControlCommand(text: string): "new" | "close" | "status" | null { - const match = /^\/(new|close|status)(?:@[\w.-]+)?\s*$/i.exec(text.trim()); - return ( - (match?.[1]?.toLowerCase() as "new" | "close" | "status" | undefined) ?? - null - ); -} - function safeTitle(text: string, fallback: string): string { const line = text .replace(/<@[A-Z0-9]+>/gi, "") @@ -1071,22 +584,6 @@ function safeTitle(text: string, fallback: string): string { return (line || fallback).slice(0, 160); } -function actionThreadMatchesConversation( - provider: ChatProvider, - actionThreadId: string, - conversationThreadId: string, -): boolean { - if (provider === "microsoft-teams") { - return sameTeamsThreadIdentity(actionThreadId, conversationThreadId); - } - if (provider !== "slack") return actionThreadId === conversationThreadId; - const actionChannelId = slackThreadChannelId(actionThreadId); - return ( - actionChannelId !== null && - actionChannelId === slackThreadChannelId(conversationThreadId) - ); -} - function sanitizeFilename(value: string | undefined): string | null { if (!value) return null; const leaf = value.replaceAll("\\", "/").split("/").pop()?.trim(); @@ -1105,57 +602,6 @@ function redactError(error: unknown): string { return redactSensitiveText(withoutTelegramBotTokens).slice(0, MAX_ERROR_TEXT); } -async function attemptProviderPublication( - send: () => Promise, -): Promise { - return await send(); -} - -async function editOrPostProviderPublication( - edit: () => Promise, - post: () => Promise, -): Promise { - try { - return await edit(); - } catch (error) { - // A definite 404 proves the old progress comment no longer exists. It is - // therefore safe to create a replacement without risking a duplicate. - // Ambiguous transport failures must still enter delivery_unknown. - if (classifyChatPublicationError(error, 1).kind !== "resource_unavailable") - throw error; - return await post(); - } -} - -function safeCardForPublication( - payload: SafeChatPublicationPayload, - provider: ChatProvider, -) { - if (!payload.card) return null; - const children = []; - if (payload.card.body) children.push(CardText(payload.card.body)); - const actions = (payload.card.actions ?? []).map((action) => - action.type === "callback" - ? Button({ - id: action.actionId, - label: action.label, - style: action.style ?? "default", - // Telegram encodes both id and value into callback_data, whose hard - // limit is 64 bytes. Its opaque action id resolves through the - // durable token ledger, so repeating the interaction UUID is both - // unnecessary and too large. Other adapters retain the value as a - // defense-in-depth binding supported by their larger envelopes. - value: provider === "telegram" ? undefined : payload.interactionId, - actionType: isChatQuestionFormOpenActionId(action.actionId) - ? "modal" - : undefined, - }) - : LinkButton({ label: action.label, url: action.url }), - ); - if (actions.length) children.push(Actions(actions)); - return Card({ title: payload.card.title, children }); -} - function absoluteBaseUrl(value: string | null | undefined): string | null { if (!value) return null; try { @@ -1182,187 +628,6 @@ function isSupportedTelegramWebhookBaseUrl(value: string | null): boolean { } } -function providerSetupState( - endpoint: Pick< - EndpointRow, - | "provider" - | "providerAccountId" - | "publicId" - | "status" - | "setup" - | "botUsername" - >, - publicBaseUrl: string | null, - assignedAgentName?: string | null, -) { - const path = endpoint.provider === "speko" ? `/api/voice-webhooks/${endpoint.publicId}/tools` : `/api/chat-webhooks/${endpoint.publicId}/${endpoint.provider}`; - const webhookUrl = publicBaseUrl ? `${publicBaseUrl}${path}` : null; - const step = endpoint.status === "active" ? "complete" : endpoint.setup.step; - switch (endpoint.provider) { - case "speko": - return { step, providerUrl: "https://platform.speko.ai/", webhookUrl } as const; - case "imessage-photon": return { step, providerUrl: "https://photon.codes/", testStartedAt: endpoint.setup.testStartedAt } as const; - case "agentmail": return endpoint.setup; - case "slack": { - const observations = (endpoint.setup as InternalSetupState) - .slackCallbackSurfaces; - const currentUrl = webhookUrl ? canonicalCallbackUrl(webhookUrl) : null; - const surfaceState = ( - surface: SlackCallbackSurface, - ): ChatEndpointCallbackSurfaces[SlackCallbackSurface] => { - const observation = observations?.[surface]; - if (!observation) return { status: "unverified" }; - return { - status: - currentUrl !== null && - slackCallbackMatchesPublicUrl(observation.url, currentUrl) - ? "current" - : "stale", - observedAt: observation.observedAt, - }; - }; - const callbackSurfaces: ChatEndpointCallbackSurfaces = { - events: surfaceState("events"), - interactivity: surfaceState("interactivity"), - slashCommands: surfaceState("slashCommands"), - }; - return { - step, - testStartedAt: endpoint.setup.testStartedAt ?? null, - testSkipped: endpoint.setup.testSkipped ?? false, - authorizationUrl: "https://api.slack.com/apps?new_app=1", - providerUrl: "https://app.slack.com/", - webhookUrl, - webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null, - callbackSurfaces, - callbacksNeedUpdate: Object.values(callbackSurfaces).some( - (surface) => surface.status === "stale", - ), - slackApp: endpoint.setup.slackApp, - slackSetupMethod: endpoint.setup.slackSetupMethod, - slackAvatar: endpoint.setup.slackAvatar, - slackManagerError: endpoint.setup.slackManagerError, - slackAccount: endpoint.setup.slackAccount, - // Slack registers the slash command in the provider configuration. - // Keep that identity immutable when the assigned agent is renamed; - // deriving it remains only a compatibility path for older rows. - command: - typeof endpoint.setup.command === "string" - ? endpoint.setup.command - : assignedAgentName - ? slackCommandForAgent(assignedAgentName, endpoint.publicId) - : null, - } as const; - } - case "github": - return { - step, - github: endpoint.setup.github, - testStartedAt: endpoint.setup.testStartedAt, - testSkipped: endpoint.setup.testSkipped, - authorizationUrl: "https://github.com/settings/apps/new", - providerUrl: "https://github.com/settings/installations", - webhookUrl, - webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null, - } as const; - case "discord": - return { - step, - authorizationUrl: "https://discord.com/developers/applications", - providerUrl: endpoint.providerAccountId - ? `https://discord.com/channels/${encodeURIComponent(endpoint.providerAccountId)}` - : "https://discord.com/channels/@me", - webhookUrl, - } as const; - case "microsoft-teams": - return { - step, - authorizationUrl: "https://dev.teams.microsoft.com/apps", - providerUrl: "https://teams.microsoft.com/", - messagingEndpoint: webhookUrl, - } as const; - case "telegram": - return { - step, - providerUrl: endpoint.botUsername - ? `https://t.me/${encodeURIComponent(endpoint.botUsername)}` - : "https://t.me/BotFather", - webhookUrl, - } as const; - } -} - -export function createChatSdkStatePersistence(db: Db): ChatSdkStatePersistence { - return { - async read(scope: ChatSdkStateScope, key: string) { - return db - .select({ - value: chatSdkState.value, - version: chatSdkState.version, - expiresAt: chatSdkState.expiresAt, - }) - .from(chatSdkState) - .where( - and( - eq(chatSdkState.companyId, scope.companyId), - eq(chatSdkState.endpointId, scope.endpointId), - eq(chatSdkState.stateKey, key), - ), - ) - .then((rows) => rows[0] ?? null); - }, - async compareAndSet(input: ChatSdkStateCompareAndSetInput) { - if (input.expectedVersion === null) { - const inserted = await db - .insert(chatSdkState) - .values({ - companyId: input.companyId, - endpointId: input.endpointId, - stateKey: input.key, - version: 1, - value: input.value, - expiresAt: input.expiresAt, - }) - .onConflictDoNothing() - .returning({ id: chatSdkState.id }); - return inserted.length === 1; - } - const updated = await db - .update(chatSdkState) - .set({ - version: input.expectedVersion + 1, - value: input.value, - expiresAt: input.expiresAt, - updatedAt: new Date(), - }) - .where( - and( - eq(chatSdkState.companyId, input.companyId), - eq(chatSdkState.endpointId, input.endpointId), - eq(chatSdkState.stateKey, input.key), - eq(chatSdkState.version, input.expectedVersion), - ), - ) - .returning({ id: chatSdkState.id }); - return updated.length === 1; - }, - async deleteIfVersion(input: ChatSdkStateDeleteInput) { - const deleted = await db - .delete(chatSdkState) - .where( - and( - eq(chatSdkState.companyId, input.companyId), - eq(chatSdkState.endpointId, input.endpointId), - eq(chatSdkState.stateKey, input.key), - eq(chatSdkState.version, input.expectedVersion), - ), - ) - .returning({ id: chatSdkState.id }); - return deleted.length === 1; - }, - }; -} - export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { const runtime = options.runtime ?? createChatRuntime(); const githubManualMessages = new WeakMap(); @@ -1385,10 +650,11 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { version: string; } >(); + const { recordCurrentMicrosoftTeamsRoute } = createTeamsRoutes(db, { runtime, runtimeCallbackEndpoint }); const persistence = createChatSdkStatePersistence(db); const fetchImpl = options.fetch ?? globalThis.fetch; const { inspectPhotonCredentials, verifyPhotonCredentials } = createPhotonCredentialChecks(fetchImpl); - const { hydrateSlackResourceLabel } = createSlackInventory(db, { endpointRecord, resolveCredentials, fetchImpl }); + const { hydrateSlackResourceLabel } = createSlackInventory(db, { endpointRecord, resolveCredentials: (...args) => resolveCredentials(...args), fetchImpl }); const configuredPublicBaseUrl = absoluteBaseUrl(options.publicBaseUrl); const configuredWebhookPublicBaseUrl = parseChatWebhookPublicBaseUrl(options.webhookPublicBaseUrl); // A warm Cloud instance is constructed before it receives its signed claim. @@ -1404,6 +670,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { const getTaskBaseUrl = () => runtimeCanonicalOrigin() ?? options.publicBaseUrl; const getWebhookPublicBaseUrl = () => configuredWebhookPublicBaseUrl ?? getPublicBaseUrl(); const voiceStore = voiceSessionStore(db, { allowLocalBoard: options.allowLocalVoiceBoard === true }); + const { findAuthorizedIdentityLink, ensurePrincipal, sponsorAllowsGuest, lockCurrentPrincipalAuthorization, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, ensureResource } = createChatInboundAuthority(db, { voiceStore, runtimeCallbackEndpoint }); const voice = voiceSessionService(db, { allowLocalBoard: options.allowLocalVoiceBoard === true, credentials: async (endpoint, expectedFingerprint) => { @@ -1421,6 +688,11 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { const issuesSvc = issueService(db); const { runIdFromMilestonePublication, runOwnershipMilestoneSupersessionReason, runPublicationToReplace, receiptReactionCompletionRunId, inboundWakePublicationToReplace, interactionResolutionPublicationToReplace, interactionPromptPublicationToReplace, progressSupersededByFastResponse, runProgressSupersededByPublishedInteraction, taskStatusPublicationToReplace, closeProgressToReplace } = createChatPublicationReplacement(db, { endpointRecord, authorizeRetainedChatSourcePublication }); const secrets = secretService(db); + const { normalizedCredentials, verifyCredentials, nativeBotIdentityMatches, legacyGitHubLabelsMatchVerifiedCredentials, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, persistCredentials, clearCredentials, resolveCredentialRefs, resolveCredentials, prepareProviderInventory } = createChatCredentials(db, { + secrets, PROVIDER_LABELS, REQUIRED_CREDENTIALS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, + fetchImpl, redactError, credentialFingerprint, verifyPhotonCredentials, + }); + const { inspectPhoton } = createPhotonSetupInspection(db, { endpointRecord, inspectPhotonCredentials }); const questionResponses = questionResponseDeliveryService(db, { heartbeat: options.heartbeat as QuestionResponseDeliveryServiceOptions["heartbeat"], @@ -2051,34 +1323,6 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { : null; } - async function recordCurrentMicrosoftTeamsRoute( - endpoint: EndpointRow, - context: InboundRuntimeContext | undefined, - threadId: string, - raw: unknown, - recordAcceptedActivity = true, - ): Promise { - if (endpoint.provider !== "microsoft-teams" || !context?.endpointRuntime) { - return; - } - const serviceUrl = microsoftTeamsServiceUrl(raw); - if (!serviceUrl) return; - await db.transaction(async (tx) => { - const current = await runtimeCallbackEndpoint(tx, endpoint.id, context, [ - "verifying", - "active", - ]); - if (!current || runtime.get(endpoint.id) !== context.endpointRuntime) { - return; - } - await context.endpointRuntime.recordMicrosoftTeamsRoute( - threadId, - serviceUrl, - recordAcceptedActivity ? raw : undefined, - ); - }); - } - function discordGatewayRuntimeIsCurrent( endpointId: string, context: RuntimeContext, @@ -2095,762 +1339,12 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { ); } - function serializeEndpoint( - row: Awaited>, - ): ChatEndpoint { - if (!row) throw notFound("Chat endpoint not found"); - const endpoint = row.endpoint; - return { - id: endpoint.id, - companyId: endpoint.companyId, - connectionId: endpoint.connectionId, - publicationMode: endpoint.publicationMode, - externalExecutionPolicy: endpoint.externalExecutionPolicy, - provider: endpoint.provider, - publicId: endpoint.publicId, - status: endpoint.status, - deploymentMode: endpoint.deploymentMode, - assignedAgentId: endpoint.assignedAgentId, - assignedAgentName: row.assignedAgentName, - sponsorUserId: endpoint.sponsorUserId, - providerAccountId: endpoint.providerAccountId, - providerAccountLabel: endpoint.providerAccountLabel, - botExternalId: endpoint.botExternalId, - botUsername: endpoint.botUsername, - botLabel: endpoint.botDisplayName ?? row.assignedAgentName, - botAvatarUrl: endpoint.botAvatarUrl, - communicationInstructions: endpoint.communicationInstructions, - executionDefaults: endpoint.executionDefaults, - ...(endpoint.provider === "imessage-photon" && endpoint.botExternalId ? { photonAllocation: endpoint.botExternalId.startsWith("photon-project:") ? "shared" as const : "dedicated" as const } : {}), - allowDirectMessages: endpoint.allowDirectMessages, - requireAtMention: endpoint.requireAtMention, - allowGroupChats: endpoint.allowGroupChats, - allowUnlinkedPeople: endpoint.allowUnlinkedPeople, - replyMode: "subscribed", - capabilities: endpoint.capabilities, - setup: { - ...providerSetupState( - endpoint, - getWebhookPublicBaseUrl(), - row.assignedAgentName, - ), - ...(endpoint.provider === "github" - ? { - webhookSecretConfigured: row.credentialSecretRefs.some( - (ref) => ref.configPath === "credentials.webhookSecret", - ), - } - : endpoint.provider === "speko" ? { - webhookSecretConfigured: row.credentialSecretRefs.some( - (ref) => ref.configPath === "credentials.signingSecret", - ), - } : {}), - }, - healthMessage: endpoint.healthMessage, - lastError: endpoint.lastError, - lastActivityAt: iso(endpoint.lastEventAt), - lastPublicationAt: iso(endpoint.lastPublicationAt), - activatedAt: iso(endpoint.activatedAt), - createdAt: endpoint.createdAt.toISOString(), - updatedAt: endpoint.updatedAt.toISOString(), - }; - } - - async function list(companyId: string) { - const rows = await db - .select({ - endpoint: chatEndpoints, - assignedAgentName: agents.name, - connectionName: toolConnections.name, - applicationId: toolConnections.applicationId, - credentialSecretRefs: toolConnections.credentialSecretRefs, - }) - .from(chatEndpoints) - .innerJoin( - agents, - and( - eq(agents.id, chatEndpoints.assignedAgentId), - eq(agents.companyId, chatEndpoints.companyId), - ), - ) - .innerJoin( - toolConnections, - and( - eq(toolConnections.id, chatEndpoints.connectionId), - eq(toolConnections.companyId, chatEndpoints.companyId), - ), - ) - .where( - and( - eq(chatEndpoints.companyId, companyId), - ne(chatEndpoints.status, "archived"), - ), - ) - .orderBy(desc(chatEndpoints.updatedAt)); - return rows.map((row) => serializeEndpoint(row)); - } - - async function get(endpointId: string) { - const result = serializeEndpoint(await endpointRecord(endpointId)); - if (result.provider === "slack") { - const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId)); - result.setup.slackRegistration = registration ? slackRegistrationProjection(registration) : undefined; - const publicOrigin = getPublicBaseUrl(); - result.setup.slackOAuthCallbackUri = publicOrigin ? `${publicOrigin}/api/chat-slack/oauth/callback` : null; - } - return result; - } - - async function create( - companyId: string, - input: CreateChatEndpointInput, - actorUserId?: string | null, - ) { - if ((input.provider as string) === "agentmail") throw badRequest("Use the email inbox setup API for AgentMail"); - if ((input.slackApp || input.slackSetupMethod) && input.provider !== "slack") throw unprocessable("Slack app details only apply to Slack connections"); - if (input.slackSetupMethod === "managed" && slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable"); - const agent = await db - .select({ id: agents.id, name: agents.name, status: agents.status }) - .from(agents) - .where( - and( - eq(agents.companyId, companyId), - eq(agents.id, input.assignedAgentId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!agent) - throw unprocessable("The selected agent does not belong to this company"); - if (!isAgentStatusInvokable(agent.status)) { - throw unprocessable( - "The selected agent must be active before it can be connected to chat", - { - code: "chat_agent_not_invokable", - }, - ); - } - - const endpointId = randomUUID(); - const publicId = randomBytes(32).toString("base64url"); - const applicationId = input.applicationId ?? randomUUID(); - const connectionId = randomUUID(); - const suffix = endpointId.slice(0, 8); - const name = ( - input.name?.trim() || `${PROVIDER_LABELS[input.provider]} — ${agent.name}` - ).slice(0, 145); - await db.transaction(async (tx) => { - if (input.applicationId) { - const application = await tx - .select({ - id: toolApplications.id, - applicationKey: toolApplications.applicationKey, - metadata: toolApplications.metadata, - status: toolApplications.status, - }) - .from(toolApplications) - .where( - and( - eq(toolApplications.companyId, companyId), - eq(toolApplications.id, input.applicationId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!application) throw notFound("App not found"); - const sourceTemplateKey = - typeof application.metadata.sourceTemplateKey === "string" - ? application.metadata.sourceTemplateKey - : null; - if ( - application.status === "archived" || - (application.applicationKey !== input.provider && - sourceTemplateKey !== input.provider) - ) { - throw unprocessable( - `The selected App is not the ${PROVIDER_LABELS[input.provider]} connector`, - ); - } - } else { - await tx.insert(toolApplications).values({ - id: applicationId, - companyId, - applicationKey: `chat:${input.provider}:${endpointId}`, - name: `${name} ${suffix}`, - description: `People talk to ${agent.name} through ${PROVIDER_LABELS[input.provider]}.`, - type: "chat", - status: "draft", - metadata: { sourceTemplateKey: input.provider, purpose: "channel" }, - ownerUserId: actorUserId ?? null, - }); - } - await tx.insert(toolConnections).values({ - id: connectionId, - companyId, - applicationId, - name, - uid: `chat-${input.provider}-${endpointId}`, - connectionKind: "managed", - connectionPurpose: "channel", - ownership: "customer", - transport: input.provider === "speko" ? "voice" : "chat_sdk", - authKind: "api_key", - credentialPolicy: "shared", - status: "draft", - enabled: false, - config: { provider: input.provider }, - transportConfig: {}, - createdByUserId: actorUserId ?? null, - }); - await tx.insert(chatEndpoints).values({ - id: endpointId, - companyId, - connectionId, - provider: input.provider, - publicId, - assignedAgentId: agent.id, - sponsorUserId: actorUserId ?? null, - // Group chat is a broad Teams reach grant rather than a discovered - // destination toggle. Keep it closed until the operator explicitly - // enables that surface, even when this process is running against a - // database created before the column default was hardened. - requireAtMention: input.provider === "slack", - allowGroupChats: !["microsoft-teams", "speko"].includes(input.provider), - allowUnlinkedPeople: !["slack", "imessage-photon", "speko"].includes(input.provider), - capabilities: CAPABILITIES[input.provider], - setup: { - step: "provider_setup", - ...(input.provider === "slack" - ? { slackApp: input.slackApp ?? defaultSlackAppConfiguration(agent.name), - command: (input.slackApp ?? defaultSlackAppConfiguration(agent.name)).command, slackSetupMethod: input.slackSetupMethod ?? slackRegistration.managerGrants.defaultMethod() } - : {}), - }, - }); - }); - await logActivity(db, { - companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action: "chat_endpoint.created", - entityType: "tool_connection", - entityId: connectionId, - details: { - endpointId, - provider: input.provider, - assignedAgentId: agent.id, - }, - }); - return get(endpointId); - } - - async function update( - endpointId: string, - input: UpdateChatEndpointInput, - actorUserId?: string | null, - ) { - const initial = await endpointRecord(endpointId); - if (!initial) throw notFound("Chat endpoint not found"); - if (initial.endpoint.provider === "agentmail" && Object.keys(input).some((key) => key !== "executionDefaults")) throw badRequest("Use the email inbox API for AgentMail"); - await withCredentialMutationLease( - initial.endpoint, - async (credentialLease) => { - // Reach changes share the provider-transport lane with publications. - // Once a disable returns, no send that sampled the previous grant can - // still begin or finish behind it. - const existing = await endpointRecord(endpointId); - if (!existing) throw notFound("Chat endpoint not found"); - const values: Partial = { - updatedAt: new Date(), - }; - if (input.communicationInstructions !== undefined) { - if (existing.endpoint.provider !== "slack") { - throw unprocessable("Communication instructions are currently supported for Slack connections"); - } - values.communicationInstructions = input.communicationInstructions; - } - if (input.slackSetupMethod === "managed" && slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable"); - if (input.slackSetupMethod && (existing.endpoint.provider !== "slack" || existing.endpoint.botExternalId || existing.endpoint.status !== "draft")) { - throw conflict("Setup method can only change before connecting the app"); - } - if (input.slackApp || input.slackSetupMethod) { - const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId)); - if (registration && (registration.managerGrantId || registration.status !== "failed")) { - if (registration.managerGrantId || input.slackSetupMethod === "managed" || input.slackApp || input.slackSetupMethod === "automatic") throw conflict("Slack app details are locked after creation starts"); - // Manual recovery fences pending authorization under the same credential lease. - await slackRegistration.cleanup(endpointId, credentialLease); - } - } - if (input.slackSetupMethod) { - values.setup = { ...existing.endpoint.setup, slackSetupMethod: input.slackSetupMethod }; - } - if (input.slackApp) { - if (existing.endpoint.provider !== "slack") { - throw unprocessable("Slack app details only apply to Slack connections"); - } - if (existing.endpoint.status !== "draft" || existing.endpoint.botExternalId) { - throw conflict("Slack app details can only be edited before connecting the app"); - } - values.setup = { - ...(values.setup ?? existing.endpoint.setup), - slackApp: input.slackApp, - command: input.slackApp.command, - }; - } - if (input.allowDirectMessages !== undefined) - values.allowDirectMessages = input.allowDirectMessages; - if (input.requireAtMention !== undefined) { - if (existing.endpoint.provider !== "slack") - throw unprocessable("Require at-mention only applies to Slack connections"); - values.requireAtMention = input.requireAtMention; - } - if (input.allowGroupChats && existing.endpoint.provider === "imessage-photon" && (!existing.endpoint.botExternalId || existing.endpoint.botExternalId.startsWith("photon-project:"))) - throw unprocessable("Photon shared channels support direct messages only; groups require a dedicated channel"); - if (input.allowGroupChats !== undefined) - values.allowGroupChats = input.allowGroupChats; - if (input.allowUnlinkedPeople !== undefined) - values.allowUnlinkedPeople = input.allowUnlinkedPeople; - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - if (input.executionDefaults !== undefined) { - await assertChatExecutionDefaultsAccess(db, { - companyId: existing.endpoint.companyId, - assigneeAgentId: values.assignedAgentId ?? existing.endpoint.assignedAgentId, - actor: { type: "board", userId: actorUserId, source: actorUserId ? "session" : "local_implicit" }, - defaults: input.executionDefaults, - }, tx); - values.executionDefaults = input.executionDefaults; - } - await tx - .update(chatEndpoints) - .set(values) - .where(eq(chatEndpoints.id, endpointId)); - await logActivity(tx as unknown as Db, { - companyId: existing.endpoint.companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action: "chat_endpoint.updated", - entityType: "tool_connection", - entityId: existing.endpoint.connectionId, - details: { endpointId, fields: Object.keys(input) }, - }); - await credentialLease.assertOwned(tx); - }); - }, - ); - return get(endpointId); - } - - async function normalizedCredentials( - endpoint: EndpointRow, - supplied: Record | undefined, - ) { - const values = await resolveCredentials(endpoint).catch( - () => ({}) as Record, - ); - for (const [key, rawValue] of Object.entries(supplied ?? {})) { - if (typeof rawValue === "string" && rawValue.trim()) - values[key] = key === "privateKey" ? rawValue : rawValue.trim(); - } - if (endpoint.provider === "telegram" && !values.webhookSecret) { - values.webhookSecret = randomBytes(32).toString("hex"); - } - const required = - endpoint.provider === "github" - ? ["appId", "privateKey", "webhookSecret"] - : REQUIRED_CREDENTIALS[endpoint.provider]; - const missing = required.filter((key) => !values[key]); - if (missing.length > 0) { - throw unprocessable( - `Missing required ${PROVIDER_LABELS[endpoint.provider]} credentials: ${missing.join(", ")}`, - ); - } - return endpoint.provider === "microsoft-teams" - ? normalizeMicrosoftTeamsCredentialIds(values) - : values; - } - - async function verifyCredentials( - provider: ChatProvider, - credentials: Record, - ): Promise { - if (provider === "imessage-photon") return verifyPhotonCredentials(credentials); - if (provider === "slack") return verifySlackCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); - if (provider === "discord") return verifyDiscordCredentials(credentials, fetchImpl, redactError); - if (provider === "telegram") return verifyTelegramCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); - if (provider === "github") return verifyGitHubCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); - if (provider === "speko") return verifySpekoCredentials(credentials, fetchImpl); - return verifyTeamsCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); - } - - function nativeBotIdentityKey( - provider: ChatProvider, - identity: VerifiedProviderIdentity, - ): string | null { - const account = identity.providerAccountId?.trim().toLowerCase(); - const bot = identity.botExternalId?.trim().toLowerCase(); - if (provider === "github" || provider === "discord") { - if (bot) return `${provider}:app:${bot}`; - return null; - } - if (!account || !bot) return null; - return `${provider}:${account}:${bot}`; - } - - function providerUsesGlobalApplicationIdentity( - provider: ChatProvider, - ): boolean { - return ( - provider === "github" || - provider === "discord" || - provider === "microsoft-teams" || provider === "imessage-photon" - ); - } - - function nativeBotIdentityMatches( - provider: ChatProvider, - current: VerifiedProviderIdentity, - incoming: VerifiedProviderIdentity, - ): boolean { - const currentKey = nativeBotIdentityKey(provider, current); - return ( - currentKey !== null && - currentKey === nativeBotIdentityKey(provider, incoming) - ); - } - - function legacyGitHubLabelsMatchVerifiedCredentials( - current: VerifiedProviderIdentity, - verified: VerifiedProviderIdentity, - ): boolean { - return ( - !current.botExternalId && - Boolean(verified.botExternalId) && - current.providerAccountId?.trim().toLowerCase() === - verified.providerAccountId?.trim().toLowerCase() && - current.botUsername?.trim().toLowerCase() === - verified.botUsername?.trim().toLowerCase() - ); - } - - async function assertNativeBotIdentityAvailable( - endpoint: EndpointRow, - identity: VerifiedProviderIdentity, - ): Promise { - const key = nativeBotIdentityKey(endpoint.provider, identity); - if (!key) { - throw unprocessable( - `${PROVIDER_LABELS[endpoint.provider]} did not return a stable native bot identity`, - ); - } - const candidates = await db - .select({ - id: chatEndpoints.id, - companyId: chatEndpoints.companyId, - assignedAgentId: chatEndpoints.assignedAgentId, - providerAccountId: chatEndpoints.providerAccountId, - botExternalId: chatEndpoints.botExternalId, - botUsername: chatEndpoints.botUsername, - }) - .from(chatEndpoints) - .where( - and( - providerUsesGlobalApplicationIdentity(endpoint.provider) - ? undefined - : eq(chatEndpoints.companyId, endpoint.companyId), - eq(chatEndpoints.provider, endpoint.provider), - ne(chatEndpoints.id, endpoint.id), - endpoint.provider === "imessage-photon" ? ne(chatEndpoints.status, "archived") : inArray(chatEndpoints.status, [ - "verifying", - "active", - "paused", - "attention", - ]), - ), - ); - const conflictEndpoint = candidates.find((candidate) => { - if (!providerUsesGlobalApplicationIdentity(endpoint.provider)) { - return nativeBotIdentityKey(endpoint.provider, candidate) === key; - } - const candidateAppId = candidate.botExternalId?.trim().toLowerCase(); - const incomingAppId = identity.botExternalId?.trim().toLowerCase(); - if (candidateAppId && incomingAppId) - return candidateAppId === incomingAppId; - return ( - candidate.providerAccountId?.trim().toLowerCase() === - identity.providerAccountId?.trim().toLowerCase() && - candidate.botUsername?.trim().toLowerCase() === - identity.botUsername?.trim().toLowerCase() - ); - }); - if (conflictEndpoint) { - throw conflict( - `This ${PROVIDER_LABELS[endpoint.provider]} bot already represents another Paperclip agent connection`, - conflictEndpoint.companyId === endpoint.companyId - ? { - code: "chat_bot_identity_in_use", - endpointId: conflictEndpoint.id, - assignedAgentId: conflictEndpoint.assignedAgentId, - } - : { code: "chat_bot_identity_in_use" }, - ); - } - } - - function isNativeBotIdentityUniqueViolation(error: unknown): boolean { - return ( - isUniqueViolation(error, "chat_endpoints_photon_number_uq") || - isUniqueViolation(error, "chat_endpoints_live_bot_external_uq") || - isUniqueViolation(error, "chat_endpoints_live_discord_bot_external_uq") || - isUniqueViolation( - error, - "chat_endpoints_live_global_app_bot_external_uq", - ) || - isUniqueViolation(error, "chat_endpoints_live_bot_username_uq") - ); - } - - function nativeBotIdentityConflict(provider: ChatProvider) { - return conflict( - `This ${PROVIDER_LABELS[provider]} bot already represents another Paperclip agent connection`, - { code: "chat_bot_identity_in_use" }, - ); - } - - async function persistCredentials( - endpoint: EndpointRow, - credentials: Record, - credentialLease: CredentialMutationLeaseGuard, - actorUserId?: string | null, - ) { - await credentialLease.assertOwned(); - const previousRefs = await db - .select({ refs: toolConnections.credentialSecretRefs }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.id, endpoint.connectionId), - ), - ) - .then((rows) => rows[0]?.refs ?? []); - const refs: ToolCredentialSecretRef[] = []; - try { - for (const [key, value] of Object.entries(credentials)) { - if (endpoint.provider === "imessage-photon" && key !== "projectSecret") continue; - await credentialLease.assertOwned(); - const suffix = randomUUID(); - const secret = await secrets.create( - endpoint.companyId, - { - name: `chat.${endpoint.provider}.${endpoint.id.slice(0, 8)}.${key}.${suffix.slice(0, 8)}`, - key: `CHAT_${endpoint.provider.replaceAll("-", "_")}_${endpoint.id.replaceAll("-", "_")}_${key}_${suffix}`.toUpperCase(), - provider: "local_encrypted", - managedMode: "paperclip_managed", - value, - description: `${PROVIDER_LABELS[endpoint.provider]} channel credential for endpoint ${endpoint.id}`, - }, - { userId: actorUserId ?? undefined }, - ); - refs.push({ - secretId: secret.id, - versionSelector: "latest", - configPath: `credentials.${key}`, - required: true, - label: key, - projectionClass: "unclassified", - }); - await credentialLease.assertOwned(); - } - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const current = await tx - .select({ refs: toolConnections.credentialSecretRefs }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.id, endpoint.connectionId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) throw notFound("Chat connection not found"); - if ( - credentialFingerprint(current.refs) !== - credentialFingerprint(previousRefs) - ) { - throw conflict("Chat credentials changed during rotation", { - code: "chat_endpoint_credentials_changed", - }); - } - await tx - .delete(companySecretBindings) - .where( - and( - eq(companySecretBindings.companyId, endpoint.companyId), - eq(companySecretBindings.targetType, "tool_connection"), - eq(companySecretBindings.targetId, endpoint.connectionId), - ), - ); - if (refs.length > 0) { - await tx.insert(companySecretBindings).values( - refs.map((ref) => ({ - companyId: endpoint.companyId, - secretId: ref.secretId, - targetType: "tool_connection", - targetId: endpoint.connectionId, - configPath: ref.configPath, - versionSelector: String(ref.versionSelector ?? "latest"), - required: ref.required, - label: ref.label ?? null, - projectionClass: ref.projectionClass ?? "unclassified", - projectionAllowlistKey: ref.projectionAllowlistKey ?? null, - })), - ); - } - await tx - .update(toolConnections) - .set({ credentialSecretRefs: refs, updatedAt: new Date() }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await credentialLease.assertOwned(tx); - }); - } catch (error) { - // A lost database acknowledgement can be ambiguous even when the - // transaction committed. Delete only secrets proven unbound after the - // failure; when that proof cannot be read, leak safely for later cleanup - // instead of deleting a possibly-authoritative credential. - const boundSecretIds = - refs.length === 0 - ? new Set() - : await db - .select({ secretId: companySecretBindings.secretId }) - .from(companySecretBindings) - .where( - and( - eq(companySecretBindings.companyId, endpoint.companyId), - inArray( - companySecretBindings.secretId, - refs.map((ref) => ref.secretId), - ), - ), - ) - .then((rows) => new Set(rows.map((row) => row.secretId))) - .catch(() => null); - if (boundSecretIds) { - await Promise.allSettled( - refs - .filter((ref) => !boundSecretIds.has(ref.secretId)) - .map((ref) => secrets.remove(ref.secretId)), - ); - } - throw error; - } - const currentIds = new Set(refs.map((ref) => ref.secretId)); - const retired = previousRefs.filter((ref) => !currentIds.has(ref.secretId)); - const results = await Promise.allSettled( - retired.map((ref) => secrets.remove(ref.secretId)), - ); - if (results.some((result) => result.status === "rejected")) { - logger.warn( - { endpointId: endpoint.id }, - "chat credential rotation left an unbound retired secret for later cleanup", - ); - } - return refs; - } - - async function clearCredentials( - endpoint: EndpointRow, - credentialLease: CredentialMutationLeaseGuard, - ) { - let refs: ToolCredentialSecretRef[] = []; - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const current = await tx - .select({ refs: toolConnections.credentialSecretRefs }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.id, endpoint.connectionId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) throw notFound("Chat connection not found"); - refs = current.refs; - await tx - .delete(companySecretBindings) - .where( - and( - eq(companySecretBindings.companyId, endpoint.companyId), - eq(companySecretBindings.targetType, "tool_connection"), - eq(companySecretBindings.targetId, endpoint.connectionId), - ), - ); - await tx - .update(toolConnections) - .set({ credentialSecretRefs: [], updatedAt: new Date() }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await credentialLease.assertOwned(tx); - }); - const results = await Promise.allSettled( - refs.map((ref) => secrets.remove(ref.secretId)), - ); - if (results.some((result) => result.status === "rejected")) { - logger.warn( - { endpointId: endpoint.id }, - "chat endpoint removal left an unbound retired secret for later cleanup", - ); - } - } - - async function resolveCredentialRefs( - endpoint: EndpointRow, - refs: ToolCredentialSecretRef[], - database: DbOrTransaction = db, - ): Promise> { - const values: Record = {}; - for (const ref of refs) { - const key = ref.configPath.replace(/^credentials\./, ""); - values[key] = await (database === db ? secrets : secretService(database as Db)).resolveSecretValue( - endpoint.companyId, - ref.secretId, - ref.versionSelector ?? "latest", - { - consumerType: "tool_connection", - consumerId: endpoint.connectionId, - configPath: ref.configPath, - actorType: "system", - actorId: null, - }, - ); - } - return values; - } - - async function resolveCredentials( - endpoint: EndpointRow, - database: DbOrTransaction = db, - ): Promise> { - const connection = await database - .select({ refs: toolConnections.credentialSecretRefs, config: toolConnections.config }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.id, endpoint.connectionId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!connection) throw notFound("Chat connection not found"); - const values = await resolveCredentialRefs(endpoint, connection.refs, database); - if (endpoint.provider === "imessage-photon") { - const configuration = photonChannelConfigurationSchema.parse(connection.config.photon); - return { ...values, ...configuration, lineId: configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId }; - } - return values; - } - const withCredentialMutationLease = chatCredentialMutationLease(db, options); + const { list, get, create, update } = createChatEndpoints(db, { + endpointRecord, CAPABILITIES, PROVIDER_LABELS, withCredentialMutationLease, + getPublicBaseUrl, getWebhookPublicBaseUrl, + get slackRegistration() { return slackRegistration; }, + }); const { generateSetupSecret } = createGitHubSetupSecret(db, { endpointRecord, withCredentialMutationLease, resolveCredentials, persistCredentials, runtimeGeneration, credentialFingerprint, invalidateRuntime, redactError, @@ -2895,92 +1389,6 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { normalizedLifecycleWithRuntimeFence, MAX_ERROR_TEXT, }); - async function prepareProviderInventory( - endpoint: EndpointRow, - credentials: Record, - ): Promise<{ - credentials: Record; - inventory: ChatProviderInventoryResult | null; - managementUrl?: string; - }> { - try { - if (endpoint.provider === "slack") return await prepareSlackInventory(credentials, fetchImpl); - if (endpoint.provider === "github") return await prepareGitHubInventory(credentials, fetchImpl); - if (endpoint.provider === "discord") return await prepareDiscordInventory(credentials, fetchImpl); - return { credentials, inventory: null }; - } catch (error) { - throw unprocessable(redactError(error), { - code: "chat_provider_inventory_failed", - }); - } - } - - function runtimeConfig( - endpoint: EndpointRow, - userName: string, - credentials: Record, - ): ResolvedChatSdkProviderConfig { - if (endpoint.provider === "speko") throw conflict("Speko uses the voice runtime"); - if (endpoint.provider === "imessage-photon") return { - provider: "imessage-photon", userName, - intakeAfter: Date.parse(String((endpoint.setup as InternalSetupState).photonIntakeAfter ?? endpoint.setup.testStartedAt ?? endpoint.createdAt.toISOString())), - credentials: { allocation: credentials.allocation === "shared" ? "shared" : "dedicated", projectId: credentials.projectId, lineId: credentials.lineId, projectSecret: credentials.projectSecret, phoneNumber: endpoint.botExternalId! }, - }; - if (endpoint.provider === "slack") - return { - provider: "slack", - userName, - credentials: { - botToken: credentials.botToken, - signingSecret: credentials.signingSecret, - botUserId: endpoint.botExternalId ?? undefined, - }, - }; - if (endpoint.provider === "telegram") - return { - provider: "telegram", - userName, - maxDownloadBytes: MAX_ATTACHMENT_BYTES, - credentials: { - botToken: credentials.botToken, - secretToken: credentials.webhookSecret, - }, - }; - if (endpoint.provider === "discord") - return { - provider: "discord", - userName, - credentials: { - applicationId: credentials.applicationId, - botToken: credentials.botToken, - guildId: credentials.guildId, - }, - }; - if (endpoint.provider === "microsoft-teams") - return { - provider: "microsoft-teams", - userName, - credentials: { - appId: credentials.clientId, - appPassword: credentials.clientSecret, - appTenantId: credentials.tenantId, - appType: "SingleTenant", - }, - }; - return { - provider: "github", - userName, - credentials: { - appId: credentials.appId, - privateKey: credentials.privateKey, - installationId: credentials.installationId - ? Number(credentials.installationId) - : undefined, - webhookSecret: credentials.webhookSecret, - }, - }; - } - async function runtimeFor( endpoint: EndpointRow, optionsForRuntime: { @@ -3403,1331 +1811,6 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { }; } - async function inspectPhoton( - endpointId: string, - input: { projectId: string; projectSecret: string }, - ) { - const record = await endpointRecord(endpointId); - if (!record || record.endpoint.provider !== "imessage-photon") - throw notFound("iMessage Photon endpoint not found"); - const inspection = structuredClone(await inspectPhotonCredentials( - input.projectId, - input.projectSecret, - )); - const reserved = await db - .select({ number: chatEndpoints.botExternalId }) - .from(chatEndpoints) - .where( - and( - eq(chatEndpoints.provider, "imessage-photon"), - ne(chatEndpoints.id, endpointId), - ne(chatEndpoints.status, "archived"), - ), - ); - for (const line of inspection.lines) { - if (reserved.some((row) => row.number === line.phoneNumber)) { - line.eligible = false; - line.unavailableReason = - "This number already belongs to another channel"; - } - } - inspection.eligible = inspection.allocation === "shared" ? inspection.eligible && !reserved.some((row) => row.number === photonSharedIdentity(inspection.projectId)) : inspection.lines.some((line) => line.eligible); - return inspection; - } - - async function configure( - endpointId: string, - input: ConfigureChatEndpointInput, - actorUserId?: string | null, - ) { - const record = await endpointRecord(endpointId); - if (!record) throw notFound("Chat endpoint not found"); - if (record.endpoint.provider === "agentmail") throw badRequest("Use the email inbox API for AgentMail"); - if (input.photon && record.endpoint.provider !== "imessage-photon") throw badRequest("Photon configuration is only valid for iMessage Photon"); - const suppliedCredentialKeys = Object.keys(input.credentials ?? {}); - if (suppliedCredentialKeys.length > 0) { - const credentialAction = - input.action === "configure" || input.action === "reconnect"; - const allowedKeys = new Set( - credentialAction - ? SUPPLIED_CREDENTIAL_KEYS[record.endpoint.provider] - : [], - ); - const unsupportedKeys = suppliedCredentialKeys.filter( - (key) => !allowedKeys.has(key), - ); - if (unsupportedKeys.length > 0) { - throw unprocessable( - `Unsupported ${PROVIDER_LABELS[record.endpoint.provider]} credential fields for ${input.action}: ${unsupportedKeys.join(", ")}`, - { - code: "chat_endpoint_credentials_invalid", - provider: record.endpoint.provider, - action: input.action, - unsupportedKeys, - }, - ); - } - } - return withCredentialMutationLease(record.endpoint, (credentialLease) => - configureWithCredentialLease( - endpointId, - input, - credentialLease, - actorUserId, - ), - ); - } - - async function configureWithCredentialLease( - endpointId: string, - input: ConfigureChatEndpointInput, - credentialLease: CredentialMutationLeaseGuard, - actorUserId?: string | null, - ) { - const record = await endpointRecord(endpointId); - if (!record) throw notFound("Chat endpoint not found"); - const endpoint = record.endpoint; - if (input.action === "pause") { - if (endpoint.status !== "active") { - throw conflict("Only an active chat connection can be paused", { - code: "chat_endpoint_not_active", - }); - } - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const pausedAt = new Date(); - const current = await tx - .select({ setup: chatEndpoints.setup, status: chatEndpoints.status }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current || current.status !== "active") { - throw conflict("Only an active chat connection can be paused", { - code: "chat_endpoint_not_active", - }); - } - await notifyChatEndpointWork(tx); - await tx - .update(chatEndpoints) - .set({ - status: "paused", - setup: { - ...current.setup, - runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1), - } as InternalSetupState, - updatedAt: pausedAt, - }) - .where(eq(chatEndpoints.id, endpoint.id)); - await tx - .update(toolConnections) - .set({ status: "disabled", enabled: false, updatedAt: new Date() }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - nextAttemptAt: null, - redactedError: "Connection was paused before processing", - processedAt: pausedAt, - updatedAt: pausedAt, - }) - .where( - and( - endpoint.provider === "imessage-photon" ? sql`false` : undefined, - eq(chatDeliveries.endpointId, endpoint.id), - inArray(chatDeliveries.state, ["received", "retry"]), - ), - ); - await credentialLease.assertOwned(tx); - }); - await invalidateRuntime(endpoint.id); - await logActivity(db, { - companyId: endpoint.companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action: "chat_endpoint.paused", - entityType: "tool_connection", - entityId: endpoint.connectionId, - details: { endpointId: endpoint.id }, - }); - return get(endpoint.id); - } - if (input.action === "resume") { - if (endpoint.status !== "paused" || endpoint.setup.step !== "complete") { - throw conflict( - "Only a previously active chat connection can be resumed", - { - code: "chat_endpoint_not_resumable", - }, - ); - } - let credentials = await resolveCredentials(endpoint).catch(() => { - throw conflict("Reconnect the provider credentials before resuming", { - code: "chat_endpoint_credentials_missing", - }); - }); - const identity = await verifyCredentials(endpoint.provider, credentials); - const upgradingLegacyGitHubIdentity = - endpoint.provider === "github" && - !endpoint.botExternalId && - legacyGitHubLabelsMatchVerifiedCredentials(endpoint, identity); - if ( - !upgradingLegacyGitHubIdentity && - !nativeBotIdentityMatches(endpoint.provider, endpoint, identity) - ) { - throw conflict( - "The provider credentials now identify a different bot; reconnect this connection instead", - { - code: "chat_bot_identity_changed", - }, - ); - } - await assertNativeBotIdentityAvailable(endpoint, identity); - const prepared = await prepareProviderInventory(endpoint, credentials); - if ( - endpoint.provider === "github" && - prepared.credentials.installationId !== credentials.installationId - ) { - await persistCredentials( - endpoint, - prepared.credentials, - credentialLease, - actorUserId, - ); - } - credentials = prepared.credentials; - if (prepared.inventory) - await reconcileProviderResourceRows( - endpoint, - prepared.inventory, - credentialLease, - ); - let activatedEndpoint!: EndpointRow; - try { - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const resumedAt = new Date(); - const current = await tx - .select() - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current || current.status !== "paused") { - throw conflict( - "Only a previously active chat connection can be resumed", - { - code: "chat_endpoint_not_resumable", - }, - ); - } - // Fail closed for any legacy or racing delivery that remained open - // while this endpoint was paused. Resume must never execute traffic - // that Paperclip acknowledged during the inactive interval. - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - nextAttemptAt: null, - redactedError: "Connection was paused when this event arrived", - processedAt: resumedAt, - updatedAt: resumedAt, - }) - .where( - and( - endpoint.provider === "imessage-photon" ? sql`false` : undefined, - eq(chatDeliveries.endpointId, endpoint.id), - inArray(chatDeliveries.state, ["received", "retry"]), - ), - ); - await notifyChatEndpointWork(tx); - [activatedEndpoint] = await tx - .update(chatEndpoints) - .set({ - status: "active", - ...(upgradingLegacyGitHubIdentity - ? { botExternalId: identity.botExternalId } - : {}), - healthMessage: "Connected", - lastError: null, - setup: { - ...current.setup, - ...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: resumedAt.toISOString() } : {}), - runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1), - } as InternalSetupState, - updatedAt: resumedAt, - }) - .where(eq(chatEndpoints.id, endpoint.id)) - .returning(); - await tx - .update(toolConnections) - .set({ - status: "active", - enabled: true, - healthStatus: "healthy", - healthMessage: "Connected", - lastError: null, - healthCheckedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await credentialLease.assertOwned(tx); - }); - await invalidateRuntime(endpoint.id); - if (activatedEndpoint.provider === "discord") - await reconcileDiscordCommands(endpoint.id, credentialLease, true); - await runtimeFor(activatedEndpoint, { - requireDiscordOwnership: leasedChatProvider(activatedEndpoint.provider), - waitForDiscordOwnership: leasedChatProvider(activatedEndpoint.provider), - }); - } catch (error) { - await invalidateRuntime(endpoint.id).catch(() => undefined); - if (activatedEndpoint) { - const failure = redactError(error); - const failedAt = new Date(); - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await notifyChatEndpointWork(tx); - await tx - .update(chatEndpoints) - .set({ - status: "attention", - healthMessage: "Provider resume needs attention", - lastError: failure, - updatedAt: failedAt, - }) - .where( - and( - eq(chatEndpoints.id, endpoint.id), - eq(chatEndpoints.status, "active"), - sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer, 0) = ${runtimeGeneration(activatedEndpoint.setup)}`, - ), - ); - await tx - .update(toolConnections) - .set({ - status: "disabled", - enabled: false, - healthStatus: "degraded", - healthMessage: "Provider resume failed", - lastError: failure, - healthCheckedAt: failedAt, - updatedAt: failedAt, - }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - nextAttemptAt: null, - redactedError: "Connection activation failed", - processedAt: failedAt, - updatedAt: failedAt, - }) - .where( - and( - eq(chatDeliveries.endpointId, endpoint.id), - inArray(chatDeliveries.state, ["received", "retry"]), - ), - ); - await credentialLease.assertOwned(tx); - }); - } - throw error; - } - await logActivity(db, { - companyId: endpoint.companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action: "chat_endpoint.resumed", - entityType: "tool_connection", - entityId: endpoint.connectionId, - details: { endpointId: endpoint.id }, - }); - return get(endpoint.id); - } - if (input.action === "remove") { - if (endpoint.status === "archived") { - if (endpoint.provider === "slack") await slackRegistration.cleanup(endpoint.id, credentialLease); - // Archival is the durable ingress fence and intentionally commits - // before secret-store cleanup. If that cleanup failed, a repeated - // remove is the recovery operation; once refs are empty, retain the - // normal already-removed conflict contract. - if (record.credentialSecretRefs.length > 0) { - await invalidateRuntime(endpoint.id).catch(() => undefined); - if (endpoint.provider === "telegram") { - await credentialLease.assertOwned(); - const action = await stageTelegramMaintenance( - endpoint, - "remove_endpoint", - ); - if (action) - await processTelegramMaintenance( - action.id, - true, - credentialLease, - ); - } else { - await clearCredentials(endpoint, credentialLease); - } - return get(endpoint.id); - } - throw conflict("This chat connection has already been removed", { - code: "chat_endpoint_already_removed", - }); - } - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const current = await tx - .select({ setup: chatEndpoints.setup, status: chatEndpoints.status }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current || current.status === "archived") { - throw conflict("This chat connection has already been removed", { - code: "chat_endpoint_already_removed", - }); - } - await notifyChatEndpointWork(tx); - await tx - .update(chatEndpoints) - .set({ - status: "archived", - setup: { - ...current.setup, - runtimeGeneration: runtimeGeneration(current.setup) + 1, - } as InternalSetupState, - archivedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(chatEndpoints.id, endpoint.id)); - await tx - .update(chatConversations) - .set({ state: "endpoint_removed", updatedAt: new Date() }) - .where(eq(chatConversations.endpointId, endpoint.id)); - await tx - .update(toolConnections) - .set({ status: "archived", enabled: false, updatedAt: new Date() }) - .where(eq(toolConnections.id, endpoint.connectionId)); - const otherConnection = await tx - .select({ id: toolConnections.id }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.applicationId, record.applicationId), - ne(toolConnections.id, endpoint.connectionId), - ne(toolConnections.status, "archived"), - ), - ) - .limit(1) - .then((rows) => rows[0] ?? null); - if (!otherConnection) { - await tx - .update(toolApplications) - .set({ - status: "archived", - archivedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(toolApplications.id, record.applicationId)); - } - await credentialLease.assertOwned(tx); - }); - if (endpoint.provider === "slack") await slackRegistration.cleanup(endpoint.id, credentialLease); - await invalidateRuntime(endpoint.id).catch(() => undefined); - if (endpoint.provider === "telegram") { - await credentialLease.assertOwned(); - const action = await stageTelegramMaintenance( - endpoint, - "remove_endpoint", - ); - if (action) { - await processTelegramMaintenance(action.id, true, credentialLease); - } - } else { - await clearCredentials(endpoint, credentialLease); - } - return get(endpoint.id); - } - if ( - input.action !== "configure" && - input.action !== "reconnect" && - input.action !== "verify" - ) { - throw unprocessable("Unsupported chat endpoint setup action"); - } - if (!getWebhookPublicBaseUrl() && endpoint.provider !== "discord" && endpoint.provider !== "imessage-photon" && !(endpoint.provider === "github" && endpoint.setup.github?.cloudRegistrationId)) { - throw unprocessable( - `A public HTTPS Paperclip URL is required before connecting ${PROVIDER_LABELS[endpoint.provider]}`, - ); - } - if ( - endpoint.provider === "telegram" && - (input.action === "configure" || input.action === "reconnect") && - !isSupportedTelegramWebhookBaseUrl(getWebhookPublicBaseUrl()) - ) { - throw unprocessable( - "Telegram webhooks require PAPERCLIP_CHAT_WEBHOOK_PUBLIC_URL to use HTTPS on port 443, 80, 88, or 8443", - { - code: "chat_telegram_webhook_url_unsupported", - provider: "telegram", - supportedPorts: [443, 80, 88, 8443], - }, - ); - } - if (input.action === "verify") { - if ( - endpoint.provider !== "slack" || - endpoint.status !== "verifying" || - endpoint.setup.step !== "provider_setup" - ) { - throw conflict( - "Provider verification is not available at this setup step", - { - code: "chat_endpoint_invalid_setup_step", - }, - ); - } - if (!endpoint.setup.webhookVerifiedAt) { - throw conflict("Paperclip has not received a verified Slack callback yet", { - code: "chat_webhook_not_verified", - }); - } - } else if ( - input.action === "configure" && - !["draft", "attention", "revoked"].includes(endpoint.status) && - // GitHub installation discovery can advance the status before the - // wizard has configured its runtime. Finish that exact setup stage; - // configured connections still require the explicit reconnect flow. - !( - endpoint.provider === "github" && - endpoint.status === "verifying" && - endpoint.setup.step === "provider_setup" - ) - ) { - throw conflict( - "This connection is already configured; use its reconnect flow if credentials need repair", - { - code: "chat_endpoint_already_configured", - }, - ); - } else if ( - input.action === "reconnect" && - !["verifying", "active", "attention", "revoked", "paused"].includes( - endpoint.status, - ) - ) { - throw conflict( - "This chat connection does not currently need reconnecting", - { - code: "chat_endpoint_not_reconnectable", - }, - ); - } - if ( - endpoint.provider === "github" && - (input.action === "configure" || input.action === "reconnect") && - !endpoint.setup.webhookVerifiedAt - ) { - throw conflict( - "GitHub has not delivered a signed webhook ping for this secret yet", - { - code: "chat_webhook_not_verified", - }, - ); - } - - let verifiedCurrentIdentity: VerifiedProviderIdentity = endpoint; - if ( - input.action === "reconnect" && - endpoint.provider === "github" && - !endpoint.botExternalId - ) { - const storedCredentials = await resolveCredentials(endpoint).catch(() => { - throw conflict( - "Reconnect the existing GitHub App credentials before changing them", - { - code: "chat_bot_identity_unverifiable", - }, - ); - }); - const storedIdentity = await verifyCredentials( - "github", - storedCredentials, - ); - if ( - !legacyGitHubLabelsMatchVerifiedCredentials(endpoint, storedIdentity) - ) { - throw conflict( - "The stored credentials no longer verify the GitHub App that owns this connection", - { - code: "chat_bot_identity_unverifiable", - }, - ); - } - verifiedCurrentIdentity = storedIdentity; - } - - const previousSpekoSigningSecret = endpoint.provider === "speko" - ? (await resolveCredentials(endpoint).catch(() => ({} as Record))).signingSecret : undefined; - let credentials = - input.credentials && Object.keys(input.credentials).length > 0 - ? await normalizedCredentials(endpoint, input.credentials) - : await resolveCredentials(endpoint).catch(() => { - throw unprocessable( - `${PROVIDER_LABELS[endpoint.provider]} credentials are required`, - ); - }); - if (endpoint.provider === "imessage-photon") { - const configuration = photonChannelConfigurationSchema.parse(input.photon ?? (credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { projectId: credentials.projectId, lineId: credentials.lineId })); - const lineId = configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId; - if (endpoint.botExternalId && (configuration.projectId !== endpoint.providerAccountId || credentials.lineId && lineId !== credentials.lineId)) throw conflict("A different Photon identity requires a new channel"); - credentials = { ...credentials, ...configuration, lineId }; - } - const identity = await verifyCredentials(endpoint.provider, credentials); - // Once setup has claimed a provider bot identity, every credential repair - // must prove that same identity before secrets can be replaced. A process - // interruption can leave the endpoint in `attention` after the claim but - // before credential persistence; a repeated first-setup request must not - // turn that recovery state into an identity-swap escape hatch. - if (input.action === "reconnect" || Boolean(endpoint.botExternalId)) { - if ( - !nativeBotIdentityMatches( - endpoint.provider, - verifiedCurrentIdentity, - identity, - ) - ) { - throw conflict( - "The provider credentials identify a different bot; create a new connection for that bot instead", - { code: "chat_bot_identity_changed" }, - ); - } - } - await assertNativeBotIdentityAvailable(endpoint, identity); - const prepared = await prepareProviderInventory(endpoint, credentials); - const credentialsChangedByDiscovery = - endpoint.provider === "github" && - prepared.credentials.installationId !== credentials.installationId; - credentials = prepared.credentials; - await options.nativeBotIdentityClaimBarrier?.(); - // Claim a fresh provider identity before any newly supplied secrets are - // persisted. The provider-global unique indexes are the multi-process - // arbitration boundary when two setup requests both pass the optimistic - // preflight. A process that stops after this small claim leaves a - // fail-closed attention endpoint that the same operator can reconnect; - // it never leaves credentials bound to the losing endpoint. - if (!endpoint.botExternalId) { - try { - const claimed = await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await notifyChatEndpointWork(tx); - const rows = await tx - .update(chatEndpoints) - .set({ - status: "attention", - providerAccountId: identity.providerAccountId ?? null, - providerAccountLabel: identity.providerAccountLabel ?? null, - botExternalId: identity.botExternalId ?? null, - botUsername: identity.botUsername ?? null, - botDisplayName: - identity.botLabel ?? - endpoint.botDisplayName ?? - record.assignedAgentName, - healthMessage: "Provider setup must be completed", - lastError: null, - updatedAt: new Date(), - }) - .where( - and( - eq(chatEndpoints.id, endpoint.id), - eq(chatEndpoints.status, endpoint.status), - isNull(chatEndpoints.botExternalId), - ), - ) - .returning({ id: chatEndpoints.id }); - await credentialLease.assertOwned(tx); - return rows; - }); - if (claimed.length !== 1) { - throw conflict("This chat connection changed during setup", { - code: "chat_endpoint_setup_changed", - }); - } - } catch (error) { - if (isNativeBotIdentityUniqueViolation(error)) { - throw nativeBotIdentityConflict(endpoint.provider); - } - throw error; - } - } - if (endpoint.provider === "imessage-photon") { - await invalidateRuntime(endpoint.id); - await credentialLease.assertOwned(); - if (credentials.allocation === "shared") await db.update(chatEndpoints).set({ allowGroupChats: false }).where(eq(chatEndpoints.id, endpoint.id)); - await db.update(toolConnections).set({ config: { provider: endpoint.provider, photon: credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { allocation: "dedicated", projectId: credentials.projectId, lineId: credentials.lineId } } }).where(and(eq(toolConnections.companyId, endpoint.companyId), eq(toolConnections.id, endpoint.connectionId))); - } - if ( - (input.credentials && Object.keys(input.credentials).length > 0) || - credentialsChangedByDiscovery - ) - await persistCredentials( - endpoint, - credentials, - credentialLease, - actorUserId, - ); - if (prepared.inventory) - await reconcileProviderResourceRows( - endpoint, - prepared.inventory, - credentialLease, - ); - const updatedAt = new Date(); - const waitingForSlackConfiguration = - endpoint.provider === "slack" && - (input.action === "configure" || input.action === "reconnect"); - try { - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const current = await tx - .select({ setup: chatEndpoints.setup }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) throw notFound("Chat endpoint not found"); - if (endpoint.provider === "speko") { - if (!actorUserId) throw forbidden("An authenticated connection owner is required to install phone tools"); - await syncSpekoVoiceTools(tx, endpoint, actorUserId, true, options.allowLocalVoiceBoard === true); - } - const observedSlackUrl = (current.setup as InternalSetupState).slackCallbackSurfaces?.events?.url; - const preserveSlackVerification = isProvisionedSlackSetup(current.setup.slackSetupMethod) - && (current.setup as InternalSetupState).slackVerificationSigningFingerprint === createHash("sha256").update(credentials.signingSecret ?? "").digest("hex") - && Boolean(observedSlackUrl && slackCallbackMatchesPublicUrl(observedSlackUrl, `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/slack`)); - await notifyChatEndpointWork(tx); - await tx - .update(chatEndpoints) - .set({ - status: "verifying", - providerAccountId: identity.providerAccountId ?? null, - providerAccountLabel: identity.providerAccountLabel ?? null, - botExternalId: identity.botExternalId ?? null, - botUsername: identity.botUsername ?? null, - botDisplayName: - identity.botLabel ?? - endpoint.botDisplayName ?? - record.assignedAgentName, - healthMessage: waitingForSlackConfiguration - ? "Finish provider webhook configuration" - : "Waiting for a test conversation", - lastError: null, - lastEventAt: null, - setup: { - ...current.setup, - ...(endpoint.provider === "github" ? { github: { ...current.setup.github, stage: current.setup.github?.stage ?? "connect", appOwnerType: identity.githubOwnerType, appOwnerLogin: identity.githubOwnerLogin } } : {}), - step: waitingForSlackConfiguration ? "provider_setup" : "test", - ...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: (current.setup as InternalSetupState).photonIntakeAfter ?? updatedAt.toISOString() } : {}), - testStartedAt: waitingForSlackConfiguration - ? null - : updatedAt.toISOString(), - webhookVerifiedAt: waitingForSlackConfiguration - ? preserveSlackVerification ? (current.setup.webhookVerifiedAt ?? null) : null - : (current.setup.webhookVerifiedAt ?? null), - runtimeGeneration: runtimeGeneration(current.setup) + 1, - } as InternalSetupState, - updatedAt, - }) - .where(eq(chatEndpoints.id, endpoint.id)); - await tx - .update(toolConnections) - .set({ - status: "active", - enabled: true, - healthStatus: "healthy", - healthMessage: "Provider credentials verified", - healthCheckedAt: updatedAt, - updatedAt, - }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await tx - .update(toolApplications) - .set({ status: "active", archivedAt: null, updatedAt }) - .where(eq(toolApplications.id, record.applicationId)); - await credentialLease.assertOwned(tx); - }); - } catch (error) { - if (isNativeBotIdentityUniqueViolation(error)) { - throw nativeBotIdentityConflict(endpoint.provider); - } - throw error; - } - try { - const next = await endpointRecord(endpoint.id); - if (!next) throw notFound("Chat endpoint not found"); - await invalidateRuntime(endpoint.id); - if (endpoint.provider === "github" && input.action === "reconnect" && endpoint.setup.github?.cloudRegistrationId) { - if (!actorUserId) throw forbidden("A configuring member is required to recover this App"); - await credentialLease.assertOwned(); - await githubWizard.repairCloud(endpoint.id, actorUserId); - await credentialLease.assertOwned(); - } - if (endpoint.provider === "github" && input.action === "reconnect" && !endpoint.setup.github?.cloudRegistrationId) { - // The immutable App and its installation were verified above. Repair - // only its callback using the stored secret; never reinstall it or - // change repository access. Keep historical signed-ping evidence, but - // require the fresh test round trip established above before activation. - const auditWebhookSync = async (action: string) => { - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await logActivity(tx as unknown as Db, { - companyId: endpoint.companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action, - entityType: "tool_connection", - entityId: endpoint.connectionId, - details: { endpointId: endpoint.id, provider: "github" }, - }); - await credentialLease.assertOwned(tx); - }); - }; - await auditWebhookSync("chat_endpoint.webhook_sync_started"); - await credentialLease.assertOwned(); - await resyncGitHubAppWebhook({ - fetch: fetchImpl, - appToken: githubAppJwt(credentials.appId, credentials.privateKey), - webhookUrl: `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/github`, - webhookSecret: credentials.webhookSecret, - }); - await auditWebhookSync("chat_endpoint.webhook_synced"); - } - if (next.endpoint.provider === "speko") { - const webhookPublicBaseUrl = getWebhookPublicBaseUrl(); - if (!webhookPublicBaseUrl?.startsWith("https://")) throw unprocessable("Speko needs an explicitly configured public HTTPS callback origin"); - await configureSpekoSessionTools(db, { companyId: endpoint.companyId, endpointId: endpoint.id, agentId: credentials.agentId, callbackUrl: `${webhookPublicBaseUrl}/api/voice-webhooks/${endpoint.publicId}/tools`, signingSecret: credentials.signingSecret, previousSigningSecret: previousSpekoSigningSecret, client: createSpekoProvider(credentials.apiKey, fetchImpl), assertOwned: () => credentialLease.assertOwned() }); - } - if (next.endpoint.provider === "discord") - await reconcileDiscordCommands(endpoint.id, credentialLease, true); - await runtimeFor(next.endpoint, { - requireDiscordOwnership: next.endpoint.provider === "discord", - waitForDiscordOwnership: next.endpoint.provider === "discord", - }); - - if (endpoint.provider === "telegram" && getWebhookPublicBaseUrl()) { - const webhookUrl = `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/telegram`; - const infoResponse = await fetchImpl( - `https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/getWebhookInfo`, - { signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS) }, - ); - const infoResult = (await infoResponse.json()) as { - ok?: boolean; - description?: string; - result?: { url?: string }; - }; - if (!infoResponse.ok || !infoResult.ok) { - throw unprocessable( - `Telegram could not inspect the existing webhook: ${infoResult.description ?? infoResponse.status}`, - ); - } - await credentialLease.assertOwned(); - const response = await fetchImpl( - `https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/setWebhook`, - { - method: "POST", - signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS), - headers: { "content-type": "application/json" }, - body: JSON.stringify({ - url: webhookUrl, - secret_token: credentials.webhookSecret, - allowed_updates: [ - "message", - "edited_message", - "callback_query", - "stopped_message_generation", - "message_reaction", - "my_chat_member", - ], - // A first-time connection may inherit stale updates from a bot - // that was used before Paperclip owned it. Reconnect is - // different: Telegram may be holding legitimate updates while - // the old callback URL is unavailable. Preserve that backlog - // even when an operator is rotating the public origin. - drop_pending_updates: input.action === "configure", - }), - }, - ); - const result = (await response.json()) as { - ok?: boolean; - description?: string; - result?: unknown; - }; - if (!response.ok || !result.ok) - throw unprocessable( - `Telegram could not register the webhook: ${result.description ?? response.status}`, - ); - await credentialLease.assertOwned(); - if (result.ok === true && result.result === true) { - // Telegram's successful boolean receipt acknowledges the exact URL, - // secret and allowed_updates request above. GET alone cannot prove - // secret_token, which Telegram deliberately does not return. - await recordTelegramStopSubscription( - next.endpoint, - runtimeContextForRecord(next), - credentialLease, - ); - } - // Command discovery is optional for ingress, but its provider mutation - // is still a durable, auditable action. A transient or rate-limited - // failure is retried by the maintenance worker without blocking the - // otherwise-valid webhook connection. - const commandAction = await stageTelegramMaintenance( - endpoint, - "register_commands", - ); - if (commandAction) - await processTelegramMaintenance( - commandAction.id, - false, - credentialLease, - ); - } - } catch (error) { - const failure = redactError(error); - await invalidateRuntime(endpoint.id).catch(() => undefined); - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await notifyChatEndpointWork(tx); - await tx - .update(chatEndpoints) - .set({ - status: "attention", - healthMessage: "Provider setup needs attention", - lastError: failure, - updatedAt: new Date(), - }) - .where(eq(chatEndpoints.id, endpoint.id)); - await tx - .update(toolConnections) - .set({ - healthStatus: "degraded", - healthMessage: "Provider setup failed", - lastError: failure, - healthCheckedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(toolConnections.id, endpoint.connectionId)); - await credentialLease.assertOwned(tx); - }); - throw unprocessable(failure, { code: "chat_provider_setup_failed" }); - } - - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await logActivity(tx as unknown as Db, { - companyId: endpoint.companyId, - actorType: "user", - actorId: actorUserId ?? "board", - action: - input.action === "reconnect" - ? "chat_endpoint.reconnected" - : "chat_endpoint.credentials_verified", - entityType: "tool_connection", - entityId: endpoint.connectionId, - details: { endpointId: endpoint.id, provider: endpoint.provider }, - }); - await credentialLease.assertOwned(tx); - }); - return get(endpoint.id); - } - - async function findAuthorizedIdentityLink(endpoint: EndpointRow, userId: string) { - return db.select({ id: chatIdentityLinks.id }).from(chatIdentityLinks) - .innerJoin(companyMemberships, and( - eq(companyMemberships.companyId, chatIdentityLinks.companyId), eq(companyMemberships.principalType, "user"), - eq(companyMemberships.principalId, userId), eq(companyMemberships.status, "active"), ne(companyMemberships.membershipRole, "viewer"), - )) - .where(and(eq(chatIdentityLinks.companyId, endpoint.companyId), eq(chatIdentityLinks.endpointId, endpoint.id), - eq(chatIdentityLinks.paperclipUserId, userId), eq(chatIdentityLinks.status, "linked"))) - .limit(1).then((rows) => rows[0] ?? null); - } - - async function ensurePrincipal( - endpoint: EndpointRow, - author: Author, - raw?: unknown, - ) { - const providerAccountId = endpoint.providerAccountId ?? "unknown"; - const externalId = stableExternalPrincipalId( - endpoint.provider, - author, - raw, - ); - const [principal] = await db - .insert(chatExternalPrincipals) - .values({ - companyId: endpoint.companyId, - provider: endpoint.provider, - providerAccountId, - externalId, - kind: - author.isBot === true ? "bot" : author.isSystem ? "system" : "user", - displayName: author.fullName, - handle: author.userName, - isBot: author.isBot === true || author.isMe, - lastSeenAt: new Date(), - }) - .onConflictDoUpdate({ - target: [ - chatExternalPrincipals.companyId, - chatExternalPrincipals.provider, - chatExternalPrincipals.providerAccountId, - chatExternalPrincipals.externalId, - ], - set: { - displayName: author.fullName, - handle: author.userName, - isBot: author.isBot === true || author.isMe, - lastSeenAt: new Date(), - updatedAt: new Date(), - }, - }) - .returning(); - if (endpoint.provider === "speko") { - try { - const authorization = await db.transaction((tx) => voiceStore.authorizePrincipal(tx, endpoint.companyId, endpoint.id, externalId)); - return { principal, userId: authorization.userId, linkedDenied: false }; - } catch (error) { - if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error; - return { principal, userId: null, linkedDenied: true }; - } - } - const link = await db - .select({ - userId: chatIdentityLinks.paperclipUserId, - status: chatIdentityLinks.status, - }) - .from(chatIdentityLinks) - .where( - and( - eq(chatIdentityLinks.endpointId, endpoint.id), - eq(chatIdentityLinks.principalId, principal.id), - ), - ) - .then((rows) => rows[0] ?? null); - if (link?.status === "linked" && link.userId) { - const membership = await db - .select({ - status: companyMemberships.status, - membershipRole: companyMemberships.membershipRole, - }) - .from(companyMemberships) - .where( - and( - eq(companyMemberships.companyId, endpoint.companyId), - eq(companyMemberships.principalType, "user"), - eq(companyMemberships.principalId, link.userId), - ), - ) - .then((rows) => rows[0] ?? null); - // External chat performs the same task mutations as the board. A linked - // viewer therefore remains read-only instead of gaining write authority - // merely by using a provider account. - if ( - membership?.status !== "active" || - membership.membershipRole === "viewer" - ) { - return { principal, userId: null, linkedDenied: true }; - } - return { principal, userId: link.userId, linkedDenied: false }; - } - return { principal, userId: null, linkedDenied: false }; - } - - async function sponsorAllowsGuest(endpoint: EndpointRow): Promise { - // A null sponsor is the local board/operator context used by self-hosted - // instances. Cloud endpoints retain the creating user and must revalidate - // that user's current company authority on every sponsored guest turn. - if (!endpoint.sponsorUserId) return true; - const membership = await db - .select({ - status: companyMemberships.status, - membershipRole: companyMemberships.membershipRole, - }) - .from(companyMemberships) - .where( - and( - eq(companyMemberships.companyId, endpoint.companyId), - eq(companyMemberships.principalType, "user"), - eq(companyMemberships.principalId, endpoint.sponsorUserId), - ), - ) - .then((rows) => rows[0] ?? null); - return ( - membership?.status === "active" && membership.membershipRole !== "viewer" - ); - } - - async function lockCurrentPrincipalAuthorization( - tx: DbOrTransaction, - endpoint: EndpointRow, - principalId: string, - ): Promise<{ - allowed: boolean; - linkedDenied: boolean; - userId: string | null; - sponsorUserId?: string | null; - }> { - if (endpoint.provider === "speko") { - const principal = await tx.select({ externalId: chatExternalPrincipals.externalId }).from(chatExternalPrincipals) - .where(and(eq(chatExternalPrincipals.id, principalId), eq(chatExternalPrincipals.companyId, endpoint.companyId), eq(chatExternalPrincipals.provider, "speko"))).then((rows) => rows[0]); - if (!principal) return { allowed: false, linkedDenied: true, userId: null }; - try { return await voiceStore.authorizePrincipal(tx as import("./voice/voice-session-store.js").VoiceTransaction, endpoint.companyId, endpoint.id, principal.externalId); } - catch (error) { - if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error; - return { allowed: false, linkedDenied: true, userId: null }; - } - } - const githubAccess = await githubChatPrincipalAccess(tx, endpoint, principalId); - if (githubAccess) return githubAccess; - // Link confirmation already uses this transaction-scoped identity key. - // Taking it at the final task mutation boundary prevents a newly confirmed - // identity from racing the authorization snapshot. Row locks below also - // serialize revocation and membership changes that do not use this key. - await tx.execute( - sql`select pg_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0))`, - ); - const link = await tx - .select({ - status: chatIdentityLinks.status, - userId: chatIdentityLinks.paperclipUserId, - }) - .from(chatIdentityLinks) - .where( - and( - eq(chatIdentityLinks.companyId, endpoint.companyId), - eq(chatIdentityLinks.endpointId, endpoint.id), - eq(chatIdentityLinks.principalId, principalId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - if (link?.status === "linked") { - if (!link.userId) { - return { allowed: false, linkedDenied: true, userId: null }; - } - const membership = await tx - .select({ - status: companyMemberships.status, - membershipRole: companyMemberships.membershipRole, - }) - .from(companyMemberships) - .where( - and( - eq(companyMemberships.companyId, endpoint.companyId), - eq(companyMemberships.principalType, "user"), - eq(companyMemberships.principalId, link.userId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - const allowed = - membership?.status === "active" && - membership.membershipRole !== "viewer"; - return { - allowed, - linkedDenied: !allowed, - userId: allowed ? link.userId : null, - }; - } - if (!endpoint.allowUnlinkedPeople) { - return { allowed: false, linkedDenied: false, userId: null }; - } - if (!endpoint.sponsorUserId) { - return { allowed: true, linkedDenied: false, userId: null }; - } - const sponsorMembership = await tx - .select({ - status: companyMemberships.status, - membershipRole: companyMemberships.membershipRole, - }) - .from(companyMemberships) - .where( - and( - eq(companyMemberships.companyId, endpoint.companyId), - eq(companyMemberships.principalType, "user"), - eq(companyMemberships.principalId, endpoint.sponsorUserId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - return { - allowed: - sponsorMembership?.status === "active" && - sponsorMembership.membershipRole !== "viewer", - linkedDenied: false, - userId: null, - }; - } - - async function requireCurrentExternalActionAuthorization( - tx: DbTransaction, - input: { - conversationId: string; - endpointId: string; - expectedUserId: string; - principalId: string; - runtimeContext: LifecycleRuntimeFence; - }, - ): Promise { - const endpoint = await runtimeCallbackEndpoint( - tx, - input.endpointId, - input.runtimeContext, - ["active", "verifying"], - ); - // Photon setup already admits linked senders and publishes agent prompts. - // Let those prompts resolve so a clarifying question cannot deadlock the - // actual-reply qualification. Other providers retain their active-only gate. - if (!endpoint || (endpoint.status !== "active" && - (endpoint.provider !== "imessage-photon" || endpoint.setup.step !== "test"))) { - throw forbidden("This chat action is no longer authorized", { - code: "chat_action_authorization_changed", - }); - } - const conversation = await tx - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, endpoint.companyId), - eq(chatConversations.endpointId, endpoint.id), - eq(chatConversations.id, input.conversationId), - inArray(chatConversations.state, ["active", "waiting"]), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - if (!conversation) { - throw forbidden("This chat action is no longer authorized", { - code: "chat_action_authorization_changed", - }); - } - const resource = conversation.resourceId - ? await tx - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.companyId, endpoint.companyId), - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.id, conversation.resourceId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const destinationAllowed = conversation.isDirectMessage - ? endpoint.allowDirectMessages - : nonDirectDestinationAllowed(endpoint, resource); - const authorization = await lockCurrentPrincipalAuthorization( - tx, - endpoint, - input.principalId, - ); - // Governed actions never fall back to sponsored-guest authority. If the - // provider identity was relinked while this callback waited, reject this - // stale actor snapshot and let a fresh callback use the new identity. - if ( - !destinationAllowed || - !authorization.allowed || - authorization.userId !== input.expectedUserId - ) { - throw forbidden("This chat action is no longer authorized", { - code: "chat_action_authorization_changed", - }); - } - } - - function isExternalActionAuthorizationChange(error: unknown): boolean { - if (!error || typeof error !== "object") return false; - const value = error as { details?: unknown }; - return ( - Boolean(value.details) && - typeof value.details === "object" && - (value.details as { code?: unknown }).code === - "chat_action_authorization_changed" - ); - } - - async function ensureResource( - endpoint: EndpointRow, - thread: Thread, - enabledBySetupActivation: boolean, - database: DbOrTransaction = db, - ) { - const type = providerResourceType( - endpoint.provider, - chatSurfaceKind(endpoint.provider, thread), - ); - const providerResourceId = canonicalProviderResourceId( - endpoint.provider, - thread, - ); - const resourceLabel = providerResourceLabelFromThread( - endpoint.provider, - thread, - ); - const [resource] = await database - .insert(chatEndpointResources) - .values({ - companyId: endpoint.companyId, - endpointId: endpoint.id, - type, - providerResourceId, - label: resourceLabel.label, - availability: "available", - // Slack can deliver the invitation's app_mention before the membership - // callback. Give either arrival order the same new-channel default. - enabled: - thread.isDM || - enabledBySetupActivation || - (endpoint.provider === "slack" && type === "channel"), - }) - .onConflictDoUpdate({ - target: [ - chatEndpointResources.endpointId, - chatEndpointResources.type, - chatEndpointResources.providerResourceId, - ], - set: { - // Provider inventory and membership lifecycle own availability. - // A delayed message must not resurrect a destination after the bot - // was removed or an installation lost access. - // Slack callbacks are not guaranteed to carry a channel name. Keep - // an inventory- or conversations.info-derived label instead of - // downgrading it to C… / slack:C… when a delayed message arrives. - ...(resourceLabel.fallback - ? {} - : { - label: sql`case - when ${chatEndpointResources.availability} = 'available' then ${resourceLabel.label} - else ${chatEndpointResources.label} - end`, - }), - updatedAt: new Date(), - }, - }) - .returning(); - return resource; - } - async function stageInboundWakeup( tx: DbOrTransaction, input: { @@ -5942,274 +3025,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { // One provider message can match both a mention handler and the catch-all // new-message handler. The trigger is policy metadata, not provider event // identity, so it must not defeat durable deduplication. - const githubAutomatic = endpoint.provider === "github" ? githubAutomaticMessages.get(message) : undefined; - const githubIssue = endpoint.provider === "github" ? githubIssueMessages.get(message) : undefined; - let githubManual = githubManualMessages.get(message); - if (endpoint.provider === "github" && !githubAutomatic && !githubIssue && !githubManual) { - const [config] = await db.select().from(chatGitHubConfigurations).where(eq(chatGitHubConfigurations.endpointId, endpoint.id)); - if (config) { - const repository = thread.channelId.replace(/^github:/, "").toLowerCase(); - const [repo] = await db.select().from(chatEndpointResources).where(and(eq(chatEndpointResources.endpointId, endpoint.id), eq(chatEndpointResources.providerResourceId, repository))); - const repositoryId = String(repo?.metadata?.providerRepositoryId ?? ""); - githubManual = { policy: { ...config.configuration.defaults, ...config.configuration.repositories[repositoryId] }, revision: config.revision, event: trigger === "mention" || message.isMention ? "mention" : "comment" }; - } - } - const providerEventId = `${durableExternalThreadIdentity(thread.id)}:${message.id}`; - const surfaceKind = chatSurfaceKind(endpoint.provider, thread); - const addressed = - endpoint.provider === "imessage-photon" || - trigger === "mention" || - trigger === "direct_message" || - message.isMention === true; - const eventKind: ChatEventKind = - trigger === "direct_message" - ? "direct_message" - : trigger === "mention" - ? "mention" - : "message"; - // A callback remains tied to the runtime that authenticated and parsed it. - // In particular, do not resolve the registry again here: pause or rotation - // may already have removed that instance, and recreating a runtime before - // the durable admission fence would either throw or bind old payload data - // to new credentials. - const endpointRuntime = - runtimeContext?.endpointRuntime ?? (await runtimeFor(endpoint)); - // Non-personal generic files still require a separate Graph grant. Only - // exact inline image resources from the authenticated original activity - // receive a scope-bound, URL-free recovery locator. Reconstructed input - // can retain that proof, but cannot mint one from legacy fetch metadata. - const teamsNonPersonal = - endpoint.provider === "microsoft-teams" && !thread.isDM; - const attachmentSource = { - threadId: thread.id, - messageId: message.id, - runtimeGeneration: runtimeContext?.generation, - credentialFingerprint: runtimeContext?.credentialFingerprint, - principalExternalId: stableExternalPrincipalId( - endpoint.provider, - message.author, - message.raw, - ), - isDirectMessage: thread.isDM, - ...(runtimeContext?.endpointRuntime && !admittedDeliveryId - ? { message } - : {}), - }; - const inlineRecovery = new Map< - Attachment, - ReturnType - >(); - const nativeInboundAttachments = teamsNonPersonal - ? message.attachments.slice(0, 20).flatMap((attachment) => { - const descriptor = endpointRuntime.attachmentRecoveryDescriptor( - attachment, - attachmentSource, - ); - if (descriptor?.locator.kind !== "teams_inline_image") return []; - const restored = endpointRuntime.rehydrateAttachment( - descriptor, - attachmentSource, - ); - if (!restored) return []; - inlineRecovery.set(attachment, descriptor); - return [restored]; - }) - : message.attachments; - const unavailableTeamsReferences = teamsNonPersonal - ? Math.min(message.attachments.length, 20) - - nativeInboundAttachments.length - : 0; - const providerSentAt = - endpoint.provider === "microsoft-teams" - ? teamsActivitySentAt(message.raw) - : endpoint.provider === "telegram" - ? telegramMessageSentAt(message.raw) - : endpoint.provider === "slack" - ? slackMessageSentAt(message.raw, message.id) - : message.metadata.dateSent instanceof Date && - Number.isFinite(message.metadata.dateSent.getTime()) - ? message.metadata.dateSent - : null; - const providerSentAtSource = providerSentAt - ? endpoint.provider === "microsoft-teams" - ? "teams_activity_timestamp" - : endpoint.provider === "telegram" - ? "telegram_message_date" - : endpoint.provider === "slack" - ? "slack_message_ts" - : null - : null; - const providerUrl = - (githubAutomatic - ? `https://github.com/${githubAutomatic.context.repository}/pull/${githubAutomatic.context.pullNumber}` - : githubIssue ? `https://github.com/${githubIssue.context.repository}/issues/${githubIssue.context.issueNumber}` : null) ?? - // Description/review requests and edits have synthetic ledger IDs, not - // native comment IDs. Their verified source URL must not acquire a fake - // issuecomment anchor from that ledger identity. - (endpoint.provider === "github" && githubManual && !receiptReactionSupported ? recoveredProviderUrl : null) ?? - chatProviderConversationUrl({ - provider: endpoint.provider, - providerAccountId: endpoint.providerAccountId, - botUsername: endpoint.botUsername, - threadId: thread.id, - providerMessageId: message.id, - raw: message.raw, - }) ?? recoveredProviderUrl; - // Preserve only the closed personal-recipient proof from the runtime that - // authenticated this original activity. A reconstructed delivery's AAD-only - // author and SDK route caches cannot recreate its Bot Framework recipient. - const teamsPersonalRecipient = - endpoint.provider === "microsoft-teams" && - runtimeContext?.endpointRuntime && - !admittedDeliveryId && - endpoint.providerAccountId && - endpoint.botExternalId - ? deriveTeamsPersonalRecipient(message, { - companyId: endpoint.companyId, - endpointId: endpoint.id, - runtimeGeneration: runtimeContext.generation, - credentialFingerprint: runtimeContext.credentialFingerprint, - tenantId: endpoint.providerAccountId, - botAppId: endpoint.botExternalId, - providerEventId, - threadId: thread.id, - isDirectMessage: thread.isDM, - }) - : null; - const normalized = { - providerEventId, - kind: eventKind, - trigger, - ...(teamsPersonalRecipient ? { teamsPersonalRecipient } : {}), - ...(slackSlashControl - ? { admission: { origin: "slack_slash_control" } } - : suppressSetupDestinationActivation - ? // A slash-command root is provider-confirmed only after an enabled - // destination authorized its transport. Persist that closed origin so - // crash recovery can never reinterpret it as first-time setup traffic - // and undo a later operator reach revocation. - { admission: { origin: "provider_confirmed_action" } } - : {}), - ...(runtimeContext - ? { - runtimeContext: { - credentialFingerprint: runtimeContext.credentialFingerprint, - generation: runtimeContext.generation, - }, - } - : {}), - acknowledgement: { - // Some provider callbacks represent an auditable user command without - // a provider message that can receive a reaction. Persist the - // distinction so a crash/retry cannot mistake the synthetic ledger id - // for a native message id. - receiptReactionSupported, - }, - ...(githubAutomatic ? { githubAutomatic } : {}), - ...(githubIssue ? { githubIssue } : {}), - ...(githubManual ? { githubManual } : {}), - principal: { - // Bot/system authors have already been rejected at this intake boundary. - isBot: false, - externalId: stableExternalPrincipalId( - endpoint.provider, - message.author, - message.raw, - ), - displayName: message.author.fullName, - handle: message.author.userName, - }, - resource: { - type: providerResourceType(endpoint.provider, surfaceKind), - providerResourceId: canonicalProviderResourceId( - endpoint.provider, - thread, - ), - label: thread.channel.name ?? thread.channelId, - }, - conversation: { - externalConversationId: thread.channelId, - externalThreadId: thread.id, - label: thread.channel.name ?? thread.channelId, - isDirectMessage: thread.isDM, - providerUrl, - }, - message: { - providerMessageId: message.id, - ...(endpoint.provider === "github" && typeof (message.raw as { githubSourceMessageId?: unknown })?.githubSourceMessageId === "string" && /^[1-9][0-9]*$/.test((message.raw as { githubSourceMessageId: string }).githubSourceMessageId) - ? { sourceProviderMessageId: (message.raw as { githubSourceMessageId: string }).githubSourceMessageId } : {}), - ...(endpoint.provider === "imessage-photon" - ? { photonReply: photonReplyReference(message.raw) } - : {}), - providerMessageSequence: - endpoint.provider === "telegram" - ? telegramMessageSequence(message.raw) - : null, - providerUpdateId: - endpoint.provider === "telegram" ? (providerUpdateId ?? null) : null, - text: message.text.slice(0, MAX_INBOUND_TEXT), - mentionedBot: message.isMention === true, - providerSentAt: providerSentAt?.toISOString() ?? null, - ...(providerSentAtSource ? { providerSentAtSource } : {}), - ...((endpoint.provider === "github" || - endpoint.provider === "imessage-photon") && - (githubAttachmentLimitOmissions(message) || - Math.max(0, message.attachments.length - 20)) - ? { - attachmentLimitOmissions: - githubAttachmentLimitOmissions(message) || - Math.max(0, message.attachments.length - 20), - } - : {}), - attachments: message.attachments.slice(0, 20).map((attachment) => ({ - name: sanitizeFilename(attachment.name), - mimeType: normalizeContentType(attachment.mimeType), - size: attachment.size ?? null, - recovery: teamsNonPersonal - ? (inlineRecovery.get(attachment) ?? null) - : nativeInboundAttachments.includes(attachment) - ? endpointRuntime.attachmentRecoveryDescriptor( - attachment, - ["telegram", "imessage-photon"].includes(endpoint.provider) - ? attachmentSource - : undefined, - ) - : null, - })), - }, - }; - // Teams RSC can deliver messages from every installed channel, and a - // Telegram bot can receive addressed traffic from a provider-available - // group that the operator has not enabled in Paperclip. Until the reach - // gate admits that destination, retain only identifiers required for - // deduplication and operator-visible filtering—not user text, attachment - // metadata, or an external principal profile. - const redactedDestinationNormalized = { - providerEventId, - kind: eventKind, - trigger, - ...(runtimeContext - ? { - runtimeContext: { - credentialFingerprint: runtimeContext.credentialFingerprint, - generation: runtimeContext.generation, - }, - } - : {}), - resource: { - type: providerResourceType(endpoint.provider, surfaceKind), - providerResourceId: canonicalProviderResourceId( - endpoint.provider, - thread, - ), - }, - conversation: { externalThreadId: thread.id }, - message: { - providerMessageId: message.id, - providerSentAt: providerSentAt?.toISOString() ?? null, - }, - filtering: { contentRetained: false }, - }; + const { githubAutomatic, githubIssue, githubManual, providerEventId, surfaceKind, addressed, eventKind, endpointRuntime, nativeInboundAttachments, unavailableTeamsReferences, providerSentAt, providerUrl, normalized, redactedDestinationNormalized } = await prepareInboundEnvelope({ endpoint, thread, message, trigger, receiptReactionSupported, recoveredProviderUrl, runtimeContext, providerUpdateId, admittedDeliveryId, suppressSetupDestinationActivation, slackSlashControl }); const reorderWindow = INGRESS_REORDER_WINDOW_MS[endpoint.provider]; const scheduledAt = ingressOnly && reorderWindow @@ -6224,406 +3040,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { ownership: DiscordGatewayOwnership; }; } = {}; - const admission = await db.transaction(async (tx) => { - const currentEndpoint = await tx - .select({ - status: chatEndpoints.status, - setup: chatEndpoints.setup, - allowDirectMessages: chatEndpoints.allowDirectMessages, - requireAtMention: chatEndpoints.requireAtMention, - allowGroupChats: chatEndpoints.allowGroupChats, - botUsername: chatEndpoints.botUsername, - }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!currentEndpoint) return null; - // An SDK callback can finish parsing after a pause/resume cycle. Each - // runtime captures the durable endpoint generation at registration; - // resume advances it under the same row lock used by admission. This - // rejects stale callbacks without relying on provider/server clock sync. - const currentRuntimeGeneration = runtimeGeneration(currentEndpoint.setup); - const currentCredentialFingerprint = runtimeContext - ? await tx - .select({ refs: toolConnections.credentialSecretRefs }) - .from(toolConnections) - .where( - and( - eq(toolConnections.companyId, endpoint.companyId), - eq(toolConnections.id, endpoint.connectionId), - ), - ) - .then((rows) => credentialFingerprint(rows[0]?.refs ?? [])) - : null; - const staleActivation = - runtimeContext !== undefined && - (runtimeContext.generation !== currentRuntimeGeneration || - runtimeContext.credentialFingerprint !== - currentCredentialFingerprint); - const endpointAccepting = - ["verifying", "active"].includes(currentEndpoint.status) && - !staleActivation; - if ( - endpointAccepting && - leasedChatProvider(endpoint.provider) && - runtimeContext && - !admittedDeliveryId - ) { - // The first process-local ownership check can precede a host pause. - // Reclaim the same durable token while the endpoint admission row is - // locked so a standby takeover cannot be followed by an obsolete - // callback write. This renews only the exact token that authenticated - // the callback; generation and credential checks above remain intact. - const gatewayOwnership = - await renewDiscordGatewayOwnershipForMessageAdmission( - tx, - endpoint.id, - runtimeContext, - ); - if (gatewayOwnership.kind === "lost") { - if (gatewayOwnership.ownership) { - discordGatewayAdmissionState.lost = gatewayOwnership.ownership; - } - return null; - } - discordGatewayAdmissionState.renewed = gatewayOwnership; - } - let provisionalTeamsSetupReply = false; - let destinationAccepting = true; - if (endpointAccepting && thread.isDM) { - destinationAccepting = currentEndpoint.allowDirectMessages; - } - if ( - endpointAccepting && - endpoint.provider === "microsoft-teams" && - !thread.isDM - ) { - let resource = await ensureResource(endpoint, thread, false, tx); - const enabledChannelCount = await tx - .select({ count: sql`count(*)::int` }) - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.enabled, true), - eq(chatEndpointResources.availability, "available"), - eq(chatEndpointResources.type, "channel"), - ), - ) - .then((rows) => rows[0]?.count ?? 0); - const canActivateFirstSetupChannel = - currentEndpoint.status === "verifying" && - resource.type === "channel" && - resource.availability === "available" && - enabledChannelCount === 0 && - addressed; - if (canActivateFirstSetupChannel && !resource.enabled) { - resource = await tx - .update(chatEndpointResources) - .set({ enabled: true, updatedAt: new Date() }) - .where(eq(chatEndpointResources.id, resource.id)) - .returning() - .then((rows) => rows[0] ?? resource); - } - const teamsRootMessageId = teamsThreadRootMessageId(thread.id); - provisionalTeamsSetupReply = - currentEndpoint.status === "verifying" && - resource.type === "channel" && - resource.availability === "available" && - enabledChannelCount === 0 && - !addressed && - Boolean(teamsRootMessageId) && - teamsRootMessageId !== message.id; - destinationAccepting = - nonDirectDestinationAllowed( - { ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats }, - resource, - ) || provisionalTeamsSetupReply; - } - if ( - endpointAccepting && - endpoint.provider === "telegram" && - !thread.isDM - ) { - let resource = await ensureResource(endpoint, thread, false, tx); - const enabledResourceCount = await tx - .select({ count: sql`count(*)::int` }) - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.enabled, true), - eq(chatEndpointResources.availability, "available"), - ne(chatEndpointResources.type, "direct_message"), - ne(chatEndpointResources.type, "group_chat"), - ), - ) - .then((rows) => rows[0]?.count ?? 0); - const canActivateFirstSetupGroup = - currentEndpoint.status === "verifying" && - resource.availability === "available" && - enabledResourceCount === 0 && - addressed; - if (canActivateFirstSetupGroup && !resource.enabled) { - resource = await tx - .update(chatEndpointResources) - .set({ enabled: true, updatedAt: new Date() }) - .where(eq(chatEndpointResources.id, resource.id)) - .returning() - .then((rows) => rows[0] ?? resource); - } - // Telegram's privacy-mode contract is explicit: in a group or forum - // topic, every admitted turn must address the bot. A direct reply to a - // bot message is normalized as a mention by the pinned adapter, while - // unrelated subscribed traffic remains unaddressed and is filtered. - destinationAccepting = - addressed && - nonDirectDestinationAllowed( - { ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats }, - resource, - ); - } - if ( - endpointAccepting && - endpoint.provider === "imessage-photon" && - !thread.isDM - ) { - const resource = await ensureResource(endpoint, thread, false, tx); - destinationAccepting = - resource.enabled && - resource.availability === "available" && - currentEndpoint.allowGroupChats; - } - const mentionRequired = - endpoint.provider === "slack" && - currentEndpoint.requireAtMention && - trigger !== "mention" && - message.isMention !== true; - // GitHub sends the same comment to every installed App. Subscribed - // conversations are a fallback for unaddressed replies, not permission - // to wake every bot when the person names a different connected App. - // Recheck inside durable admission so recovered deliveries follow the - // same company-scoped routing decision before any task or reaction. - const githubAddressedElsewhere = - endpointAccepting && - endpoint.provider === "github" && - !githubAutomatic && - !githubIssue && - (await githubMessageAddressesAnotherBot(tx, { - id: endpoint.id, - companyId: endpoint.companyId, - botUsername: currentEndpoint.botUsername, - }, message.text)); - const accepting = endpointAccepting && destinationAccepting && !githubAddressedElsewhere && !mentionRequired; - const redactDestinationDelivery = - githubAddressedElsewhere || mentionRequired || (!accepting && thread.isDM) || - - (!thread.isDM && - (endpoint.provider === "microsoft-teams" || - endpoint.provider === "telegram" || - endpoint.provider === "imessage-photon") && - (!accepting || provisionalTeamsSetupReply)); - const ignoredAt = accepting ? null : new Date(); - const inactiveReason = !endpointAccepting - ? staleActivation - ? "Connection activation changed before admission" - : "Connection is not active" - : githubAddressedElsewhere - ? "GitHub message explicitly mentions a different connected bot" - : mentionRequired - ? "Message did not @mention the bot" - : endpoint.provider === "telegram" && !thread.isDM && !addressed - ? "Message did not address the agent" - : "Destination is not enabled in Paperclip"; - let candidate = admittedDeliveryId - ? await tx - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.id, admittedDeliveryId), - eq(chatDeliveries.endpointId, endpoint.id), - eq(chatDeliveries.providerEventId, providerEventId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - if (!admittedDeliveryId) { - await notifyChatDeliveryWork(tx); - const [delivery] = await tx - .insert(chatDeliveries) - .values({ - companyId: endpoint.companyId, - endpointId: endpoint.id, - providerEventId, - deduplicationKey: createHash("sha256") - .update(providerEventId) - .digest("hex"), - eventKind, - normalizedEvent: redactDestinationDelivery - ? redactedDestinationNormalized - : deferDrainUntilFollowup && accepting - ? { - ...normalized, - providerThreadPending: true, - } - : normalized, - state: accepting - ? deferDrainUntilFollowup - ? "retry" - : "received" - : "filtered", - nextAttemptAt: - accepting && deferDrainUntilFollowup - ? new Date( - Date.now() + DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS, - ) - : accepting && scheduledAt - ? new Date(scheduledAt) - : null, - redactedError: - accepting && deferDrainUntilFollowup - ? "Waiting for Discord thread creation confirmation" - : accepting - ? null - : inactiveReason, - processedAt: ignoredAt, - }) - .onConflictDoNothing() - .returning(); - candidate = delivery ?? null; - } - if (!candidate && !admittedDeliveryId) { - const existingDelivery = await tx - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.endpointId, endpoint.id), - eq(chatDeliveries.providerEventId, providerEventId), - ), - ) - .then((rows) => rows[0] ?? null); - if (existingDelivery) { - const hydrateTeamsDelivery = - endpoint.provider === "microsoft-teams" && - !thread.isDM && - accepting && - !provisionalTeamsSetupReply && - ["received", "retry", "processing"].includes( - existingDelivery.state, - ) && - deliveryContentWasRedacted(existingDelivery.normalizedEvent); - await notifyChatDeliveryWork(tx); - [candidate] = await tx - .update(chatDeliveries) - .set({ - // Increment under PostgreSQL's row lock so simultaneous handler - // fanout and provider retries cannot lose duplicate telemetry. - normalizedEvent: redactDestinationDelivery - ? sql`${JSON.stringify(redactedDestinationNormalized)}::jsonb - || jsonb_build_object( - 'deduplication', - jsonb_build_object( - 'duplicateCount', - coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, - 'lastDuplicateAt', - ${new Date().toISOString()}::text - ) - )` - : hydrateTeamsDelivery - ? sql`${JSON.stringify(normalized)}::jsonb - || jsonb_build_object( - 'deduplication', - coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb) - || jsonb_build_object( - 'duplicateCount', - coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, - 'lastDuplicateAt', - ${new Date().toISOString()}::text - ) - )` - : sql`coalesce(${chatDeliveries.normalizedEvent}, '{}'::jsonb) - || jsonb_build_object( - 'deduplication', - coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb) - || jsonb_build_object( - 'duplicateCount', - coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, - 'lastDuplicateAt', - ${new Date().toISOString()}::text - ) - )`, - ...(!accepting && - ["received", "retry"].includes(existingDelivery.state) - ? { - state: "filtered" as const, - nextAttemptAt: null, - redactedError: inactiveReason, - processedAt: ignoredAt, - } - : {}), - updatedAt: new Date(), - }) - .where(eq(chatDeliveries.id, existingDelivery.id)) - .returning(); - } - } - if ( - candidate && - admittedDeliveryId && - accepting && - !provisionalTeamsSetupReply && - endpoint.provider === "microsoft-teams" && - !thread.isDM && - ["received", "retry", "processing"].includes(candidate.state) && - deliveryContentWasRedacted(candidate.normalizedEvent) - ) { - [candidate] = await tx - .update(chatDeliveries) - .set({ normalizedEvent: normalized, updatedAt: new Date() }) - .where(eq(chatDeliveries.id, candidate.id)) - .returning(); - } - if ( - candidate && - admittedDeliveryId && - !accepting && - ["received", "retry", "processing"].includes(candidate.state) - ) { - await notifyChatDeliveryWork(tx); - [candidate] = await tx - .update(chatDeliveries) - .set({ - state: "filtered", - ...(redactDestinationDelivery - ? { - normalizedEvent: redactedDestinationNormalized, - principalId: null, - } - : {}), - nextAttemptAt: null, - redactedError: inactiveReason, - processedAt: ignoredAt, - updatedAt: new Date(), - }) - .where(eq(chatDeliveries.id, candidate.id)) - .returning(); - } - if (!accepting && ignoredAt) { - await tx - .update(chatEndpoints) - .set({ lastEventAt: ignoredAt, updatedAt: ignoredAt }) - .where(eq(chatEndpoints.id, endpoint.id)); - } - return { - accepting, - candidate: candidate ?? null, - provisionalTeamsSetupReply, - }; - }); + const admission = await admitInboundMessage({ endpoint, thread, message, trigger, runtimeContext, admittedDeliveryId, deferDrainUntilFollowup, scheduledAt, envelope: { providerEventId, addressed, eventKind, normalized, redactedDestinationNormalized, githubAutomatic, githubIssue } }, discordGatewayAdmissionState); // The database token is authoritative. Publish its new expiry to the // process-local fast path only after the admission transaction commits. // Conversely, synchronously fence an obsolete local owner after a failed @@ -6930,76 +3347,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { message.author, message.raw, ); - const mayEnableSetupDestination = - // Slack channels start enabled. Never use first-test activation to - // override an existing channel the operator explicitly disabled. - endpoint.provider !== "slack" && - endpoint.provider !== "imessage-photon" && - !thread.isDM && - endpoint.status === "verifying" && - addressed && - !suppressSetupDestinationActivation && - !( - endpoint.provider === "microsoft-teams" && - surfaceKind === "linear_group" - ); - const resourceAdmission = mayEnableSetupDestination - ? await db.transaction(async (tx) => { - const currentEndpoint = await tx - .select({ status: chatEndpoints.status }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpoint.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!currentEndpoint) throw notFound("Chat endpoint not found"); - const enabledResourceCount = await tx - .select({ count: sql`count(*)::int` }) - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.enabled, true), - eq(chatEndpointResources.availability, "available"), - ne(chatEndpointResources.type, "direct_message"), - ne(chatEndpointResources.type, "group_chat"), - ), - ) - .then((rows) => rows[0]?.count ?? 0); - const enableSetupDestination = - currentEndpoint.status === "verifying" && - enabledResourceCount === 0; - let resource = await ensureResource( - endpoint, - thread, - enableSetupDestination, - tx, - ); - if (enableSetupDestination && !resource.enabled) { - resource = await tx - .update(chatEndpointResources) - .set({ enabled: true, updatedAt: new Date() }) - .where(eq(chatEndpointResources.id, resource.id)) - .returning() - .then((rows) => rows[0] ?? resource); - } - return { enabledResourceCount, resource }; - }) - : { - enabledResourceCount: await db - .select({ count: sql`count(*)::int` }) - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.enabled, true), - eq(chatEndpointResources.availability, "available"), - ne(chatEndpointResources.type, "direct_message"), - ne(chatEndpointResources.type, "group_chat"), - ), - ) - .then((rows) => rows[0]?.count ?? 0), - resource: await ensureResource(endpoint, thread, false), - }; + const resourceAdmission = await admitMessageResource({ endpoint, thread, addressed, suppressSetupDestinationActivation, surfaceKind }); const { enabledResourceCount, resource } = resourceAdmission; await db .update(chatDeliveries) @@ -8079,816 +4427,10 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { throw error; } } - - function deliveryContentWasRedacted(normalizedEvent: unknown): boolean { - if ( - !normalizedEvent || - typeof normalizedEvent !== "object" || - Array.isArray(normalizedEvent) - ) - return false; - const filtering = (normalizedEvent as { filtering?: unknown }).filtering; - return ( - Boolean(filtering) && - typeof filtering === "object" && - !Array.isArray(filtering) && - (filtering as { contentRetained?: unknown }).contentRetained === false - ); + async function drainConversationDeliveries(...args: Parameters["drainConversationDeliveries"]>) { + return conversationDeliveryQueue.drainConversationDeliveries(...args); } - function setupDestinationActivationSuppressed( - normalizedEvent: unknown, - ): boolean { - if ( - !normalizedEvent || - typeof normalizedEvent !== "object" || - Array.isArray(normalizedEvent) - ) - return false; - const normalized = normalizedEvent as { - admission?: { origin?: unknown }; - admissionAuthorization?: { version?: unknown }; - }; - return ( - normalized.admission?.origin === "provider_confirmed_action" || - normalized.admissionAuthorization?.version === 1 - ); - } - - function normalizedDeliveryThreadId(delivery: Pick): string | null { - const normalized = delivery.normalizedEvent as { - conversation?: { externalThreadId?: unknown }; - }; - return typeof normalized.conversation?.externalThreadId === "string" - ? normalized.conversation.externalThreadId - : null; - } - - function normalizedLifecycleTargetEventId( - delivery: Pick, - ): string | null { - if ( - delivery.eventKind !== "message_updated" && - delivery.eventKind !== "message_deleted" && - delivery.eventKind !== "message_restored" - ) - return null; - const normalized = delivery.normalizedEvent as { - message?: { targetProviderEventId?: unknown; admissionDependencyEventId?: unknown }; - }; - // An edit can also introduce a new addressed request. Process that durable - // admission before waiting for the old, potentially unadmitted source. - if (typeof normalized.message?.admissionDependencyEventId === "string") - return normalized.message.admissionDependencyEventId; - return typeof normalized.message?.targetProviderEventId === "string" - ? normalized.message.targetProviderEventId - : null; - } - - function normalizedLifecycleEffect( - delivery: Pick, - ): ChatProviderLifecycleEffect | null { - const normalized = delivery.normalizedEvent as { lifecycle?: unknown }; - const value = normalized.lifecycle; - if (!value || typeof value !== "object" || Array.isArray(value)) - return null; - const effect = value as Record; - if ( - (effect.provider !== "slack" && - effect.provider !== "github" && - effect.provider !== "discord" && - effect.provider !== "microsoft-teams" && - effect.provider !== "telegram") || - typeof effect.providerEventId !== "string" - ) - return null; - if ( - effect.kind === "endpoint" && - (effect.availability === "available" || - effect.availability === "attention" || - effect.availability === "revoked") && - typeof effect.reason === "string" - ) - return effect as ChatProviderLifecycleEffect; - if ( - effect.kind === "resource" && - (effect.availability === "available" || - effect.availability === "unavailable" || - effect.availability === "removed") && - typeof effect.providerResourceId === "string" && - (effect.previousProviderResourceId === undefined || - typeof effect.previousProviderResourceId === "string") && - typeof effect.resourceType === "string" && - typeof effect.label === "string" - ) - return effect as ChatProviderLifecycleEffect; - return null; - } - - function deliveryReady(delivery: DeliveryRow, now: Date): boolean { - // Accepted input may still own the conversation head through its wake - // outbox. The action dispatcher checks its own retry/claim deadline. - if (delivery.state === "processed") return true; - if (delivery.state === "received") - return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now; - if (delivery.state === "retry") { - return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now; - } - return ( - delivery.state === "processing" && - delivery.updatedAt <= - new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS) - ); - } - - function conversationDeliveryOrder() { - return [ - asc( - sql`coalesce(nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')::timestamptz, ${chatDeliveries.receivedAt})`, - ), - // GitHub timestamps and Telegram message dates have one-second - // resolution. GitHub's numeric comment id and Telegram's message_id - // are monotonic within one conversation, so use them before receipt - // order. The Telegram update_id is a final provider-native tie-breaker - // for unusual payloads that lack a usable message_id. - asc(sql`coalesce( - case - when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence' ~ '^[0-9]+$' - then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence')::numeric - else null - end, - case - when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId' ~ '^[0-9]+$' - then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId')::numeric - else null - end - )`), - asc(sql`case - when ${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId' ~ '^[0-9]+$' - then (${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId')::numeric - else null - end`), - // GitHub exposes no sortable webhook sequence. When an edit and delete - // share its whole-second updated_at value, preserve the only valid - // lifecycle state transition: update before delete. Provider-native - // update ids (Telegram) remain the stronger preceding key. - asc(sql`case ${chatDeliveries.eventKind} - when 'message_updated' then 1 - when 'message_deleted' then 2 - when 'message_restored' then 3 - else 0 - end`), - asc(chatDeliveries.receivedAt), - asc(chatDeliveries.id), - ]; - } - - async function earliestOpenConversationDelivery( - endpointId: string, - threadId: string, - ): Promise { - const externalThreadId = sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId'`; - const candidate = await db - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.endpointId, endpointId), - or( - inArray(chatDeliveries.state, ["received", "retry", "processing"]), - and( - eq(chatDeliveries.state, "processed"), - pendingInboundWakeupCondition(), - ), - ), - notInArray(chatDeliveries.eventKind, [ - "reaction_added", - "reaction_removed", - ]), - externalThreadIdentityCondition(externalThreadId, threadId), - ), - ) - .orderBy(...conversationDeliveryOrder()) - .limit(1) - .then((rows) => rows[0] ?? null); - if (!candidate) return null; - - // A lifecycle callback can reach Paperclip before its create callback. If - // the root becomes durable while the lifecycle row is waiting, always - // return that exact dependency first even when a coarse or malformed - // provider timestamp would otherwise put the edit/delete at the head. - const targetProviderEventId = normalizedLifecycleTargetEventId(candidate); - if (!targetProviderEventId) return candidate; - return db - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.endpointId, endpointId), - eq(chatDeliveries.providerEventId, targetProviderEventId), - inArray(chatDeliveries.state, ["received", "retry", "processing"]), - ), - ) - .limit(1) - .then((rows) => rows[0] ?? candidate); - } - - function pendingInboundWakeupCondition(readyOnly = false): SQL { - return sql`exists ( - select 1 from ${chatActions} - where ${chatActions.deliveryId} = ${chatDeliveries.id} - and ${chatActions.endpointId} = ${chatDeliveries.endpointId} - and ${chatActions.companyId} = ${chatDeliveries.companyId} - and ${chatActions.kind} = 'inbound_wakeup' - and ${chatActions.status} in ('issued', 'processing') - ${ - readyOnly - ? sql`and ( - (${chatActions.status} = 'issued' and (${chatActions.result}->>'retryAt' is null or (${chatActions.result}->>'retryAt')::timestamptz <= now())) - or (${chatActions.status} = 'processing' and ${chatActions.updatedAt} <= ${new Date(Date.now() - DELIVERY_PROCESSING_STALE_MS).toISOString()}::timestamptz) - )` - : sql`` - } - )`; - } - - async function acquireConversationDeliveryLease(input: { - companyId: string; - endpointId: string; - threadId: string; - }): Promise<{ leaseKey: string; token: string } | null> { - const now = new Date(); - const token = randomUUID(); - const leaseKey = `inbound:${createHash("sha256").update(durableExternalThreadIdentity(input.threadId)).digest("hex")}`; - const expiresAt = new Date(now.getTime() + DELIVERY_LEASE_TTL_MS); - const inserted = await db - .insert(chatEndpointLeases) - .values({ - companyId: input.companyId, - endpointId: input.endpointId, - leaseKey, - token, - expiresAt, - }) - .onConflictDoNothing() - .returning({ id: chatEndpointLeases.id }); - if (inserted.length > 0) return { leaseKey, token }; - const reclaimed = await db - .update(chatEndpointLeases) - .set({ token, expiresAt, updatedAt: now }) - .where( - and( - eq(chatEndpointLeases.companyId, input.companyId), - eq(chatEndpointLeases.endpointId, input.endpointId), - eq(chatEndpointLeases.leaseKey, leaseKey), - lte(chatEndpointLeases.expiresAt, now), - ), - ) - .returning({ id: chatEndpointLeases.id }); - return reclaimed.length > 0 ? { leaseKey, token } : null; - } - - async function releaseConversationDeliveryLease(input: { - endpointId: string; - leaseKey: string; - token: string; - }): Promise { - await db - .delete(chatEndpointLeases) - .where( - and( - eq(chatEndpointLeases.endpointId, input.endpointId), - eq(chatEndpointLeases.leaseKey, input.leaseKey), - eq(chatEndpointLeases.token, input.token), - ), - ); - } - - function messageFromDelivery( - delivery: DeliveryRow, - thread: Thread, - endpointRuntime: ChatEndpointRuntime, - ): { - message: Message; - receiptReactionSupported: boolean; - suppressSetupDestinationActivation: boolean; - trigger: ChatSdkMessageCallbackEvent["trigger"]; - providerUrl: string | null; - } | null { - if (telegramDeliveryHasZeroMessageId(delivery, endpointRuntime.provider)) - return null; - const normalized = delivery.normalizedEvent as { - acknowledgement?: { receiptReactionSupported?: unknown }; - kind?: unknown; - trigger?: unknown; - principal?: { - externalId?: unknown; - displayName?: unknown; - handle?: unknown; - }; - message?: { - providerMessageId?: unknown; - sourceProviderMessageId?: unknown; - text?: unknown; - mentionedBot?: unknown; - attachments?: Array<{ - name?: unknown; - mimeType?: unknown; - size?: unknown; - recovery?: unknown; - }>; - attachmentLimitOmissions?: unknown; - photonReply?: { guid?: unknown; part?: unknown }; - }; - conversation?: { providerUrl?: unknown }; - }; - const providerMessageId = normalized.message?.providerMessageId; - const externalId = normalized.principal?.externalId; - if (typeof providerMessageId !== "string" || typeof externalId !== "string") - return null; - const isTeams = endpointRuntime.provider === "microsoft-teams"; - const referenceOnlyTeamsAttachments = isTeams && !thread.isDM; - const originFence = lifecycleRuntimeFence(delivery); - const storedAttachments = normalized.message?.attachments ?? []; - const attachments = ( - isTeams ? storedAttachments.slice(0, 20) : storedAttachments - ) - .map((attachment) => { - if (!attachment || typeof attachment !== "object") return null; - const rehydrated = - attachment.recovery && - (!referenceOnlyTeamsAttachments || - (typeof attachment.recovery === "object" && - attachment.recovery !== null && - "locator" in attachment.recovery && - typeof attachment.recovery.locator === "object" && - attachment.recovery.locator !== null && - "kind" in attachment.recovery.locator && - attachment.recovery.locator.kind === "teams_inline_image")) - ? endpointRuntime.rehydrateAttachment(attachment.recovery, { - threadId: thread.id, - messageId: typeof normalized.message?.sourceProviderMessageId === "string" ? normalized.message.sourceProviderMessageId : providerMessageId, - runtimeGeneration: originFence?.generation, - credentialFingerprint: originFence?.credentialFingerprint, - principalExternalId: externalId, - isDirectMessage: thread.isDM, - }) - : null; - if (rehydrated) return rehydrated; - if ( - isTeams || - endpointRuntime.provider === "imessage-photon" || - (endpointRuntime.provider === "telegram" && - typeof attachment.recovery === "object" && - attachment.recovery !== null && - "locator" in attachment.recovery && - typeof attachment.recovery.locator === "object" && - attachment.recovery.locator !== null && - "kind" in attachment.recovery.locator && - attachment.recovery.locator.kind === "telegram_media") - ) { - // Non-personal references and personal files whose safe download - // capability did not survive restart still belong to this input. - // Keep bounded metadata so ingestion records an explicit omission; - // never persist bearer URLs or fabricate a download capability. - // Even a legacy URL locator cannot enable non-personal downloads. - // A Telegram media locator which fails its exact source proof also - // remains an unavailable input, never an empty attachment-free turn. - return { - type: "file", - name: - sanitizeFilename( - typeof attachment.name === "string" - ? attachment.name - : undefined, - ) ?? undefined, - mimeType: normalizeContentType( - typeof attachment.mimeType === "string" - ? attachment.mimeType - : undefined, - ).slice(0, 255), - size: - typeof attachment.size === "number" && - Number.isSafeInteger(attachment.size) && - attachment.size >= 0 - ? attachment.size - : undefined, - } as Attachment; - } - return null; - }) - .filter((attachment): attachment is Attachment => Boolean(attachment)); - const message = { - id: providerMessageId, - threadId: thread.id, - text: - typeof normalized.message?.text === "string" - ? normalized.message.text - : "", - formatted: { type: "root", children: [] }, - raw: - endpointRuntime.provider === "imessage-photon" && - typeof normalized.message?.photonReply?.guid === "string" - ? { - replyTargetGuid: normalized.message.photonReply.guid, - threadOriginatorPart: normalized.message.photonReply.part, - } - : endpointRuntime.provider === "github" && typeof normalized.message?.sourceProviderMessageId === "string" - ? { githubSourceMessageId: normalized.message.sourceProviderMessageId } : {}, - author: { - userId: externalId, - userName: - typeof normalized.principal?.handle === "string" - ? normalized.principal.handle - : externalId, - fullName: - typeof normalized.principal?.displayName === "string" - ? normalized.principal.displayName - : externalId, - isBot: false, - isMe: false, - isSystem: false, - }, - metadata: { dateSent: delivery.receivedAt, edited: false }, - attachments, - links: [], - isMention: normalized.message?.mentionedBot === true, - } as unknown as Message; - const githubManual = delivery.normalizedEvent.githubManual; - if (endpointRuntime.provider === "github" && githubManual && typeof githubManual === "object") githubManualMessages.set(message, githubManual as { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }); - const githubAutomatic = delivery.normalizedEvent.githubAutomatic; - if (endpointRuntime.provider === "github" && githubAutomatic && typeof githubAutomatic === "object") { - githubAutomaticMessages.set(message, githubAutomatic as { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }); - } - const githubIssue = delivery.normalizedEvent.githubIssue; - if (endpointRuntime.provider === "github" && githubIssue && typeof githubIssue === "object") { - githubIssueMessages.set(message, githubIssue as { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }); - } - if ( - endpointRuntime.provider === "github" || - endpointRuntime.provider === "imessage-photon" - ) { - restoreGitHubAttachmentLimitOmissions( - message, - normalized.message?.attachmentLimitOmissions, - ); - } - const normalizedTrigger = normalized.trigger; - const trigger: ChatSdkMessageCallbackEvent["trigger"] = - normalizedTrigger === "direct_message" || - normalizedTrigger === "mention" || - normalizedTrigger === "subscribed_message" || - normalizedTrigger === "unaddressed_message" - ? normalizedTrigger - : normalized.kind === "direct_message" - ? "direct_message" - : normalized.kind === "mention" - ? "mention" - : "subscribed_message"; - return { - message, - // Legacy deliveries all originated from native provider messages and - // therefore retain the historical default. - receiptReactionSupported: - normalized.acknowledgement?.receiptReactionSupported !== false, - suppressSetupDestinationActivation: setupDestinationActivationSuppressed( - delivery.normalizedEvent, - ), - trigger, - providerUrl: - typeof normalized.conversation?.providerUrl === "string" - ? normalized.conversation.providerUrl - : null, - }; - } - - /** - * Drain one external conversation in durable receipt order. The database - * lease is endpoint + thread scoped, so separate server processes can work - * on different conversations concurrently but can never mutate the same - * Paperclip task from two inbound turns at once. - */ - async function drainConversationDeliveries( - endpointId: string, - threadId: string, - ): Promise { - const endpoint = await db - .select() - .from(chatEndpoints) - .where(eq(chatEndpoints.id, endpointId)) - .then((rows) => rows[0] ?? null); - if (!endpoint) return false; - if (endpoint.status === "paused" || endpoint.status === "attention") - return false; - - const lease = await acquireConversationDeliveryLease({ - companyId: endpoint.companyId, - endpointId, - threadId, - }); - if (!lease) return false; - - let leaseOwned = true; - let renewal: Promise | null = null; - const renewTimer = setInterval( - () => { - if (!leaseOwned || renewal) return; - const expiresAt = new Date(Date.now() + DELIVERY_LEASE_TTL_MS); - renewal = Promise.resolve() - .then(async () => { - const renewed = options.renewConversationDeliveryLease - ? await options.renewConversationDeliveryLease({ - endpointId, - leaseKey: lease.leaseKey, - token: lease.token, - expiresAt, - }) - : await db - .update(chatEndpointLeases) - .set({ expiresAt, updatedAt: new Date() }) - .where( - and( - eq(chatEndpointLeases.endpointId, endpointId), - eq(chatEndpointLeases.leaseKey, lease.leaseKey), - eq(chatEndpointLeases.token, lease.token), - ), - ) - .returning({ id: chatEndpointLeases.id }) - .then((rows) => rows.length > 0); - if (!renewed) leaseOwned = false; - }) - .catch((error) => { - // A failed renewal has unknown durability. Stop this local drain at - // the next delivery boundary so another worker can never overlap - // later turns after the original lease expires or is reclaimed. - leaseOwned = false; - logger.warn( - { endpointId, error: redactError(error) }, - "could not renew external chat conversation lease", - ); - }) - .finally(() => { - renewal = null; - }); - }, - options.conversationLeaseRenewalIntervalMs ?? DELIVERY_LEASE_TTL_MS / 3, - ); - renewTimer.unref?.(); - - try { - for ( - let processed = 0; - processed < DELIVERY_DRAIN_LIMIT; - processed += 1 - ) { - if (!leaseOwned) break; - const delivery = await earliestOpenConversationDelivery( - endpointId, - threadId, - ); - if (!delivery || !deliveryReady(delivery, new Date())) break; - if (delivery.state === "processed") { - if (!(await processInboundWakeup(delivery.id))) break; - continue; - } - if (telegramDeliveryHasZeroMessageId(delivery, endpoint.provider)) { - const filtered = await db - .transaction(async (tx) => { - // Do not round-trip a PostgreSQL microsecond timestamp through - // JS Date for a CAS. Lock the exact receipt, then revalidate the - // current source and claim before changing it. A competing live - // claim is a retry boundary, not permission to settle its work. - const [current] = await tx - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.id, delivery.id), - eq(chatDeliveries.companyId, endpoint.companyId), - eq(chatDeliveries.endpointId, endpoint.id), - ), - ) - .for("update", { noWait: true }); - const filteredAt = new Date(); - if ( - !current || - current.state !== delivery.state || - current.eventKind !== delivery.eventKind || - !["received", "retry", "processing"].includes(current.state) || - normalizedDeliveryThreadId(current) !== threadId || - !deliveryReady(current, filteredAt) || - !telegramDeliveryHasZeroMessageId(current, endpoint.provider) - ) - return false; - await notifyChatDeliveryWork(tx); - const changed = await tx - .update(chatDeliveries) - .set({ - state: "filtered", - nextAttemptAt: null, - processedAt: filteredAt, - updatedAt: filteredAt, - redactedError: - "Telegram message ID zero cannot authorize ordinary work", - }) - .where(eq(chatDeliveries.id, current.id)) - .returning({ id: chatDeliveries.id }); - return changed.length === 1; - }) - .catch((error: unknown) => { - if (isExternalChatWaitAuthorizationContention(error)) - return false; - throw error; - }); - if (!filtered) break; - // Retain original evidence/comments; filtering adds no user content - // and cannot hydrate attachments, invoke commands, or send feedback. - await settleRejectedInboundWakeups(delivery.id); - liveInboundMessages.delete(delivery.id); - continue; - } - if (endpoint.status === "archived" || endpoint.status === "revoked") { - await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: "filtered", - redactedError: "Connection was removed before processing", - processedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(chatDeliveries.id, delivery.id)); - }); - liveInboundMessages.delete(delivery.id); - continue; - } - - if ( - endpoint.provider === "discord" && - discordRootThreadPending(delivery) - ) { - // This is the only recovery path for the narrow crash window after - // pre-admission but before Chat's ordinary callback. A successful - // read proves the root thread exists; an explicit 404/410 proves it - // does not. Ambiguous transport and auth failures remain retryable. - if (await reconcileDiscordRootThread(endpoint, delivery, threadId)) { - continue; - } - break; - } - - const live = liveInboundMessages.get(delivery.id); - try { - if ( - delivery.eventKind === "message_updated" || - delivery.eventKind === "message_deleted" || - delivery.eventKind === "message_restored" - ) { - await processLifecycleDelivery(endpoint, delivery); - } else if (live) { - await processMessage( - endpoint, - live.thread, - live.message, - live.trigger, - false, - live.providerUrl, - live.runtimeContext, - undefined, - delivery.id, - live.receiptReactionSupported, - undefined, - false, - setupDestinationActivationSuppressed(delivery.normalizedEvent), - ); - } else { - if (deliveryContentWasRedacted(delivery.normalizedEvent)) { - const filteredAt = new Date(); - await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: "filtered", - principalId: null, - nextAttemptAt: null, - processedAt: filteredAt, - redactedError: - "Provisional Teams setup reply could not be hydrated from a current provider event", - updatedAt: filteredAt, - }) - .where( - and( - eq(chatDeliveries.id, delivery.id), - inArray(chatDeliveries.state, [ - "received", - "retry", - "processing", - ]), - ), - ); - }); - continue; - } - const endpointRuntime = await runtimeFor(endpoint); - const deliveryRuntimeFence = lifecycleRuntimeFence(delivery); - const currentRuntimeContext = runtimeContexts.get( - endpointRuntime as object, - ); - const thread = endpointRuntime.thread(threadId); - const reconstructed = messageFromDelivery( - delivery, - thread, - endpointRuntime, - ); - if (!reconstructed) { - await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: "failed", - redactedError: "Normalized delivery is incomplete", - updatedAt: new Date(), - }) - .where(eq(chatDeliveries.id, delivery.id)); - }); - continue; - } - await processMessage( - endpoint, - thread, - reconstructed.message, - reconstructed.trigger, - false, - reconstructed.providerUrl, - deliveryRuntimeFence - ? { - ...deliveryRuntimeFence, - endpointRuntime, - } - : currentRuntimeContext, - undefined, - delivery.id, - reconstructed.receiptReactionSupported, - undefined, - false, - reconstructed.suppressSetupDestinationActivation, - ); - } - } catch (error) { - logger.warn( - { - endpointId, - deliveryId: delivery.id, - error: redactError(error), - }, - "external chat delivery will retry in receipt order", - ); - } - - const state = await db - .select({ state: chatDeliveries.state }) - .from(chatDeliveries) - .where(eq(chatDeliveries.id, delivery.id)) - .then((rows) => rows[0]?.state ?? null); - if ( - state === "processed" || - state === "filtered" || - state === "failed" - ) { - if (state !== "processed") - await settleRejectedInboundWakeups(delivery.id); - liveInboundMessages.delete(delivery.id); - continue; - } - // A retry delay or another still-live processing claim is a strict - // head-of-line barrier; later replies may not overtake it. - break; - } - } finally { - clearInterval(renewTimer); - await renewal; - await releaseConversationDeliveryLease({ endpointId, ...lease }); - } - - const next = await earliestOpenConversationDelivery(endpointId, threadId); - if (next?.state === "processed") { - const [ready] = await db - .select({ id: chatDeliveries.id }) - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.id, next.id), - pendingInboundWakeupCondition(true), - ), - ) - .limit(1); - return Boolean(ready); - } - return Boolean(next && deliveryReady(next, new Date())); - } async function handleSdkMessage( event: ChatSdkMessageCallbackEvent, @@ -8933,3947 +4475,14 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { messageOptions.slackSlashControl === true, ); } - - async function conversationForThread(endpointId: string, threadId: string) { - const exact = await db - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.endpointId, endpointId), - eq(chatConversations.externalThreadId, threadId), - ), - ) - .orderBy(desc(chatConversations.sessionGeneration)) - .then((rows) => rows[0] ?? null); - if (exact || !threadId.startsWith("teams:")) return exact; - - // Before Paperclip made Teams route state independent, the adapter - // embedded the mutable Bot Connector serviceUrl in every thread id. Read - // those rows by the immutable conversation id so a regional route change - // continues the same task. New rows use the canonical route-free id. - if (!teamsConversationId(threadId)) return null; - return db - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.endpointId, endpointId), - externalThreadIdentityCondition( - sql`${chatConversations.externalThreadId}`, - threadId, - ), - ), - ) - .orderBy(desc(chatConversations.sessionGeneration)) - .then( - (rows) => - rows.find((row) => - sameTeamsThreadIdentity(row.externalThreadId, threadId), - ) ?? null, - ); + async function handleMessageUpdated(...args: Parameters["handleMessageUpdated"]>) { + return messageLifecycle.handleMessageUpdated(...args); } - async function recordLifecycleDelivery( - input: { - actor?: LifecycleActor; - isBotMessage?: true; - endpointId: string; - threadId: string; - messageId: string; - eventKind: "message_updated" | "message_deleted" | "message_restored"; - text: string; - providerEventId?: string; - providerMessageSequence?: number | null; - providerSentAt?: string | null; - providerUpdateId?: number | null; - raw?: unknown; - revision?: string | null; - admissionDependencyEventId?: string; - }, - runtimeContext?: RuntimeContext, - githubSourceMapped = false, - ) { - if (!githubSourceMapped && input.threadId.startsWith("github:") && /^[1-9][0-9]*$/.test(input.messageId)) { - const requests = await db.select({ id: chatDeliveries.id, messageId: sql`${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId'` }) - .from(chatDeliveries).innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatDeliveries.endpointId), eq(chatEndpoints.companyId, chatDeliveries.companyId))) - .where(and(eq(chatDeliveries.endpointId, input.endpointId), eq(chatEndpoints.provider, "github"), - inArray(chatDeliveries.eventKind, ["mention", "message", "direct_message"]), - input.admissionDependencyEventId ? ne(chatDeliveries.providerEventId, input.admissionDependencyEventId) : undefined, - sql`${chatDeliveries.normalizedEvent}->'message'->>'sourceProviderMessageId' = ${input.messageId}`, - sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId' = ${input.threadId}`)); - if (requests.length) { - for (const request of requests) await recordLifecycleDelivery({ ...input, messageId: request.messageId, - providerEventId: `${input.providerEventId ?? `${input.eventKind}:${input.threadId}:${input.messageId}:${input.revision ?? "once"}`}:${request.id}` }, runtimeContext, true); - return; - } - } - const durableThreadId = durableExternalThreadIdentity(input.threadId); - const providerEventId = - input.providerEventId ?? - `${input.eventKind}:${durableThreadId}:${input.messageId}:${input.revision ?? "once"}`; - const targetProviderEventId = `${durableThreadId}:${input.messageId}`; - const admission = await db.transaction(async (tx) => { - const endpoint = await tx - .select() - .from(chatEndpoints) - .where(eq(chatEndpoints.id, input.endpointId)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!endpoint) throw notFound("Chat endpoint not found"); - if ( - runtimeContext && - !(await runtimeCallbackEndpoint(tx, endpoint.id, runtimeContext, [ - "verifying", - "active", - ])) - ) - return null; - // A provider has already authenticated this callback. During setup the - // message can belong to the test conversation, so retain its correction. - // Paused or unhealthy connections acknowledge late callbacks without - // mutating the bound task or causing provider retry storms. - if (endpoint.status !== "verifying" && endpoint.status !== "active") - return null; - const reorderWindow = INGRESS_REORDER_WINDOW_MS[endpoint.provider]; - const scheduledAt = - options.deferWebhookProcessing === true && reorderWindow - ? (scheduledConversationDrains.get( - conversationDrainKey(input.endpointId, input.threadId), - ) ?? Date.now() + reorderWindow) - : null; - await notifyChatDeliveryWork(tx); - const [delivery] = await tx - .insert(chatDeliveries) - .values({ - companyId: endpoint.companyId, - endpointId: input.endpointId, - providerEventId, - deduplicationKey: createHash("sha256") - .update(providerEventId) - .digest("hex"), - eventKind: input.eventKind, - normalizedEvent: { - providerEventId, - kind: input.eventKind, - ...(runtimeContext - ? { - runtimeContext: { - credentialFingerprint: runtimeContext.credentialFingerprint, - generation: runtimeContext.generation, - }, - } - : {}), - ...(input.actor ? { principal: input.actor } : {}), - conversation: { externalThreadId: input.threadId }, - message: { - providerMessageId: input.messageId, - targetProviderEventId, - ...(input.admissionDependencyEventId ? { admissionDependencyEventId: input.admissionDependencyEventId } : {}), - ...(endpoint.provider === "github" && input.isBotMessage === true - ? { isBotMessage: true } - : {}), - providerMessageSequence: input.providerMessageSequence ?? null, - providerSentAt: input.providerSentAt ?? null, - providerUpdateId: input.providerUpdateId ?? null, - text: input.text, - }, - }, - state: "received", - nextAttemptAt: scheduledAt ? new Date(scheduledAt) : null, - }) - .onConflictDoNothing() - .returning(); - if (delivery) return { delivery, endpoint, scheduledAt }; - const existing = await tx - .select() - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.endpointId, input.endpointId), - eq(chatDeliveries.providerEventId, providerEventId), - ), - ) - .then((rows) => rows[0] ?? null); - return existing ? { delivery: existing, endpoint, scheduledAt } : null; - }); - if (admission) { - // Lifecycle callbacks can carry a new regional Bot Connector route. Keep - // that provider-authenticated route current even when processing is - // deferred, but deliberately omit the raw activity here. Lifecycle - // callbacks never populate the adapter's user cache; the admitted root - // message already established the principal metadata they must match. - await recordCurrentMicrosoftTeamsRoute( - admission.endpoint, - runtimeContext, - input.threadId, - input.raw, - false, - ); - } - if ( - !admission || - admission.delivery.state === "processed" || - admission.delivery.state === "filtered" || - admission.delivery.state === "failed" - ) - return; - if (options.deferWebhookProcessing === true) { - scheduleConversationDrain( - input.endpointId, - input.threadId, - admission.scheduledAt ?? Date.now(), - ); - return; - } - await drainConversationDeliveries(input.endpointId, input.threadId); + async function handleMessageDeleted(...args: Parameters["handleMessageDeleted"]>) { + return messageLifecycle.handleMessageDeleted(...args); } - function lifecycleMessageFromDelivery(delivery: DeliveryRow): { - actor: LifecycleActor | null; - isBotMessage: boolean; - messageId: string; - providerSentAt: string | null; - targetProviderEventId: string; - text: string; - threadId: string; - } | null { - if ( - delivery.eventKind !== "message_updated" && - delivery.eventKind !== "message_deleted" && - delivery.eventKind !== "message_restored" - ) - return null; - const normalized = delivery.normalizedEvent as { - principal?: { - displayName?: unknown; - externalId?: unknown; - handle?: unknown; - }; - message?: { - isBotMessage?: unknown; - providerMessageId?: unknown; - providerSentAt?: unknown; - targetProviderEventId?: unknown; - text?: unknown; - }; - }; - const messageId = normalized.message?.providerMessageId; - const targetProviderEventId = normalized.message?.targetProviderEventId; - const providerSentAt = normalized.message?.providerSentAt; - const text = normalized.message?.text; - const threadId = normalizedDeliveryThreadId(delivery); - const externalId = normalized.principal?.externalId; - const actor = - typeof externalId === "string" && externalId - ? { - externalId, - displayName: - typeof normalized.principal?.displayName === "string" - ? normalized.principal.displayName - : externalId, - handle: - typeof normalized.principal?.handle === "string" - ? normalized.principal.handle - : externalId, - } - : null; - return typeof messageId === "string" && - typeof targetProviderEventId === "string" && - typeof text === "string" && - threadId !== null - ? { - actor, - isBotMessage: normalized.message?.isBotMessage === true, - messageId, - providerSentAt: - typeof providerSentAt === "string" ? providerSentAt : null, - targetProviderEventId, - text, - threadId, - } - : null; - } - - async function processLifecycleDelivery( - endpoint: EndpointRow, - candidate: DeliveryRow, - ): Promise { - const lifecycle = lifecycleMessageFromDelivery(candidate); - if (!lifecycle) { - await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: "failed", - redactedError: "Normalized lifecycle delivery is incomplete", - updatedAt: new Date(), - }) - .where(eq(chatDeliveries.id, candidate.id)); - }); - return; - } - - const now = new Date(); - const staleBefore = new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS); - const claimConditions = [ - eq(chatDeliveries.id, candidate.id), - eq(chatDeliveries.state, candidate.state), - ]; - if (candidate.state === "processing") - claimConditions.push(lte(chatDeliveries.updatedAt, staleBefore)); - const [activeDelivery] = await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: "processing", - attempts: candidate.attempts + 1, - nextAttemptAt: null, - redactedError: null, - updatedAt: now, - }) - .where(and(...claimConditions)) - .returning(); - }); - if (!activeDelivery) return; - - try { - await db.transaction(async (tx) => { - const admittedRuntimeContext = lifecycleRuntimeFence(activeDelivery); - const currentEndpoint = admittedRuntimeContext - ? await runtimeCallbackEndpoint( - tx, - endpoint.id, - admittedRuntimeContext, - ["verifying", "active"], - ) - : null; - if (!admittedRuntimeContext || !currentEndpoint) { - const filteredAt = new Date(); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - nextAttemptAt: null, - processedAt: filteredAt, - redactedError: - "Message lifecycle callback belonged to a superseded runtime", - updatedAt: filteredAt, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - let githubBotSourceEdit = false; - if ( - currentEndpoint.provider === "github" && - activeDelivery.eventKind === "message_updated" - ) { - // Bot callbacks can arrive before the outbound publication commits - // its message link. The Boolean comes only from GitHub's verified - // sender/comment-author type, never from comment text. Legacy rows - // lack it, so also recognize exact already-published outbound links. - const linkedMessages = await tx - .select({ direction: chatMessageLinks.direction }) - .from(chatMessageLinks) - .innerJoin( - chatConversations, - and( - eq(chatConversations.id, chatMessageLinks.conversationId), - eq(chatConversations.companyId, chatMessageLinks.companyId), - eq(chatConversations.endpointId, chatMessageLinks.endpointId), - ), - ) - .where( - and( - eq(chatMessageLinks.companyId, currentEndpoint.companyId), - eq(chatMessageLinks.endpointId, currentEndpoint.id), - eq(chatMessageLinks.providerMessageId, lifecycle.messageId), - eq(chatConversations.externalThreadId, lifecycle.threadId), - ), - ); - const outbound = linkedMessages.some( - (row) => row.direction === "outbound", - ); - // A bot editor can change an already-admitted human source. That - // authenticated change revokes the old source, but never admits the - // bot's new content. Unknown/self/outbound updates remain suppressed. - githubBotSourceEdit = - lifecycle.isBotMessage && - !outbound && - linkedMessages.some((row) => row.direction === "inbound"); - if (outbound || (lifecycle.isBotMessage && !githubBotSourceEdit)) { - const filteredAt = new Date(); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - principalId: null, - nextAttemptAt: null, - processedAt: filteredAt, - updatedAt: filteredAt, - redactedError: - "Provider bot or outbound message updates do not create inbound work", - normalizedEvent: { - providerEventId: - activeDelivery.normalizedEvent.providerEventId ?? - activeDelivery.providerEventId, - kind: activeDelivery.eventKind, - conversation: { externalThreadId: lifecycle.threadId }, - message: { - providerMessageId: lifecycle.messageId, - targetProviderEventId: lifecycle.targetProviderEventId, - }, - filtering: { contentRetained: false }, - }, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - } - if ( - activeDelivery.eventKind === "message_updated" || - activeDelivery.eventKind === "message_deleted" || - activeDelivery.eventKind === "message_restored" - ) { - const latestDeleteTransition = await tx - .select({ - eventKind: chatDeliveries.eventKind, - providerSentAt: sql< - string | null - >`nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')`, - }) - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.companyId, activeDelivery.companyId), - eq(chatDeliveries.endpointId, endpoint.id), - inArray(chatDeliveries.eventKind, [ - "message_deleted", - "message_restored", - ]), - eq(chatDeliveries.state, "processed"), - externalThreadIdentityCondition( - sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId'`, - lifecycle.threadId, - ), - sql`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId' = ${lifecycle.targetProviderEventId}`, - ), - ) - .orderBy( - desc( - sql`coalesce(nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')::timestamptz, ${chatDeliveries.processedAt})`, - ), - desc(sql`case ${chatDeliveries.eventKind} - when 'message_restored' then 2 - when 'message_deleted' then 1 - else 0 - end`), - desc(chatDeliveries.processedAt), - desc(chatDeliveries.id), - ) - .limit(1) - .then((rows) => rows[0] ?? null); - const latestTransitionAt = latestDeleteTransition?.providerSentAt - ? Date.parse(latestDeleteTransition.providerSentAt) - : Number.NaN; - const incomingTransitionAt = lifecycle.providerSentAt - ? Date.parse(lifecycle.providerSentAt) - : Number.NaN; - const incomingTransitionIsStale = - latestDeleteTransition !== null && - Number.isFinite(latestTransitionAt) && - Number.isFinite(incomingTransitionAt) && - (incomingTransitionAt < latestTransitionAt || - (incomingTransitionAt === latestTransitionAt && - activeDelivery.eventKind === "message_deleted" && - latestDeleteTransition.eventKind === "message_restored")); - const editTargetsDeletedMessage = - activeDelivery.eventKind === "message_updated" && - latestDeleteTransition?.eventKind === "message_deleted"; - const editPredatesRestore = - activeDelivery.eventKind === "message_updated" && - latestDeleteTransition?.eventKind === "message_restored" && - Number.isFinite(latestTransitionAt) && - Number.isFinite(incomingTransitionAt) && - incomingTransitionAt < latestTransitionAt; - if ( - incomingTransitionIsStale || - editTargetsDeletedMessage || - editPredatesRestore - ) { - const filteredAt = new Date(); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "filtered", - normalizedEvent: { - providerEventId: - activeDelivery.normalizedEvent.providerEventId ?? - activeDelivery.providerEventId, - kind: activeDelivery.eventKind, - conversation: { externalThreadId: lifecycle.threadId }, - message: { - providerMessageId: lifecycle.messageId, - targetProviderEventId: lifecycle.targetProviderEventId, - }, - filtering: { contentRetained: false }, - }, - principalId: null, - nextAttemptAt: null, - processedAt: filteredAt, - redactedError: editTargetsDeletedMessage - ? "Message edit arrived after the provider message was deleted" - : "Message lifecycle callback was older than the current provider state", - updatedAt: filteredAt, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - } - const originalDelivery = await tx - .select({ state: chatDeliveries.state }) - .from(chatDeliveries) - .where( - and( - eq(chatDeliveries.endpointId, endpoint.id), - eq( - chatDeliveries.providerEventId, - lifecycle.targetProviderEventId, - ), - ), - ) - .then((rows) => rows[0] ?? null); - const linkedTargets = await tx - .select({ - conversationId: chatConversations.id, - externalThreadId: chatConversations.externalThreadId, - issueId: chatConversations.issueId, - }) - .from(chatMessageLinks) - .innerJoin( - chatConversations, - and( - eq(chatConversations.companyId, chatMessageLinks.companyId), - eq(chatConversations.id, chatMessageLinks.conversationId), - ), - ) - .where( - and( - eq(chatMessageLinks.endpointId, endpoint.id), - eq(chatMessageLinks.providerMessageId, lifecycle.messageId), - eq(chatMessageLinks.direction, "inbound"), - ), - ) - .orderBy(desc(chatConversations.sessionGeneration)) - .then((rows) => rows); - const linkedTarget = - linkedTargets.find((candidate) => - currentEndpoint.provider === "microsoft-teams" - ? sameTeamsThreadIdentity( - candidate.externalThreadId, - lifecycle.threadId, - ) - : candidate.externalThreadId === lifecycle.threadId, - ) ?? null; - - if (!linkedTarget) { - const originalIsOpen = - originalDelivery !== null && - ["received", "processing", "retry"].includes( - originalDelivery.state, - ); - const waitForRoot = - originalIsOpen || - activeDelivery.attempts <= ORPHAN_FOLLOW_UP_MAX_ATTEMPTS; - const resolutionAt = new Date(); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set( - waitForRoot - ? { - state: "retry", - nextAttemptAt: new Date( - resolutionAt.getTime() + ORPHAN_FOLLOW_UP_GRACE_MS, - ), - redactedError: originalIsOpen - ? "Waiting for the original message to finish processing" - : "Waiting briefly for the original message", - updatedAt: resolutionAt, - } - : { - state: "filtered", - nextAttemptAt: null, - redactedError: - "Original message was not admitted to this conversation", - processedAt: resolutionAt, - updatedAt: resolutionAt, - }, - ) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - - const currentConversation = await tx - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, currentEndpoint.companyId), - eq(chatConversations.endpointId, currentEndpoint.id), - eq(chatConversations.id, linkedTarget.conversationId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - const currentResource = - currentConversation?.resourceId && - !currentConversation.isDirectMessage - ? await tx - .select() - .from(chatEndpointResources) - .where( - and( - eq( - chatEndpointResources.companyId, - currentEndpoint.companyId, - ), - eq(chatEndpointResources.endpointId, currentEndpoint.id), - eq( - chatEndpointResources.id, - currentConversation.resourceId, - ), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const retainSourceInvalidation = async ( - principalId: string | null, - reason: string, - ): Promise => { - if ( - !currentConversation || - activeDelivery.eventKind !== "message_updated" - ) - return false; - const invalidatedAt = new Date(); - // Provider authentication proves the exact source changed. The - // editor's current Paperclip rights govern admitting new content, - // not whether a later regrant can resurrect the stale old source. - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: "processed", - conversationId: currentConversation.id, - principalId, - normalizedEvent: { - providerEventId: - activeDelivery.normalizedEvent.providerEventId ?? - activeDelivery.providerEventId, - kind: activeDelivery.eventKind, - runtimeContext: { - generation: admittedRuntimeContext.generation, - credentialFingerprint: - admittedRuntimeContext.credentialFingerprint, - }, - conversation: { externalThreadId: lifecycle.threadId }, - message: { - providerMessageId: lifecycle.messageId, - targetProviderEventId: lifecycle.targetProviderEventId, - providerSentAt: lifecycle.providerSentAt, - }, - filtering: { contentRetained: false }, - }, - nextAttemptAt: null, - processedAt: invalidatedAt, - redactedError: reason, - updatedAt: invalidatedAt, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return true; - }; - if ( - githubBotSourceEdit && - (await retainSourceInvalidation( - null, - "Provider bot edit invalidated an existing source without admitting bot content", - )) - ) - return; - - let lifecyclePrincipalId: string | null = null; - const requiresLifecycleActorAuthorization = - (activeDelivery.eventKind === "message_updated" || - activeDelivery.eventKind === "message_restored") && - (lifecycle.actor !== null || - currentEndpoint.provider === "github" || - currentEndpoint.provider === "telegram" || - currentEndpoint.provider === "microsoft-teams"); - if (requiresLifecycleActorAuthorization) { - const lifecyclePrincipal = lifecycle.actor - ? await tx - .select({ - id: chatExternalPrincipals.id, - isBot: chatExternalPrincipals.isBot, - kind: chatExternalPrincipals.kind, - }) - .from(chatExternalPrincipals) - .where( - and( - eq( - chatExternalPrincipals.companyId, - currentEndpoint.companyId, - ), - eq( - chatExternalPrincipals.provider, - currentEndpoint.provider, - ), - eq( - chatExternalPrincipals.providerAccountId, - currentEndpoint.providerAccountId ?? "unknown", - ), - eq( - chatExternalPrincipals.externalId, - lifecycle.actor.externalId, - ), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const authorization = - lifecyclePrincipal && - lifecyclePrincipal.kind === "user" && - !lifecyclePrincipal.isBot - ? await lockCurrentPrincipalAuthorization( - tx, - currentEndpoint, - lifecyclePrincipal.id, - ) - : null; - const authorizedPrincipalId = authorization?.allowed - ? (lifecyclePrincipal?.id ?? null) - : null; - if (!authorizedPrincipalId) { - if ( - await retainSourceInvalidation( - lifecyclePrincipal?.id ?? null, - "Provider edit invalidation retained without admitting content from an unauthorized actor", - ) - ) - return; - const filteredAt = new Date(); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - principalId: lifecyclePrincipal?.id ?? null, - state: "filtered", - normalizedEvent: { - providerEventId: - activeDelivery.normalizedEvent.providerEventId ?? - activeDelivery.providerEventId, - kind: activeDelivery.eventKind, - conversation: { externalThreadId: lifecycle.threadId }, - message: { - providerMessageId: lifecycle.messageId, - targetProviderEventId: lifecycle.targetProviderEventId, - }, - filtering: { contentRetained: false }, - }, - nextAttemptAt: null, - processedAt: filteredAt, - redactedError: - "External message actor is no longer authorized for lifecycle events", - updatedAt: filteredAt, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - lifecyclePrincipalId = authorizedPrincipalId; - } - - const destinationAllowed = - currentConversation !== null && - ["active", "waiting"].includes(currentConversation.state) && - (currentConversation.isDirectMessage - ? currentEndpoint.allowDirectMessages - : nonDirectDestinationAllowed(currentEndpoint, currentResource)); - if (!destinationAllowed) { - const filteredAt = new Date(); - // A verified deletion or authorized edit still revokes the exactly - // linked source while reach is disabled. Re-enabling access must not - // revive stale files. Keep only the invalidation, never the new - // content: no task comment, attachment import or wake is allowed. - const retainInvalidation = - currentConversation !== null && - (activeDelivery.eventKind === "message_deleted" || - (activeDelivery.eventKind === "message_updated" && - lifecyclePrincipalId !== null)); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - state: retainInvalidation ? "processed" : "filtered", - ...(retainInvalidation - ? { - conversationId: currentConversation.id, - principalId: lifecyclePrincipalId, - } - : {}), - normalizedEvent: { - providerEventId: - activeDelivery.normalizedEvent.providerEventId ?? - activeDelivery.providerEventId, - kind: activeDelivery.eventKind, - ...(retainInvalidation - ? { - runtimeContext: { - generation: admittedRuntimeContext.generation, - credentialFingerprint: - admittedRuntimeContext.credentialFingerprint, - }, - } - : {}), - conversation: { externalThreadId: lifecycle.threadId }, - message: { - providerMessageId: lifecycle.messageId, - targetProviderEventId: lifecycle.targetProviderEventId, - ...(retainInvalidation - ? { providerSentAt: lifecycle.providerSentAt } - : {}), - }, - filtering: { contentRetained: false }, - }, - nextAttemptAt: null, - processedAt: filteredAt, - redactedError: retainInvalidation - ? activeDelivery.eventKind === "message_deleted" - ? "Provider deletion retained without task content while destination access is disabled" - : "Provider edit invalidation retained without task content while destination access is disabled" - : "Destination is no longer enabled for message lifecycle events", - updatedAt: filteredAt, - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - return; - } - - await issuesSvc.addComment( - linkedTarget.issueId, - lifecycle.text, - {}, - { authorType: "system" }, - tx, - ); - await notifyChatDeliveryWork(tx); - await tx - .update(chatDeliveries) - .set({ - conversationId: linkedTarget.conversationId, - principalId: lifecyclePrincipalId, - state: "processed", - processedAt: new Date(), - redactedError: null, - updatedAt: new Date(), - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - }); - } catch (error) { - const terminal = activeDelivery.attempts >= 5; - await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - return tx - .update(chatDeliveries) - .set({ - state: terminal ? "failed" : "retry", - nextAttemptAt: terminal - ? null - : new Date( - Date.now() + - Math.min(60_000, 1000 * 2 ** activeDelivery.attempts), - ), - redactedError: redactError(error), - updatedAt: new Date(), - }) - .where( - and( - eq(chatDeliveries.id, activeDelivery.id), - eq(chatDeliveries.state, "processing"), - ), - ); - }); - throw error; - } - } - - async function handleMessageUpdated( - event: ChatSdkMessageUpdatedCallbackEvent, - runtimeContext: RuntimeContext, - ) { - if ( - event.provider === "discord" && - !(await ensureDiscordGatewayRuntimeIsCurrent( - event.endpointId, - runtimeContext, - )) - ) { - return; - } - const text = `An external message was edited:\n\n${event.message.text.slice(0, MAX_INBOUND_TEXT)}`; - const telegramRich = - event.provider === "telegram" - ? normalizeTelegramRichMessage(event.message.raw) - : null; - await recordLifecycleDelivery( - { - actor: lifecycleActorFromAuthor( - event.provider, - event.message.author, - event.message.raw, - ), - endpointId: event.endpointId, - threadId: event.thread.id, - messageId: event.message.id, - eventKind: "message_updated", - text, - providerMessageSequence: - event.provider === "telegram" - ? telegramMessageSequence(event.message.raw) - : null, - providerSentAt: - ( - event.message.metadata.editedAt ?? event.message.metadata.dateSent - )?.toISOString() ?? null, - raw: event.message.raw, - revision: `${event.message.metadata.editedAt?.toISOString() ?? "unknown"}:${createHash("sha256").update(event.message.text).digest("hex")}${event.provider === "slack" ? `:${slackLifecycleFilesDigest(event.message.raw)}` : event.provider === "discord" ? `:${discordLifecycleFilesDigest(event.message.raw)}` : telegramRich ? `:${telegramRich.mediaDigest}` : ""}`, - }, - runtimeContext, - ); - } - - async function handleMessageDeleted( - event: ChatSdkCallbackEvent, - runtimeContext: RuntimeContext, - ) { - if ( - event.provider === "discord" && - !(await ensureDiscordGatewayRuntimeIsCurrent( - event.endpointId, - runtimeContext, - )) - ) { - return; - } - await recordLifecycleDelivery( - { - endpointId: event.endpointId, - threadId: event.event.threadId, - messageId: event.event.messageId, - eventKind: "message_deleted", - text: "An external message in this conversation was deleted.", - providerSentAt: event.event.deletedAt?.toISOString() ?? null, - raw: event.event.raw, - revision: event.event.deletedAt?.toISOString() ?? null, - }, - runtimeContext, - ); - } - - async function denyExternalAction( - endpoint: EndpointRow, - event: ChatSdkCallbackEvent, - runtimeContext: LifecycleRuntimeFence, - safelyKnown: { - conversationId?: string | null; - principalId?: string | null; - } = {}, - ): Promise { - const telegramCallback = - endpoint.provider === "telegram" && endpoint.botExternalId - ? readTelegramCallbackProvenance(event.telegramCallback, { - companyId: endpoint.companyId, - endpointId: endpoint.id, - botUserId: endpoint.botExternalId, - threadId: event.event.threadId, - messageId: event.event.messageId, - userId: event.event.user.userId, - }) - : null; - const fingerprint = createHash("sha256") - .update( - JSON.stringify( - telegramCallback - ? [ - endpoint.id, - "telegram_callback_denial", - telegramCallback.botUserId, - telegramCallback.callbackId, - ] - : [ - endpoint.id, - event.provider, - event.event.threadId, - event.event.messageId, - event.event.user.userId, - event.event.actionId, - event.event.triggerId ?? null, - ], - ), - ) - .digest("hex"); - const providerEventId = `action-denied:${fingerprint}`; - const processedAt = new Date(); - const actionThread = event.event.thread; - const effectPayload = - actionThread && - typeof actionThread.postEphemeral === "function" && - CAPABILITIES[endpoint.provider].ephemeralMessages - ? ({ - version: 1, - effect: "ephemeral_message", - authorizationMode: "safe_notice", - threadId: event.event.threadId, - userId: event.event.user.userId, - text: "This action is no longer available. Open the linked Paperclip task or ask an operator to link this account.", - fallbackText: "This Paperclip action is no longer available.", - settleDelivery: false, - } as const) - : null; - let effect: typeof chatActions.$inferSelect | null = null; - try { - effect = await db.transaction(async (tx) => { - const currentEndpoint = telegramCallback - ? await runtimeCallbackEndpoint(tx, endpoint.id, runtimeContext, [ - "active", - ]) - : null; - const telegramAuthorizationSha256 = - currentEndpoint && telegramCallback - ? await lockTelegramCallbackNoticeAuthority( - tx, - currentEndpoint, - telegramCallback, - safelyKnown.principalId ?? null, - ) - : null; - await notifyChatDeliveryWork(tx); - const [inserted] = await tx - .insert(chatDeliveries) - .values({ - companyId: endpoint.companyId, - endpointId: endpoint.id, - conversationId: safelyKnown.conversationId ?? null, - principalId: safelyKnown.principalId ?? null, - providerEventId, - deduplicationKey: fingerprint, - eventKind: "action", - normalizedEvent: { - providerEventId, - kind: "action", - authorization: { outcome: "denied" }, - ...(telegramCallback - ? { telegramCallback, telegramAuthorizationSha256 } - : {}), - }, - state: "filtered", - attempts: 1, - redactedError: "External action denied by Paperclip authorization", - processedAt, - updatedAt: processedAt, - }) - .onConflictDoNothing() - .returning(); - const notice: Omit< - ProviderEffectPayload, - "runtimeGeneration" | "credentialFingerprint" - > | null = - telegramCallback && telegramAuthorizationSha256 - ? { - version: 1, - effect: "telegram_callback_notice", - authorizationMode: "safe_notice", - telegramCallback, - telegramAuthorizationSha256, - threadId: telegramCallbackThreadId(telegramCallback), - userId: telegramCallback.receiverUserId, - text: TELEGRAM_PRIVATE_ACTION_UNAVAILABLE, - settleDelivery: false, - } - : effectPayload; - if (!inserted || !notice) return null; - return stageProviderEffect(tx, { - endpoint, - deliveryId: inserted.id, - conversationId: safelyKnown.conversationId ?? null, - principalId: safelyKnown.principalId ?? null, - providerActionId: `provider_effect:${providerEventId}`, - payload: notice, - runtimeContext, - }); - }); - } catch (error) { - logger.warn( - { - endpointId: endpoint.id, - provider: endpoint.provider, - error: redactError(error), - }, - "could not record denied external chat action", - ); - // Provider retries are useful only while the authoritative denial could - // not be recorded. Once the filtered delivery exists, a policy denial - // is a successfully handled webhook and must not be surfaced as a - // callback failure (which the Chat SDK correctly converts to a 503). - throw error; - } - if (effect && actionThread) { - // The callback is acknowledged after the authoritative denial and its - // provider effect are durable. Slow provider I/O runs out of band; a - // redelivery finds the same action and cannot enqueue another notice. - scheduleProviderEffect(effect.id, actionThread); - } - } - - async function recordModalOpenFailure( - endpoint: EndpointRow, - event: ChatSdkCallbackEvent, - runtimeContext: LifecycleRuntimeFence, - safelyKnown: { - attemptActionId?: string | null; - conversationId?: string | null; - principalId?: string | null; - resource?: ResourceRow | null; - }, - error: unknown, - ): Promise { - const disposition = classifyChatPublicationError(error, 1); - const fingerprint = createHash("sha256") - .update( - JSON.stringify([ - endpoint.id, - event.provider, - "modal_open", - event.event.triggerId ?? null, - event.event.threadId, - event.event.messageId, - event.event.user.userId, - event.event.actionId, - ]), - ) - .digest("hex"); - const providerEventId = `modal-open-failed:${fingerprint}`; - const failure = redactError(error); - const processedAt = new Date(); - const actionThread = event.event.thread; - const effectPayload = - actionThread && - typeof actionThread.postEphemeral === "function" && - CAPABILITIES[endpoint.provider].ephemeralMessages - ? ({ - version: 1, - effect: "ephemeral_message", - authorizationMode: "safe_notice", - threadId: event.event.threadId, - userId: event.event.user.userId, - text: "Paperclip could not open this form. Try the action again or open the linked Paperclip task.", - fallbackText: "Paperclip could not open this form.", - settleDelivery: false, - } as const) - : null; - let effect: typeof chatActions.$inferSelect | null = null; - let invalidateEndpointRuntime = false; - try { - effect = await db.transaction(async (tx) => { - await notifyChatDeliveryWork(tx); - const [inserted] = await tx - .insert(chatDeliveries) - .values({ - companyId: endpoint.companyId, - endpointId: endpoint.id, - conversationId: safelyKnown.conversationId ?? null, - principalId: safelyKnown.principalId ?? null, - providerEventId, - deduplicationKey: fingerprint, - eventKind: "action", - normalizedEvent: { - providerEventId, - kind: "action", - modal: { - phase: "open", - outcome: "failed", - disposition: disposition.kind, - }, - }, - // Modal callbacks are acknowledged after this audit row commits. - // They must never enter the replayable inbound-delivery queue. - state: "filtered", - attempts: 1, - redactedError: `Provider modal open failed: ${failure}`.slice( - 0, - MAX_ERROR_TEXT, - ), - processedAt, - updatedAt: processedAt, - }) - .onConflictDoNothing() - .returning(); - if (!inserted) return null; - - if (safelyKnown.attemptActionId) { - await tx - .update(chatActions) - .set({ - status: "failed", - result: { - code: `question_form_open_${disposition.kind}`, - retryable: false, - }, - updatedAt: processedAt, - }) - .where( - and( - eq(chatActions.id, safelyKnown.attemptActionId), - eq(chatActions.status, "processing"), - ), - ); - } - - if (disposition.kind === "endpoint_attention") { - const currentEndpoint = await runtimeCallbackEndpoint( - tx, - endpoint.id, - runtimeContext, - ["active"], - ); - if (currentEndpoint) { - await tx - .update(chatEndpoints) - .set({ - status: "attention", - healthMessage: - "Provider credentials or permissions need attention", - lastError: failure, - updatedAt: processedAt, - }) - .where(eq(chatEndpoints.id, endpoint.id)); - await tx - .update(toolConnections) - .set({ - status: "disabled", - enabled: false, - healthStatus: "degraded", - healthMessage: - "Provider credentials or permissions need attention", - lastError: failure, - healthCheckedAt: processedAt, - updatedAt: processedAt, - }) - .where(eq(toolConnections.id, endpoint.connectionId)); - invalidateEndpointRuntime = true; - } - } else if ( - disposition.kind === "resource_unavailable" && - safelyKnown.resource - ) { - await tx - .update(chatEndpointResources) - .set({ availability: "unavailable", updatedAt: processedAt }) - .where( - and( - eq(chatEndpointResources.companyId, endpoint.companyId), - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.id, safelyKnown.resource.id), - ), - ); - } - - if (!effectPayload) return null; - return stageProviderEffect(tx, { - endpoint, - deliveryId: inserted.id, - conversationId: safelyKnown.conversationId ?? null, - principalId: safelyKnown.principalId ?? null, - providerActionId: `provider_effect:${providerEventId}`, - payload: effectPayload, - runtimeContext, - }); - }); - } catch (auditError) { - logger.warn( - { - endpointId: endpoint.id, - provider: endpoint.provider, - error: redactError(auditError), - }, - "could not record external chat modal-open failure", - ); - throw auditError; - } - if (effect && actionThread) scheduleProviderEffect(effect.id, actionThread); - if (invalidateEndpointRuntime) { - await invalidateRuntime(endpoint.id).catch(() => undefined); - } - } - - async function recordModalSubmitDenial( - endpoint: EndpointRow, - event: ChatSdkCallbackEvent, - input: { - code: string; - conversationId?: string | null; - principalId?: string | null; - }, - database: DbOrTransaction = db, - ): Promise { - const fingerprint = createHash("sha256") - .update( - JSON.stringify([ - endpoint.id, - event.provider, - "modal_submit", - event.event.callbackId, - event.event.viewId, - event.event.user.userId, - input.code, - ]), - ) - .digest("hex"); - const providerEventId = `modal-submit-denied:${fingerprint}`; - const processedAt = new Date(); - try { - await database - .insert(chatDeliveries) - .values({ - companyId: endpoint.companyId, - endpointId: endpoint.id, - conversationId: input.conversationId ?? null, - principalId: input.principalId ?? null, - providerEventId, - deduplicationKey: fingerprint, - eventKind: "action", - normalizedEvent: { - providerEventId, - kind: "action", - authorization: { outcome: "denied" }, - modal: { - phase: "submit", - outcome: "denied", - code: input.code, - }, - }, - state: "filtered", - attempts: 1, - redactedError: "External chat modal submission denied by Paperclip", - processedAt, - updatedAt: processedAt, - }) - .onConflictDoNothing(); - } catch (error) { - logger.warn( - { - endpointId: endpoint.id, - provider: endpoint.provider, - error: redactError(error), - }, - "could not record denied external chat modal submission", - ); - throw error; - } - } - - async function settleTerminalConfirmationAction( - actionId: string, - ): Promise { - const action = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.id, actionId), - eq(chatActions.kind, "confirmation_response"), - ), - ) - .then((rows) => rows[0] ?? null); - if (action?.status === "processed") return true; - if (!action || action.status !== "processing" || !action.conversationId) - return false; - const token = action.payload; - if ( - token.version !== 1 || - typeof token.publicationId !== "string" || - typeof token.interactionId !== "string" || - (token.decision !== "accept" && token.decision !== "reject") - ) { - await db - .update(chatActions) - .set({ - status: "failed", - result: { code: "confirmation_action_payload_invalid" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ); - return false; - } - const [conversation, publication, endpoint] = await Promise.all([ - db - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, action.companyId), - eq(chatConversations.endpointId, action.endpointId), - eq(chatConversations.id, action.conversationId), - ), - ) - .then((rows) => rows[0] ?? null), - db - .select() - .from(chatPublications) - .where( - and( - eq(chatPublications.companyId, action.companyId), - eq(chatPublications.endpointId, action.endpointId), - eq(chatPublications.id, token.publicationId as string), - ), - ) - .then((rows) => rows[0] ?? null), - db - .select() - .from(chatEndpoints) - .where( - and( - eq(chatEndpoints.companyId, action.companyId), - eq(chatEndpoints.id, action.endpointId), - ), - ) - .then((rows) => rows[0] ?? null), - ]); - if ( - !conversation || - !publication || - !endpoint || - endpoint.provider !== "telegram" || - publication.conversationId !== conversation.id || - publication.payload.interactionId !== token.interactionId - ) { - await db - .update(chatActions) - .set({ - status: "failed", - result: { code: "confirmation_action_binding_invalid" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ); - return false; - } - const interaction = ( - await issueThreadInteractionService(db).listForIssue(conversation.issueId) - ).find((candidate) => candidate.id === token.interactionId); - if ( - !interaction || - interaction.kind !== "request_confirmation" || - !nativeTelegramConfirmation(interaction) - ) { - await db - .update(chatActions) - .set({ - status: "failed", - result: { code: "confirmation_interaction_invalid" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ); - return false; - } - if ( - interaction.status !== "accepted" && - interaction.status !== "rejected" - ) { - if (interaction.status === "pending") return false; - const [retired] = await db - .update(chatActions) - .set({ - status: "expired", - result: { - code: "confirmation_interaction_resolved_elsewhere", - interactionId: interaction.id, - interactionStatus: interaction.status, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ) - .returning({ id: chatActions.id }); - if (retired) return true; - const winner = await db - .select({ status: chatActions.status }) - .from(chatActions) - .where(eq(chatActions.id, action.id)) - .then((rows) => rows[0] ?? null); - return winner?.status === "expired" || winner?.status === "processed"; - } - const decisionMatches = - (token.decision === "accept" && interaction.status === "accepted") || - (token.decision === "reject" && interaction.status === "rejected"); - if (!decisionMatches) { - await db - .update(chatActions) - .set({ - status: "expired", - result: { code: "interaction_resolved_by_sibling_action" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ); - return false; - } - const issue = await db - .select({ - id: issues.id, - companyId: issues.companyId, - assigneeAgentId: issues.assigneeAgentId, - status: issues.status, - }) - .from(issues) - .where( - and( - eq(issues.companyId, action.companyId), - eq(issues.id, conversation.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!issue || !interaction.resolvedByUserId) { - await db - .update(chatActions) - .set({ - status: "failed", - result: { code: "confirmation_resolution_context_missing" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ); - return false; - } - const resolvedByUserId = interaction.resolvedByUserId; - const shouldWake = - Boolean(issue.assigneeAgentId) && - issue.status !== "done" && - issue.status !== "cancelled" && - (interaction.continuationPolicy === "wake_assignee" || - (interaction.continuationPolicy === "wake_assignee_on_accept" && - interaction.status === "accepted")); - const resolutionLabel = - interaction.status === "accepted" ? "Accepted" : "Rejected"; - const resolvedByThisProviderAction = - action.result?.code === "confirmation_resolution_committed_by_provider"; - return db.transaction(async (tx) => { - const [completed] = await tx - .update(chatActions) - .set({ - status: "processed", - result: { - code: resolvedByThisProviderAction - ? "confirmation_resolution_committed_by_provider" - : "confirmation_resolution_reconciled", - interactionId: interaction.id, - interactionStatus: interaction.status, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "processing"), - ), - ) - .returning({ id: chatActions.id }); - if (!completed) { - // A publication sweep may reconcile the durable action while the - // original provider callback is still in flight. Treat that CAS loss - // as success once the winner has processed the same action so the - // provider does not receive a spurious 5xx and redeliver it. - const winner = await tx - .select({ status: chatActions.status }) - .from(chatActions) - .where(eq(chatActions.id, action.id)) - .then((rows) => rows[0] ?? null); - return winner?.status === "processed"; - } - await tx - .update(chatActions) - .set({ - status: "expired", - result: { code: "interaction_resolved_by_sibling_action" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.companyId, issue.companyId), - eq(chatActions.endpointId, action.endpointId), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.kind, "confirmation_response"), - inArray(chatActions.status, ["issued", "processing"]), - ne(chatActions.id, action.id), - eq( - sql`${chatActions.payload}->>'interactionId'`, - interaction.id, - ), - ), - ); - await notifyChatPublicationWork(tx); - await tx - .insert(chatPublications) - .values({ - companyId: issue.companyId, - endpointId: action.endpointId, - conversationId: conversation.id, - issueId: issue.id, - idempotencyKey: `interaction-resolution:${interaction.id}:${action.endpointId}`, - payload: projectSafeChatPublication({ - classification: "external", - source: "issue_interaction", - text: `${resolutionLabel}: ${interaction.payload.prompt}`, - interaction: { - id: interaction.id, - card: { - kind: "confirmation", - title: interaction.payload.prompt, - body: resolutionLabel, - actions: [], - }, - }, - }), - state: "pending", - }) - .onConflictDoNothing(); - if (shouldWake && issue.assigneeAgentId) { - await tx - .insert(chatActions) - .values({ - companyId: issue.companyId, - endpointId: action.endpointId, - conversationId: conversation.id, - principalId: action.principalId, - kind: "interaction_wakeup", - providerActionId: `interaction_wakeup:${interaction.id}`, - payload: { - version: 1, - interactionId: interaction.id, - interactionKind: interaction.kind, - interactionStatus: interaction.status, - issueId: issue.id, - agentId: issue.assigneeAgentId, - sourceCommentId: interaction.sourceCommentId ?? null, - sourceRunId: interaction.sourceRunId ?? null, - requestedByUserId: resolvedByUserId, - requestedByActorType: "user", - requestedByActorId: resolvedByUserId, - }, - status: "issued", - }) - .onConflictDoNothing(); - } - // A provider callback can lose the pending -> terminal interaction race - // to the Paperclip UI. In that case this transaction only retires the - // now-redundant provider action and must not fabricate a second external - // resolution event attributed to the board winner. - if (resolvedByThisProviderAction) { - await logActivity(tx as unknown as Db, { - companyId: issue.companyId, - actorType: "user", - actorId: resolvedByUserId, - action: - interaction.status === "accepted" - ? "issue.thread_interaction_accepted" - : "issue.thread_interaction_rejected", - entityType: "issue", - entityId: issue.id, - details: { - source: "external_chat", - endpointId: action.endpointId, - provider: endpoint.provider, - conversationId: conversation.id, - publicationId: publication.id, - providerMessageId: publication.providerMessageId, - interactionId: interaction.id, - interactionKind: interaction.kind, - interactionStatus: interaction.status, - resolutionActorKind: "user", - requestedResolverPolicy: interaction.requestedResolverPolicy, - effectiveResolverPolicy: interaction.effectiveResolverPolicy, - }, - }); - } - return true; - }); - } - - async function reconcileTerminalConfirmationActions( - limit = 100, - ): Promise { - const candidates = await db - .select({ id: chatActions.id }) - .from(chatActions) - .where( - and( - eq(chatActions.kind, "confirmation_response"), - eq(chatActions.status, "processing"), - ), - ) - .orderBy(asc(chatActions.updatedAt), asc(chatActions.id)) - .limit(limit); - let settled = 0; - for (const candidate of candidates) { - if (await settleTerminalConfirmationAction(candidate.id)) settled += 1; - } - return settled; - } - - async function processPendingInteractionWakeups( - limit = 100, - ): Promise { - const staleBefore = new Date(Date.now() - 30_000); - const retryBefore = new Date( - Date.now() - CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, - ); - const candidates = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.kind, "interaction_wakeup"), - or( - and( - eq(chatActions.status, "issued"), - or( - isNull(chatActions.result), - lte(chatActions.updatedAt, retryBefore), - ), - ), - and( - eq(chatActions.status, "processing"), - lte(chatActions.updatedAt, staleBefore), - ), - ), - ), - ) - .orderBy(asc(chatActions.updatedAt), asc(chatActions.id)) - .limit(limit); - let processed = 0; - for (const candidate of candidates) { - const [claimed] = await db - .update(chatActions) - .set({ status: "processing", updatedAt: new Date() }) - .where( - and( - eq(chatActions.id, candidate.id), - or( - and( - eq(chatActions.status, "issued"), - or( - isNull(chatActions.result), - lte(chatActions.updatedAt, retryBefore), - ), - ), - and( - eq(chatActions.status, "processing"), - lte(chatActions.updatedAt, staleBefore), - ), - ), - ), - ) - .returning(); - if (!claimed) continue; - const payload = claimed.payload; - const requestedByActorType = - payload.requestedByActorType === "user" || - payload.requestedByActorType === "agent" || - payload.requestedByActorType === "system" - ? payload.requestedByActorType - : typeof payload.requestedByUserId === "string" - ? "user" - : null; - const requestedByActorId = - typeof payload.requestedByActorId === "string" - ? payload.requestedByActorId - : typeof payload.requestedByUserId === "string" - ? payload.requestedByUserId - : null; - if ( - payload.version !== 1 || - typeof payload.interactionId !== "string" || - typeof payload.interactionKind !== "string" || - !["accepted", "rejected", "answered", "cancelled", "failed"].includes( - String(payload.interactionStatus), - ) || - typeof payload.issueId !== "string" || - typeof payload.agentId !== "string" || - !requestedByActorType || - !requestedByActorId - ) { - await db - .update(chatActions) - .set({ - status: "failed", - result: { code: "interaction_wakeup_payload_invalid" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, claimed.id), - eq(chatActions.status, "processing"), - ), - ); - continue; - } - const currentIssue = await db - .select({ - assigneeAgentId: issues.assigneeAgentId, - status: issues.status, - }) - .from(issues) - .where( - and( - eq(issues.companyId, claimed.companyId), - eq(issues.id, payload.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - const currentInteraction = currentIssue - ? ( - await issueThreadInteractionService(db).listForIssue( - payload.issueId, - ) - ).find((interaction) => interaction.id === payload.interactionId) - : null; - if ( - !currentIssue || - !currentInteraction || - currentInteraction.companyId !== claimed.companyId || - currentInteraction.kind !== payload.interactionKind || - currentInteraction.status !== payload.interactionStatus || - currentIssue.assigneeAgentId !== payload.agentId || - currentIssue.status === "done" || - currentIssue.status === "cancelled" - ) { - await db - .update(chatActions) - .set({ - status: "processed", - result: { code: "interaction_wakeup_no_longer_applicable" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, claimed.id), - eq(chatActions.status, "processing"), - ), - ); - processed += 1; - continue; - } - const idempotencyKey = `interaction:${payload.interactionId}:${payload.interactionStatus}`; - try { - const sourceCommentId = currentInteraction.sourceCommentId ?? null; - const sourceRunId = currentInteraction.sourceRunId ?? null; - // A resolved interaction is still part of the originating external - // turn. Preserve one verified inbound comment edge on the continuation - // run so queued/working/failure milestones return to this exact - // endpoint and conversation instead of disappearing as an internal - // automation run. - const chatOrigin = claimed.conversationId - ? await db - .select({ - commentId: chatMessageLinks.commentId, - provider: chatEndpoints.provider, - }) - .from(chatEndpoints) - .innerJoin( - chatMessageLinks, - and( - eq(chatMessageLinks.companyId, chatEndpoints.companyId), - eq(chatMessageLinks.endpointId, chatEndpoints.id), - ), - ) - .where( - and( - eq(chatEndpoints.companyId, claimed.companyId), - eq(chatEndpoints.id, claimed.endpointId), - eq(chatMessageLinks.conversationId, claimed.conversationId), - eq(chatMessageLinks.direction, "inbound"), - isNotNull(chatMessageLinks.commentId), - ), - ) - .orderBy( - desc(chatMessageLinks.createdAt), - desc(chatMessageLinks.id), - ) - .limit(1) - .then((rows) => rows[0] ?? null) - : null; - const wakeCommentId = sourceCommentId ?? chatOrigin?.commentId ?? null; - const existing = await db - .select({ - agentId: agentWakeupRequests.agentId, - id: agentWakeupRequests.id, - }) - .from(agentWakeupRequests) - .where( - and( - eq(agentWakeupRequests.companyId, claimed.companyId), - eq(agentWakeupRequests.idempotencyKey, idempotencyKey), - notInArray(agentWakeupRequests.status, [ - "skipped", - "failed", - "cancelled", - ]), - ), - ) - .limit(1) - .then((rows) => rows[0] ?? null); - const queuedRun = !existing - ? await options.heartbeat.wakeup(payload.agentId, { - source: "automation", - triggerDetail: "system", - reason: "issue_commented", - payload: { - issueId: payload.issueId, - interactionId: payload.interactionId, - interactionKind: payload.interactionKind, - interactionStatus: payload.interactionStatus, - sourceCommentId, - sourceRunId, - ...(wakeCommentId - ? { - wakeCommentId, - wakeCommentIds: [wakeCommentId], - } - : {}), - ...(payload.planReviewInteraction && - typeof payload.planReviewInteraction === "object" - ? { planReviewInteraction: payload.planReviewInteraction } - : {}), - mutation: "interaction", - }, - idempotencyKey, - allowRunCoalescing: false, - requestedByActorType, - requestedByActorId, - contextSnapshot: { - issueId: payload.issueId, - taskId: payload.issueId, - interactionId: payload.interactionId, - interactionKind: payload.interactionKind, - interactionStatus: payload.interactionStatus, - sourceCommentId, - sourceRunId, - ...(payload.planReviewInteraction && - typeof payload.planReviewInteraction === "object" - ? { planReviewInteraction: payload.planReviewInteraction } - : {}), - wakeReason: "issue_commented", - source: chatOrigin - ? `chat:${chatOrigin.provider}` - : "external_chat.interaction.resolve", - ...(wakeCommentId - ? { - wakeCommentId, - wakeCommentIds: [wakeCommentId], - } - : {}), - ...(payload.forceFreshSession === true - ? { forceFreshSession: true } - : {}), - ...(typeof payload.workspaceRefreshReason === "string" - ? { - workspaceRefreshReason: payload.workspaceRefreshReason, - } - : {}), - }, - }) - : null; - const durable = - existing ?? - (await db - .select({ - agentId: agentWakeupRequests.agentId, - id: agentWakeupRequests.id, - }) - .from(agentWakeupRequests) - .where( - and( - eq(agentWakeupRequests.companyId, claimed.companyId), - eq(agentWakeupRequests.idempotencyKey, idempotencyKey), - notInArray(agentWakeupRequests.status, [ - "skipped", - "failed", - "cancelled", - ]), - ), - ) - .limit(1) - .then((rows) => rows[0] ?? null)); - if (!queuedRun && !durable) { - await db - .update(chatActions) - .set({ - status: "issued", - result: { code: "interaction_wakeup_deferred" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, claimed.id), - eq(chatActions.status, "processing"), - ), - ); - continue; - } - await db - .update(chatActions) - .set({ - status: "processed", - result: { - code: durable - ? durable.agentId === payload.agentId - ? "interaction_wakeup_already_durable" - : "interaction_wakeup_coalesced_after_reassignment" - : "interaction_wakeup_queued", - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, claimed.id), - eq(chatActions.status, "processing"), - ), - ); - processed += 1; - } catch (error) { - const durable = isUniqueViolation(error) - ? await db - .select({ - agentId: agentWakeupRequests.agentId, - id: agentWakeupRequests.id, - }) - .from(agentWakeupRequests) - .where( - and( - eq(agentWakeupRequests.companyId, claimed.companyId), - eq(agentWakeupRequests.idempotencyKey, idempotencyKey), - notInArray(agentWakeupRequests.status, [ - "skipped", - "failed", - "cancelled", - ]), - ), - ) - .limit(1) - .then((rows) => rows[0] ?? null) - : null; - const previousResult = - claimed.result && typeof claimed.result === "object" - ? (claimed.result as Record) - : {}; - const previousAttempts = - typeof previousResult.attemptCount === "number" && - Number.isInteger(previousResult.attemptCount) && - previousResult.attemptCount >= 0 - ? previousResult.attemptCount - : 0; - const attemptCount = previousAttempts + 1; - await db - .update(chatActions) - .set({ - status: durable ? "processed" : "issued", - result: { - code: durable - ? durable.agentId === payload.agentId - ? "interaction_wakeup_already_durable" - : "interaction_wakeup_coalesced_after_reassignment" - : "interaction_wakeup_failed", - ...(!durable ? { attemptCount } : {}), - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, claimed.id), - eq(chatActions.status, "processing"), - ), - ); - if (!durable) { - logger.warn( - { - endpointId: claimed.endpointId, - interactionId: payload.interactionId, - attemptCount, - error: redactError(error), - }, - "failed to queue external interaction continuation wake", - ); - } - } - } - return processed; - } - - async function handleAction( - event: ChatSdkCallbackEvent, - runtimeContext: RuntimeContext, - ) { - const record = await runtimeCallbackRecord( - event.endpointId, - runtimeContext, - ["active"], - ); - if (!record) { - throw forbidden("This chat action is not a current Paperclip question"); - } - const deny = async (safelyKnown?: { - conversationId?: string | null; - principalId?: string | null; - }) => { - await denyExternalAction( - record.endpoint, - event, - runtimeContext, - safelyKnown, - ); - if ( - event.provider === "discord" && - event.transport === "discord_gateway" - ) { - // Discord renders deferUpdate as a successful click. After the denial - // audit is durable, surface a transport-only sentinel so the Gateway - // adapter can withhold that misleading acknowledgement. The runtime, - // not raw JSON or discord.js methods, supplies the ingress context. - throw Object.assign( - new Error("Discord Gateway action was not admitted by Paperclip"), - { code: "chat_discord_gateway_action_rejected" }, - ); - } - }; - if ( - record.endpoint.status !== "active" || - record.endpoint.provider !== event.provider || - record.endpoint.capabilities.actions !== true - ) { - return deny(); - } - const principal = await ensurePrincipal( - record.endpoint, - event.event.user, - event.event.raw, - ); - // Executable external actions are deliberately stricter than ordinary - // sponsored-guest messages. They require a current endpoint-scoped link to - // an active non-viewer Paperclip member, and never run as a guest sponsor. - if ( - principal.linkedDenied || - !principal.userId || - principal.principal.kind !== "user" || - principal.principal.isBot - ) { - return deny({ principalId: principal.principal.id }); - } - const isCorrection = isDiscordQuestionFormCorrectionId( - event.event.actionId, - ); - const correction = - isCorrection && - event.provider === "discord" && - event.transport === "discord_gateway" - ? await loadDiscordQuestionFormCorrection( - persistence, - { - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - }, - event.event.actionId, - { - principalId: principal.principal.id, - userId: principal.userId, - externalUserId: event.event.user.userId, - }, - event.event.threadId, - ) - : null; - if ( - isCorrection && - (!correction || - discordQuestionFormThreadId(event.event.raw) !== event.event.threadId) - ) - return deny({ principalId: principal.principal.id }); - // The private correction handle selects the original publication, never - // the new ephemeral error message. It does not bypass any current source, - // destination, actor, open-token, or pending-interaction check below. - const sourceActionId = correction?.openActionId ?? event.event.actionId; - const sourceMessageId = - correction?.providerMessageId ?? event.event.messageId; - const actionBinding = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, record.endpoint.companyId), - eq(chatActions.endpointId, record.endpoint.id), - inArray(chatActions.kind, [ - "question_answer", - "question_form_open", - "confirmation_response", - ]), - eq(chatActions.providerActionId, sourceActionId), - ), - ) - .then((rows) => (rows.length === 1 ? rows[0]! : null)); - const originalPublicationId = actionBinding?.payload.publicationId; - const actionInteractionId = actionBinding?.payload.interactionId; - if ( - correction && - (actionBinding?.kind !== "question_form_open" || - actionBinding.conversationId !== correction.conversationId || - originalPublicationId !== correction.publicationId || - actionInteractionId !== correction.interactionId || - actionBinding.payload.formActionId !== correction.submitActionId) - ) - return deny({ principalId: principal.principal.id }); - if ( - !actionBinding?.conversationId || - typeof originalPublicationId !== "string" || - typeof actionInteractionId !== "string" - ) { - return deny({ principalId: principal.principal.id }); - } - // The opaque provider action is the immutable link to the original - // publication. Resolve the conversation from that token, then verify the - // provider's thread independently; never trust a callback message id to - // select a Paperclip task. - const conversation = await db - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, record.endpoint.companyId), - eq(chatConversations.endpointId, record.endpoint.id), - eq(chatConversations.id, actionBinding.conversationId), - inArray(chatConversations.state, ["active", "waiting"]), - ), - ) - .then((rows) => rows[0] ?? null); - if ( - !conversation || - !actionThreadMatchesConversation( - event.provider, - event.event.threadId, - conversation.externalThreadId, - ) - ) { - return deny({ principalId: principal.principal.id }); - } - const safelyKnown = { - conversationId: conversation.id, - principalId: principal.principal.id, - }; - const resource = conversation.resourceId - ? await db - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.companyId, record.endpoint.companyId), - eq(chatEndpointResources.endpointId, record.endpoint.id), - eq(chatEndpointResources.id, conversation.resourceId), - ), - ) - .then((rows) => rows[0] ?? null) - : null; - const resourceAllowed = conversation.isDirectMessage - ? record.endpoint.allowDirectMessages - : nonDirectDestinationAllowed(record.endpoint, resource); - if (!resourceAllowed) return deny(safelyKnown); - - let originalPublication = await db - .select() - .from(chatPublications) - .where( - and( - eq(chatPublications.companyId, record.endpoint.companyId), - eq(chatPublications.endpointId, record.endpoint.id), - eq(chatPublications.conversationId, conversation.id), - eq(chatPublications.issueId, conversation.issueId), - eq(chatPublications.id, originalPublicationId), - inArray(chatPublications.state, ["published", "delivery_unknown"]), - or( - isNull(chatPublications.providerMessageId), - eq(chatPublications.providerMessageId, sourceMessageId), - ), - ), - ) - .then((rows) => rows[0] ?? null); - if (!originalPublication) return deny(safelyKnown); - if ( - correction && - (originalPublication.state !== "published" || - originalPublication.providerMessageId !== correction.providerMessageId) - ) - return deny(safelyKnown); - if (!originalPublication.providerMessageId) { - const reconciledPublication = await db.transaction(async (tx) => { - const current = await tx - .select() - .from(chatPublications) - .where(eq(chatPublications.id, originalPublicationId)) - .for("update") - .then((rows) => rows[0] ?? null); - if ( - !current || - current.companyId !== record.endpoint.companyId || - current.endpointId !== record.endpoint.id || - current.conversationId !== conversation.id || - current.issueId !== conversation.issueId || - !["published", "delivery_unknown"].includes(current.state) || - (current.providerMessageId !== null && - current.providerMessageId !== event.event.messageId) - ) { - return null; - } - const occupied = await tx - .select({ publicationId: chatMessageLinks.publicationId }) - .from(chatMessageLinks) - .where( - and( - eq(chatMessageLinks.endpointId, record.endpoint.id), - eq(chatMessageLinks.conversationId, conversation.id), - eq(chatMessageLinks.providerMessageId, event.event.messageId), - eq(chatMessageLinks.direction, "outbound"), - ), - ) - .then((rows) => rows[0] ?? null); - if (occupied && occupied.publicationId !== current.id) return null; - const reconciledAt = new Date(); - await notifyChatPublicationWork(tx); - const [reconciled] = await tx - .update(chatPublications) - .set({ - state: "published", - providerMessageId: event.event.messageId, - publishedAt: current.publishedAt ?? reconciledAt, - nextAttemptAt: null, - redactedError: null, - updatedAt: reconciledAt, - }) - .where( - and( - eq(chatPublications.id, current.id), - inArray(chatPublications.state, [ - "published", - "delivery_unknown", - ]), - isNull(chatPublications.providerMessageId), - ), - ) - .returning(); - if (!reconciled) return null; - await tx - .insert(chatMessageLinks) - .values({ - companyId: current.companyId, - endpointId: current.endpointId, - conversationId: current.conversationId, - publicationId: current.id, - commentId: current.commentId, - providerMessageId: event.event.messageId, - direction: "outbound", - }) - .onConflictDoNothing(); - const linked = await tx - .select({ publicationId: chatMessageLinks.publicationId }) - .from(chatMessageLinks) - .where( - and( - eq(chatMessageLinks.endpointId, record.endpoint.id), - eq(chatMessageLinks.conversationId, conversation.id), - eq(chatMessageLinks.providerMessageId, event.event.messageId), - eq(chatMessageLinks.direction, "outbound"), - ), - ) - .then((rows) => rows[0] ?? null); - return linked?.publicationId === reconciled.id ? reconciled : null; - }); - if (!reconciledPublication) return deny(safelyKnown); - originalPublication = reconciledPublication; - } - const messageBinding = await db - .select({ - link: chatMessageLinks, - linkedPublication: chatPublications, - }) - .from(chatMessageLinks) - .innerJoin( - chatPublications, - and( - eq(chatPublications.companyId, chatMessageLinks.companyId), - eq(chatPublications.endpointId, chatMessageLinks.endpointId), - eq(chatPublications.conversationId, chatMessageLinks.conversationId), - eq(chatPublications.id, chatMessageLinks.publicationId), - ), - ) - .where( - and( - eq(chatMessageLinks.companyId, record.endpoint.companyId), - eq(chatMessageLinks.endpointId, record.endpoint.id), - eq(chatMessageLinks.conversationId, conversation.id), - eq(chatMessageLinks.providerMessageId, sourceMessageId), - eq(chatMessageLinks.direction, "outbound"), - eq(chatPublications.issueId, conversation.issueId), - eq(chatPublications.state, "published"), - eq(chatPublications.providerMessageId, sourceMessageId), - ), - ) - .then((rows) => (rows.length === 1 ? rows[0]! : null)); - if (!messageBinding) return deny(safelyKnown); - const linkedPayload = messageBinding.linkedPublication - .payload as SafeChatPublicationPayload; - const linkStillAuthoritative = - messageBinding.link.publicationId === originalPublicationId || - (messageBinding.linkedPublication.idempotencyKey === - `interaction-resolution:${actionInteractionId}:${record.endpoint.id}` && - linkedPayload.interactionId === actionInteractionId); - if (!originalPublication || !linkStillAuthoritative) { - return deny(safelyKnown); - } - const issued = { - publication: originalPublication, - link: messageBinding.link, - conversation, - }; - - const payload = issued.publication.payload as SafeChatPublicationPayload; - if (event.provider === "telegram") { - const raw = event.event.raw; - const callbackData = - raw && typeof raw === "object" && "data" in raw - ? (raw as { data?: unknown }).data - : null; - if ( - typeof callbackData !== "string" || - Buffer.byteLength(callbackData, "utf8") > - TELEGRAM_CALLBACK_DATA_LIMIT_BYTES || - callbackData !== telegramChatSdkCallbackData(event.event.actionId) || - event.event.value !== undefined - ) { - return deny(safelyKnown); - } - } - if ( - !payload.interactionId || - (!correction && - event.provider !== "telegram" && - event.event.value !== payload.interactionId) || - !payload.card?.actions?.some( - (action) => - action.type === "callback" && action.actionId === sourceActionId, - ) - ) { - return deny(safelyKnown); - } - - const interaction = ( - await issueThreadInteractionService(db).listForIssue(conversation.issueId) - ).find((candidate) => candidate.id === payload.interactionId); - if ( - !interaction || - interaction.companyId !== record.endpoint.companyId || - interaction.issueId !== conversation.issueId - ) { - return deny(safelyKnown); - } - if (interaction.status !== "pending") { - if (correction) return deny(safelyKnown); - const completedAction = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, record.endpoint.companyId), - eq(chatActions.endpointId, record.endpoint.id), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.providerActionId, event.event.actionId), - eq(chatActions.principalId, principal.principal.id), - eq(chatActions.status, "processed"), - ), - ) - .then((rows) => rows[0] ?? null); - const completedPayload = completedAction?.payload; - const completedResult = completedAction?.result; - const completedResultMatches = - completedResult?.interactionId === interaction.id && - completedResult?.interactionStatus === interaction.status && - (interaction.kind !== "request_confirmation" || - completedResult.code === - "confirmation_resolution_committed_by_provider"); - if ( - completedAction && - completedResultMatches && - completedPayload?.publicationId === issued.publication.id && - completedPayload?.interactionId === interaction.id && - completedPayload?.messageId === event.event.messageId && - completedPayload?.actionId === event.event.actionId - ) { - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.threadId, - event.event.raw, - ); - return; - } - return deny(safelyKnown); - } - if (interaction.kind === "request_confirmation") { - if ( - event.provider !== "telegram" || - !nativeTelegramConfirmation(interaction) - ) { - return deny(safelyKnown); - } - const action = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, record.endpoint.companyId), - eq(chatActions.endpointId, record.endpoint.id), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.kind, "confirmation_response"), - eq(chatActions.providerActionId, event.event.actionId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!action || action.status !== "issued") return deny(safelyKnown); - const tokenPayload = action.payload; - const tokenExpiresAt = - typeof tokenPayload.expiresAt === "string" - ? Date.parse(tokenPayload.expiresAt) - : Number.NaN; - if (!Number.isFinite(tokenExpiresAt) || tokenExpiresAt <= Date.now()) { - await db - .update(chatActions) - .set({ - status: "expired", - result: { code: "confirmation_action_token_expired" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "issued"), - ), - ); - return deny(safelyKnown); - } - if ( - tokenPayload.version !== 1 || - tokenPayload.publicationId !== issued.publication.id || - tokenPayload.interactionId !== interaction.id || - (tokenPayload.decision !== "accept" && - tokenPayload.decision !== "reject") - ) { - return deny(safelyKnown); - } - - const issue = await db - .select({ - id: issues.id, - companyId: issues.companyId, - projectId: issues.projectId, - goalId: issues.goalId, - status: issues.status, - assigneeAgentId: issues.assigneeAgentId, - }) - .from(issues) - .where( - and( - eq(issues.companyId, record.endpoint.companyId), - eq(issues.id, conversation.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!issue) { - return deny(safelyKnown); - } - - const decision = tokenPayload.decision; - try { - if (decision === "accept") { - await issueThreadInteractionService(db).acceptInteraction( - issue, - interaction.id, - {}, - { userId: principal.userId }, - { - beforeResolveInTransaction: async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - }, - afterResolveInTransaction: async (tx, resolved) => { - const [owned] = await tx - .update(chatActions) - .set({ - principalId: principal.principal.id, - status: "processing", - payload: { - ...tokenPayload, - messageId: event.event.messageId, - actionId: event.event.actionId, - value: null, - }, - result: { - code: "confirmation_resolution_committed_by_provider", - interactionId: resolved.id, - interactionStatus: resolved.status, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "issued"), - ), - ) - .returning({ id: chatActions.id }); - if (!owned) { - throw new Error( - "External confirmation action ownership changed before interaction commit", - ); - } - }, - }, - ); - } else { - await issueThreadInteractionService(db).rejectInteraction( - issue, - interaction.id, - {}, - { userId: principal.userId }, - { - beforeResolveInTransaction: async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - }, - afterResolveInTransaction: async (tx, resolved) => { - const [owned] = await tx - .update(chatActions) - .set({ - principalId: principal.principal.id, - status: "processing", - payload: { - ...tokenPayload, - messageId: event.event.messageId, - actionId: event.event.actionId, - value: null, - }, - result: { - code: "confirmation_resolution_committed_by_provider", - interactionId: resolved.id, - interactionStatus: resolved.status, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "issued"), - ), - ) - .returning({ id: chatActions.id }); - if (!owned) { - throw new Error( - "External confirmation action ownership changed before interaction commit", - ); - } - }, - }, - ); - } - } catch (error) { - if (isExternalActionAuthorizationChange(error)) { - return deny(safelyKnown); - } - const currentStatus = await db - .select({ status: issueThreadInteractions.status }) - .from(issueThreadInteractions) - .where(eq(issueThreadInteractions.id, interaction.id)) - .then((rows) => rows[0]?.status ?? null); - if (currentStatus !== null && currentStatus !== "pending") { - await settleTerminalConfirmationAction(action.id); - const completedAction = await db - .select({ status: chatActions.status, result: chatActions.result }) - .from(chatActions) - .where(eq(chatActions.id, action.id)) - .then((rows) => rows[0] ?? null); - if ( - completedAction?.status === "processed" && - completedAction.result?.code === - "confirmation_resolution_committed_by_provider" - ) { - return; - } - return deny(safelyKnown); - } - throw error; - } - - await options.confirmationResolutionPersistBarrier?.(); - if (!(await settleTerminalConfirmationAction(action.id))) { - throw new Error( - "External confirmation action ownership changed before settlement", - ); - } - - scheduleMessageProcessing(async () => { - await processPendingPublications(); - }); - return; - } - if (interaction.kind !== "ask_user_questions") { - return deny(safelyKnown); - } - if (isChatQuestionFormOpenActionId(sourceActionId)) { - if ( - event.provider !== "slack" && - event.provider !== "microsoft-teams" && - !( - event.provider === "discord" && - event.transport === "discord_gateway" && - record.endpoint.capabilities.modals === true - ) - ) { - return deny(safelyKnown); - } - await options.questionFormOpenAuthorizationBarrier?.(); - let providerAttempted = false; - let attemptActionId: string | null = null; - try { - const attemptProviderActionId = event.event.triggerId - ? `question_form_open_attempt:${createHash("sha256") - .update( - JSON.stringify([ - record.endpoint.id, - event.provider, - event.event.triggerId, - ]), - ) - .digest("hex")}` - : null; - const preflight = await db.transaction(async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - const resolved = await resolveChatQuestionFormOpen(tx, { - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - conversationId: conversation.id, - interaction, - openActionId: sourceActionId, - }); - if (!resolved || resolved.publicationId !== issued.publication.id) { - return { kind: "denied" as const }; - } - if (!attemptProviderActionId) { - return { - kind: "ready" as const, - attemptActionId: null, - resolved, - }; - } - const [attempt] = await tx - .insert(chatActions) - .values({ - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - conversationId: conversation.id, - principalId: principal.principal.id, - kind: "question_form_open_attempt", - providerActionId: attemptProviderActionId, - payload: { version: 1 }, - status: "processing", - }) - .onConflictDoNothing() - .returning({ id: chatActions.id }); - return attempt - ? { - kind: "ready" as const, - attemptActionId: attempt.id, - resolved, - } - : { kind: "duplicate" as const }; - }); - if (preflight.kind === "denied") return deny(safelyKnown); - if (preflight.kind === "duplicate") { - if (event.provider === "discord") { - // A prior attempt may have consumed Discord's single response. - // A fresh handler has no showModal closure state for that attempt; - // do not turn its processing/failed receipt into a success ACK. - throw Object.assign( - new Error("Discord modal response was already attempted"), - { code: "chat_discord_gateway_modal_response_indeterminate" }, - ); - } - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.threadId, - event.event.raw, - ); - return; - } - attemptActionId = preflight.attemptActionId; - - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.threadId, - event.event.raw, - ); - - // A provider modal open can take seconds and can outlive Slack's - // trigger. The final Paperclip authorization snapshot above commits - // before transport starts so provider latency never holds endpoint, - // destination, identity-link, or membership row locks. - const modal = correction - ? discordQuestionFormCorrectionModal( - preflight.resolved.modal, - correction, - ) - : preflight.resolved.modal; - if (!modal) return deny(safelyKnown); - providerAttempted = true; - const opened = await event.event.openModal(modal); - if (!opened) { - await recordModalOpenFailure( - record.endpoint, - event, - runtimeContext, - { ...safelyKnown, attemptActionId, resource }, - new Error("Provider did not confirm the modal open"), - ); - if (event.provider === "discord") return deny(safelyKnown); - return; - } - if (attemptActionId) { - const [settledAttempt] = await db - .update(chatActions) - .set({ - status: "processed", - result: { code: "question_form_opened" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, attemptActionId), - eq(chatActions.status, "processing"), - ), - ) - .returning({ id: chatActions.id }); - if (!settledAttempt) { - throw new Error( - "Question-form modal ownership changed after provider acceptance", - ); - } - } - } catch (error) { - if (isExternalActionAuthorizationChange(error)) { - return deny(safelyKnown); - } - if (!providerAttempted) throw error; - await recordModalOpenFailure( - record.endpoint, - event, - runtimeContext, - { ...safelyKnown, attemptActionId, resource }, - error, - ); - if (event.provider === "discord") return deny(safelyKnown); - // The provider attempt is now a durable, non-replayable Activity row. - // Acknowledge the callback so Chat SDK does not turn a definite denial - // or an ambiguous modal open into a duplicate retry loop. - return; - } - return; - } - - const action = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, record.endpoint.companyId), - eq(chatActions.endpointId, record.endpoint.id), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.kind, "question_answer"), - eq(chatActions.providerActionId, event.event.actionId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!action || action.status !== "issued") return deny(safelyKnown); - const tokenPayload = action.payload; - const tokenExpiresAt = - typeof tokenPayload.expiresAt === "string" - ? Date.parse(tokenPayload.expiresAt) - : Number.NaN; - if (!Number.isFinite(tokenExpiresAt) || tokenExpiresAt <= Date.now()) { - await db - .update(chatActions) - .set({ - status: "expired", - result: { code: "question_action_token_expired" }, - updatedAt: new Date(), - }) - .where( - and(eq(chatActions.id, action.id), eq(chatActions.status, "issued")), - ); - return deny(safelyKnown); - } - if ( - tokenPayload.version !== 1 || - tokenPayload.publicationId !== issued.publication.id || - tokenPayload.interactionId !== payload.interactionId || - typeof tokenPayload.questionId !== "string" || - typeof tokenPayload.optionId !== "string" - ) { - return deny(safelyKnown); - } - - const question = nativeChatQuestion(interaction); - const option = question?.options.find( - (candidate) => candidate.id === tokenPayload.optionId, - ); - if (!question || question.id !== tokenPayload.questionId || !option) - return deny(safelyKnown); - - await options.questionResolutionPersistBarrier?.(); - - const issue = await db - .select({ - id: issues.id, - companyId: issues.companyId, - status: issues.status, - }) - .from(issues) - .where( - and( - eq(issues.companyId, record.endpoint.companyId), - eq(issues.id, conversation.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!issue) return deny(safelyKnown); - try { - const answered = await issueThreadInteractionService(db).answerQuestions( - issue, - interaction.id, - { - answers: [{ questionId: question.id, optionIds: [option.id] }], - }, - { userId: principal.userId }, - { - beforeResolveInTransaction: async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - }, - afterResolveInTransaction: async (tx, resolved) => { - const [completed] = await tx - .update(chatActions) - .set({ - principalId: principal.principal.id, - status: "processed", - payload: { - ...tokenPayload, - messageId: event.event.messageId, - actionId: event.event.actionId, - value: event.event.value ?? null, - }, - result: { - interactionId: resolved.id, - interactionStatus: resolved.status, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - eq(chatActions.status, "issued"), - ), - ) - .returning({ id: chatActions.id }); - if (!completed) { - throw new Error( - "External question action ownership changed before commit", - ); - } - await tx - .update(chatActions) - .set({ - status: "expired", - result: { code: "interaction_resolved_by_sibling_action" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.companyId, issue.companyId), - eq(chatActions.endpointId, record.endpoint.id), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.kind, "question_answer"), - eq(chatActions.status, "issued"), - ne(chatActions.id, action.id), - eq( - sql`${chatActions.payload}->>'interactionId'`, - resolved.id, - ), - ), - ); - await logActivity(tx as unknown as Db, { - companyId: issue.companyId, - actorType: "user", - actorId: principal.userId!, - action: "issue.thread_interaction_answered", - entityType: "issue", - entityId: issue.id, - details: { - source: "external_chat", - endpointId: record.endpoint.id, - provider: record.endpoint.provider, - conversationId: conversation.id, - publicationId: issued.publication.id, - providerMessageId: event.event.messageId, - interactionId: resolved.id, - interactionKind: resolved.kind, - interactionStatus: resolved.status, - resolutionActorKind: "user", - answeredQuestionCount: 1, - }, - }); - await notifyChatPublicationWork(tx); - await tx - .insert(chatPublications) - .values({ - companyId: issue.companyId, - endpointId: record.endpoint.id, - conversationId: conversation.id, - issueId: issue.id, - idempotencyKey: `interaction-resolution:${resolved.id}:${record.endpoint.id}`, - payload: projectSafeChatPublication({ - classification: "external", - source: "issue_interaction", - text: `Answered: ${option.label}.`, - interaction: { - id: resolved.id, - card: { - kind: "question", - title: question.prompt, - body: `Answered: ${option.label}.`, - actions: [], - }, - }, - }), - state: "pending", - }) - .onConflictDoNothing(); - }, - }, - ); - scheduleMessageProcessing(async () => { - await processPendingPublications(); - }); - // The interaction, provider token, and response-delivery receipt are - // already durable. A provider callback must not wait for agent startup; - // drain the outbox after acknowledging the provider callback. - scheduleMessageProcessing(async () => { - await questionResponses.deliver(answered.id); - }); - } catch (error) { - if (isExternalActionAuthorizationChange(error)) { - return deny(safelyKnown); - } - // `answerQuestions` commits the interaction and this action in one - // transaction, then performs a best-effort issue timestamp touch. If - // that post-commit touch fails, the provider callback is nevertheless - // durably complete. Do not corrupt the processed token to `failed` or - // return a retryable webhook response that turns Telegram's redelivery - // into a misleading stale-action notice. - try { - const raceResult = await db.transaction(async (tx) => { - const currentAction = await tx - .select({ status: chatActions.status }) - .from(chatActions) - .where(eq(chatActions.id, action.id)) - .then((rows) => rows[0] ?? null); - const currentInteraction = ( - await issueThreadInteractionService( - tx as unknown as Db, - ).listForIssue(issue.id) - ).find((candidate) => candidate.id === interaction.id); - if (currentInteraction && currentInteraction.status !== "pending") { - await tx - .update(chatActions) - .set({ - status: "expired", - result: { code: "interaction_resolved_elsewhere" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, action.id), - inArray(chatActions.status, ["issued", "processing"]), - ), - ); - await enqueueTerminalIssueInteractionChatPublications( - tx, - currentInteraction, - ); - return { actionStatus: currentAction?.status, settledRace: true }; - } - return { actionStatus: currentAction?.status, settledRace: false }; - }); - if (raceResult.actionStatus === "processed") { - logger.warn( - { - endpointId: record.endpoint.id, - interactionId: interaction.id, - error: redactError(error), - }, - "external chat question resolved but post-commit follow-up failed", - ); - return; - } - if (raceResult.settledRace) { - scheduleMessageProcessing(async () => { - await processPendingPublications(); - }); - return; - } - } catch (recoveryError) { - logger.warn( - { - endpointId: record.endpoint.id, - interactionId: interaction.id, - error: redactError(recoveryError), - }, - "could not reconcile an external chat question after resolution failed", - ); - } - throw error; - } - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.threadId, - event.event.raw, - ); - } - - async function handleModalSubmit( - event: ChatSdkCallbackEvent, - runtimeContext: RuntimeContext, - ): Promise { - const denialError = () => - forbidden("This chat form is not a current Paperclip question"); - const record = await runtimeCallbackRecord( - event.endpointId, - runtimeContext, - ["active"], - ); - if ( - !record || - record.endpoint.status !== "active" || - record.endpoint.provider !== event.provider || - record.endpoint.capabilities.modals !== true || - (event.provider !== "slack" && - event.provider !== "microsoft-teams" && - !( - event.provider === "discord" && event.transport === "discord_gateway" - )) - ) { - const staleRecord = - record ?? - (await endpointRecord(event.endpointId).then((candidate) => - candidate?.endpoint.provider === event.provider ? candidate : null, - )); - if (staleRecord) { - await recordModalSubmitDenial(staleRecord.endpoint, event, { - code: "chat_modal_runtime_not_current", - }); - } - return event.provider === "discord" - ? discordQuestionFormDenialResponse() - : chatQuestionFormDenialResponse(); - } - const deny = async ( - code: string, - safelyKnown: { - conversationId?: string | null; - principalId?: string | null; - } = {}, - payload?: Parameters[0], - ): Promise => { - await recordModalSubmitDenial(record.endpoint, event, { - code, - ...safelyKnown, - }); - return event.provider === "discord" - ? discordQuestionFormDenialResponse() - : chatQuestionFormDenialResponse(payload); - }; - const loaded = await loadChatQuestionFormSubmissionToken(db, { - callbackId: event.event.callbackId, - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - includeProcessed: true, - }); - if (!loaded) return deny("chat_modal_token_not_current"); - const conversation = await db - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, record.endpoint.companyId), - eq(chatConversations.endpointId, record.endpoint.id), - eq(chatConversations.id, loaded.conversationId), - inArray(chatConversations.state, ["active", "waiting"]), - ), - ) - .then((rows) => rows[0] ?? null); - if ( - !conversation || - (event.provider === "discord" && - discordQuestionFormThreadId(event.event.raw) !== - conversation.externalThreadId) || - (event.event.relatedThread && - (event.provider === "microsoft-teams" - ? !sameTeamsThreadIdentity( - event.event.relatedThread.id, - conversation.externalThreadId, - ) - : event.event.relatedThread.id !== conversation.externalThreadId)) - ) { - return deny( - "chat_modal_conversation_not_current", - { - conversationId: conversation?.id ?? null, - }, - loaded.payload, - ); - } - const principal = await ensurePrincipal( - record.endpoint, - event.event.user, - event.event.raw, - ); - if ( - principal.linkedDenied || - !principal.userId || - principal.principal.kind !== "user" || - principal.principal.isBot - ) { - return deny( - "chat_modal_principal_not_authorized", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const resource = conversation.resourceId - ? await db - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.companyId, record.endpoint.companyId), - eq(chatEndpointResources.endpointId, record.endpoint.id), - eq(chatEndpointResources.id, conversation.resourceId), - ), - ) - .then((rows) => rows[0] ?? null) - : null; - const resourceAllowed = conversation.isDirectMessage - ? record.endpoint.allowDirectMessages - : nonDirectDestinationAllowed(record.endpoint, resource); - if (!resourceAllowed) { - return deny( - "chat_modal_destination_not_authorized", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - - const originalPublication = await db - .select() - .from(chatPublications) - .where( - and( - eq(chatPublications.companyId, record.endpoint.companyId), - eq(chatPublications.endpointId, record.endpoint.id), - eq(chatPublications.conversationId, conversation.id), - eq(chatPublications.issueId, conversation.issueId), - eq(chatPublications.id, loaded.publicationId), - eq(chatPublications.state, "published"), - event.provider !== "discord" && event.event.relatedMessage - ? eq( - chatPublications.providerMessageId, - event.event.relatedMessage.id, - ) - : undefined, - ), - ) - .then((rows) => rows[0] ?? null); - const providerMessageId = - (event.provider === "discord" - ? undefined - : event.event.relatedMessage?.id) ?? - originalPublication?.providerMessageId ?? - null; - if (!originalPublication || !providerMessageId) { - return deny( - "chat_modal_publication_not_current", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const currentMessageBinding = await db - .select({ - link: chatMessageLinks, - publication: chatPublications, - }) - .from(chatMessageLinks) - .innerJoin( - chatPublications, - and( - eq(chatPublications.companyId, chatMessageLinks.companyId), - eq(chatPublications.endpointId, chatMessageLinks.endpointId), - eq(chatPublications.conversationId, chatMessageLinks.conversationId), - eq(chatPublications.id, chatMessageLinks.publicationId), - ), - ) - .where( - and( - eq(chatMessageLinks.companyId, record.endpoint.companyId), - eq(chatMessageLinks.endpointId, record.endpoint.id), - eq(chatMessageLinks.conversationId, conversation.id), - eq(chatMessageLinks.providerMessageId, providerMessageId), - eq(chatMessageLinks.direction, "outbound"), - eq(chatPublications.issueId, conversation.issueId), - eq(chatPublications.state, "published"), - eq(chatPublications.providerMessageId, providerMessageId), - ), - ) - .then((rows) => (rows.length === 1 ? rows[0]! : null)); - const linkedPayload = currentMessageBinding?.publication.payload as - | SafeChatPublicationPayload - | undefined; - const linkStillAuthoritative = - currentMessageBinding?.link.publicationId === originalPublication.id || - (currentMessageBinding?.publication.idempotencyKey === - `interaction-resolution:${loaded.interactionId}:${record.endpoint.id}` && - linkedPayload?.interactionId === loaded.interactionId); - if (!currentMessageBinding || !linkStillAuthoritative) { - return deny( - "chat_modal_message_binding_not_current", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const issued = { - publication: originalPublication, - link: currentMessageBinding.link, - }; - const payload = issued.publication.payload as SafeChatPublicationPayload; - if (payload.interactionId !== loaded.interactionId) { - return deny( - "chat_modal_interaction_binding_not_current", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const interaction = ( - await issueThreadInteractionService(db).listForIssue(conversation.issueId) - ).find((candidate) => candidate.id === loaded.interactionId); - const discordReplayAnswersMatch = ( - resolved: typeof interaction, - ): boolean => { - if (event.provider !== "discord") return true; - if ( - resolved?.kind !== "ask_user_questions" || - resolved.status !== "answered" - ) - return false; - // Parse only for comparison after current token/source/actor checks. - // This local pending projection cannot reopen or resolve an interaction. - const submitted = validateChatQuestionFormSubmission({ - callbackId: event.event.callbackId, - privateMetadata: event.event.privateMetadata, - interaction: { ...resolved, status: "pending" }, - payload: loaded.payload, - values: event.event.values, - }); - return ( - submitted.ok && - JSON.stringify(submitted.answers) === - JSON.stringify(resolved.result?.answers) - ); - }; - const isExactProcessedReplay = ( - current: Awaited>, - ) => - interaction?.companyId === record.endpoint.companyId && - interaction.issueId === conversation.issueId && - interaction.kind === "ask_user_questions" && - interaction.status === "answered" && - interaction.resolvedByUserId === principal.userId && - current?.status === "processed" && - current.actionRowId === loaded.actionRowId && - current.conversationId === conversation.id && - current.publicationId === originalPublication.id && - current.interactionId === loaded.interactionId && - current.principalId === principal.principal.id && - current.result?.code === "question_form_answered" && - current.result?.interactionId === loaded.interactionId && - discordReplayAnswersMatch(interaction); - const acceptedResponse = async (): Promise => { - if (event.provider === "discord") { - try { - await deleteDiscordQuestionFormCorrection( - persistence, - { - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - }, - { - principalId: principal.principal.id, - userId: principal.userId!, - externalUserId: event.event.user.userId, - }, - loaded.payload.formActionId, - ); - } catch { - // The answer receipt is already committed. Cleanup is retryable on - // exact replay/expiry access and cannot relabel an accepted answer. - logger.warn( - { - endpointId: record.endpoint.id, - code: "discord_question_correction_cleanup_failed", - }, - "Discord correction draft cleanup will be retried", - ); - } - } - return { action: "clear" }; - }; - if (loaded.status === "processed") { - if (!isExactProcessedReplay(loaded)) { - return deny( - "chat_modal_replay_not_authorized", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - if (event.event.relatedThread) { - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.relatedThread.id, - event.event.raw, - ); - } - return acceptedResponse(); - } - if ( - !interaction || - interaction.companyId !== record.endpoint.companyId || - interaction.issueId !== conversation.issueId || - interaction.kind !== "ask_user_questions" || - interaction.status !== "pending" - ) { - // Another callback can commit the same token between the initial token - // read and this interaction read. Only its exact processed receipt may - // acknowledge the stale snapshot; an answer by another token or actor - // does not authorize this form. Recheck current policy under its locks. - let processedReplay = false; - if ( - interaction?.companyId === record.endpoint.companyId && - interaction.issueId === conversation.issueId && - interaction.kind === "ask_user_questions" && - interaction.status === "answered" - ) { - try { - processedReplay = await db.transaction(async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - return isExactProcessedReplay( - await loadChatQuestionFormSubmissionToken(tx, { - callbackId: event.event.callbackId, - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - includeProcessed: true, - }), - ); - }); - } catch (error) { - if (!isExternalActionAuthorizationChange(error)) throw error; - } - } - if (processedReplay) { - if (event.event.relatedThread) { - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.relatedThread.id, - event.event.raw, - ); - } - return acceptedResponse(); - } - return deny( - "chat_modal_interaction_not_pending", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const validation = validateChatQuestionFormSubmission({ - callbackId: event.event.callbackId, - privateMetadata: event.event.privateMetadata, - interaction, - payload: loaded.payload, - values: event.event.values, - }); - if (!validation.ok) { - if (validation.code === "invalid_form") { - if (event.provider === "discord") { - const openAction = payload.card?.actions?.find( - (action) => - action.type === "callback" && - isChatQuestionFormOpenActionId(action.actionId), - ); - const openActionId = - openAction?.type === "callback" ? openAction.actionId : null; - if (!openActionId) return deny("chat_modal_open_token_not_current"); - try { - return await db.transaction(async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: record.endpoint.id, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - const pending = await tx - .select({ id: issueThreadInteractions.id }) - .from(issueThreadInteractions) - .where( - and( - eq( - issueThreadInteractions.companyId, - record.endpoint.companyId, - ), - eq(issueThreadInteractions.issueId, conversation.issueId), - eq(issueThreadInteractions.id, interaction.id), - eq(issueThreadInteractions.status, "pending"), - ), - ) - .for("update") - .then((rows) => rows[0]); - const resolved = pending - ? await resolveChatQuestionFormOpen(tx, { - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - conversationId: conversation.id, - interaction, - openActionId, - }) - : null; - if ( - !resolved || - resolved.submitActionId !== loaded.payload.formActionId || - resolved.publicationId !== originalPublication.id - ) - throw denialError(); - return retainDiscordQuestionFormCorrection( - createChatSdkStatePersistence(tx as unknown as Db), - { - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - }, - { - principalId: principal.principal.id, - userId: principal.userId!, - externalUserId: event.event.user.userId, - conversationId: conversation.id, - publicationId: originalPublication.id, - providerMessageId, - threadId: conversation.externalThreadId, - interactionId: interaction.id, - openActionId, - submitActionId: resolved.submitActionId, - parentExpiresAt: loaded.payload.expiresAt, - modal: resolved.modal, - values: event.event.values, - fieldErrors: validation.fieldErrors, - }, - ); - }); - } catch (error) { - if ( - isExternalActionAuthorizationChange(error) || - (error && - typeof error === "object" && - "status" in error && - error.status === 403) - ) - return deny("chat_modal_correction_not_authorized"); - throw error; - } - } - return chatQuestionFormValidationResponse({ - provider: event.provider, - callbackId: event.event.callbackId, - privateMetadata: event.event.privateMetadata, - interaction, - payload: loaded.payload, - values: event.event.values, - }); - } - return deny( - `chat_modal_${validation.code}`, - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - const issue = await db - .select({ - id: issues.id, - companyId: issues.companyId, - status: issues.status, - }) - .from(issues) - .where( - and( - eq(issues.companyId, record.endpoint.companyId), - eq(issues.id, conversation.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!issue) { - return deny( - "chat_modal_issue_not_current", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - await options.questionResolutionPersistBarrier?.(); - let answered: Awaited< - ReturnType< - ReturnType["answerQuestions"] - > - >; - try { - answered = await issueThreadInteractionService(db).answerQuestions( - issue, - interaction.id, - { answers: validation.answers }, - { userId: principal.userId }, - { - beforeResolveInTransaction: async (tx) => { - await requireCurrentExternalActionAuthorization(tx, { - conversationId: conversation.id, - endpointId: event.endpointId, - expectedUserId: principal.userId!, - principalId: principal.principal.id, - runtimeContext, - }); - }, - afterResolveInTransaction: async (tx, resolved) => { - const claimed = await claimChatQuestionFormSubmission(tx, { - actionRowId: loaded.actionRowId, - principalId: principal.principal.id, - }); - if (!claimed) throw denialError(); - const completed = await completeChatQuestionFormSubmission(tx, { - actionRowId: loaded.actionRowId, - interactionId: resolved.id, - }); - if (!completed) throw denialError(); - await tx - .update(chatActions) - .set({ - status: "expired", - result: { code: "question_form_submitted" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.companyId, issue.companyId), - eq(chatActions.endpointId, record.endpoint.id), - eq(chatActions.conversationId, conversation.id), - eq(chatActions.kind, "question_form_open"), - eq(chatActions.status, "issued"), - eq( - sql`${chatActions.payload}->>'interactionId'`, - resolved.id, - ), - ), - ); - await logActivity(tx as unknown as Db, { - companyId: issue.companyId, - actorType: "user", - actorId: principal.userId!, - action: "issue.thread_interaction_answered", - entityType: "issue", - entityId: issue.id, - details: { - source: "external_chat_modal", - endpointId: record.endpoint.id, - provider: record.endpoint.provider, - conversationId: conversation.id, - publicationId: issued.publication.id, - interactionId: resolved.id, - interactionKind: resolved.kind, - interactionStatus: resolved.status, - resolutionActorKind: "user", - answeredQuestionCount: validation.answers.length, - }, - }); - }, - }, - ); - } catch (error) { - if (isExternalActionAuthorizationChange(error)) { - await db.transaction(async (tx) => { - await tx - .update(chatActions) - .set({ - status: "expired", - result: { code: "chat_action_authorization_changed" }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, loaded.actionRowId), - eq(chatActions.status, "issued"), - ), - ); - await recordModalSubmitDenial( - record.endpoint, - event, - { - code: "chat_action_authorization_changed", - conversationId: conversation.id, - principalId: principal.principal.id, - }, - tx, - ); - }); - return event.provider === "discord" - ? discordQuestionFormDenialResponse() - : chatQuestionFormDenialResponse(loaded.payload); - } - const settledRace = await db.transaction(async (tx) => { - const currentInteraction = ( - await issueThreadInteractionService(tx as unknown as Db).listForIssue( - issue.id, - ) - ).find((candidate) => candidate.id === interaction.id); - if (!currentInteraction || currentInteraction.status === "pending") { - return false; - } - await enqueueTerminalIssueInteractionChatPublications( - tx, - currentInteraction, - ); - return true; - }); - if (!settledRace) { - const status = - error && typeof error === "object" - ? (error as { status?: unknown }).status - : null; - if (status === 403) { - return deny( - "chat_modal_resolution_not_authorized", - { - conversationId: conversation.id, - principalId: principal.principal.id, - }, - loaded.payload, - ); - } - throw error; - } - scheduleMessageProcessing(async () => { - await processPendingPublications(); - }); - if (event.provider === "discord") { - const current = await loadChatQuestionFormSubmissionToken(db, { - callbackId: event.event.callbackId, - companyId: record.endpoint.companyId, - endpointId: record.endpoint.id, - includeProcessed: true, - }); - const resolved = ( - await issueThreadInteractionService(db).listForIssue( - conversation.issueId, - ) - ).find((candidate) => candidate.id === interaction.id); - if ( - current?.status !== "processed" || - current.actionRowId !== loaded.actionRowId || - current.principalId !== principal.principal.id || - current.result?.code !== "question_form_answered" || - current.result.interactionId !== interaction.id || - resolved?.status !== "answered" || - resolved.resolvedByUserId !== principal.userId || - !discordReplayAnswersMatch(resolved) - ) - return deny("chat_modal_resolution_committed_elsewhere"); - } - return acceptedResponse(); - } - if (event.event.relatedThread) { - await recordCurrentMicrosoftTeamsRoute( - record.endpoint, - runtimeContext, - event.event.relatedThread.id, - event.event.raw, - ); - } - scheduleMessageProcessing(async () => { - await questionResponses.deliver(answered.id); - }); - return acceptedResponse(); - } async function handleSlashCommand( event: ChatSdkCallbackEvent, @@ -13100,43 +4709,6 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { return true; } - function lifecycleRuntimeFence( - delivery: DeliveryRow, - ): LifecycleRuntimeFence | null { - const normalized = delivery.normalizedEvent; - if ( - !normalized || - typeof normalized !== "object" || - Array.isArray(normalized) - ) - return null; - const value = (normalized as Record).runtimeContext; - if (!value || typeof value !== "object" || Array.isArray(value)) - return null; - const context = value as Record; - return typeof context.generation === "number" && - Number.isSafeInteger(context.generation) && - typeof context.credentialFingerprint === "string" - ? { - generation: context.generation as number, - credentialFingerprint: context.credentialFingerprint, - } - : null; - } - - function normalizedLifecycleWithRuntimeFence( - delivery: DeliveryRow, - runtimeContext: LifecycleRuntimeFence, - ): Record { - const normalized = - delivery.normalizedEvent && - typeof delivery.normalizedEvent === "object" && - !Array.isArray(delivery.normalizedEvent) - ? (delivery.normalizedEvent as Record) - : {}; - return { ...normalized, runtimeContext }; - } - async function applyProviderLifecycleEffects( endpoint: EndpointRow, effects: ChatProviderLifecycleEffect[], @@ -15964,2887 +7536,18 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { publication ); } - - async function recordExactOutboundPublicationLink( - tx: DbOrTransaction, - input: { - publication: typeof chatPublications.$inferSelect; - providerMessageId: string; - }, - ): Promise { - const inserted = await tx - .insert(chatMessageLinks) - .values({ - companyId: input.publication.companyId, - endpointId: input.publication.endpointId, - conversationId: input.publication.conversationId, - publicationId: input.publication.id, - commentId: input.publication.commentId, - providerMessageId: input.providerMessageId, - direction: "outbound", - }) - .onConflictDoNothing() - .returning({ id: chatMessageLinks.id }); - if (inserted.length) return true; - return tx - .select({ id: chatMessageLinks.id }) - .from(chatMessageLinks) - .where( - and( - eq(chatMessageLinks.companyId, input.publication.companyId), - eq(chatMessageLinks.endpointId, input.publication.endpointId), - eq(chatMessageLinks.conversationId, input.publication.conversationId), - eq(chatMessageLinks.publicationId, input.publication.id), - input.publication.commentId - ? eq(chatMessageLinks.commentId, input.publication.commentId) - : isNull(chatMessageLinks.commentId), - eq(chatMessageLinks.providerMessageId, input.providerMessageId), - eq(chatMessageLinks.direction, "outbound"), - ), - ) - .limit(1) - .then((rows) => rows.length > 0); + async function recordExactOutboundPublicationLink(...args: Parameters["recordExactOutboundPublicationLink"]>) { + return publicationTransport.recordExactOutboundPublicationLink(...args); } - async function postSafePublication(input: { - endpoint: EndpointRow; - conversation: ConversationRow; - publication: typeof chatPublications.$inferSelect; - payload: SafeChatPublicationPayload; - replaceProviderMessageId?: string | null; - telegramDraftControl?: TelegramDraftControl; - beforePhotonWrite?(): Promise; - onSlackFileUploadAccepted?: ( - receipt: SlackFileUploadAcceptedReceipt, - ) => Promise; - }) { - if (input.endpoint.provider === "speko") { - const receipt = await voiceStore.enqueuePublication(input.endpoint.companyId, input.publication.id); - scheduleMessageProcessing(() => voice.pushReplies(25, input.endpoint.companyId, input.endpoint.id).then(() => undefined)); - return receipt; - } - if (input.endpoint.provider === "github" && ["failed", "completed"].includes(input.payload.progressState ?? "") && - !input.payload.interactionId && !isExplicitOperatorPublication(input.publication) && - (await db.select({ endpointId: chatGitHubConfigurations.endpointId }).from(chatGitHubConfigurations).where(and( - eq(chatGitHubConfigurations.companyId, input.endpoint.companyId), eq(chatGitHubConfigurations.endpointId, input.endpoint.id))).limit(1)).length > 0) { - const runId = runIdFromMilestonePublication(input.publication); - if (!runId) throw forbidden("GitHub failure has no run binding"); - const receipt = await githubResponseCommentService(db, fetchImpl).failRun({ - companyId: input.publication.companyId, endpointId: input.endpoint.id, - issueId: input.publication.issueId, runId, body: input.payload.progressState === "completed" - ? "This turn ended without publishing a final response. See the Paperclip task for details." - : renderPublicationTransportText(input.payload), - }); - if (!receipt) throw conflict("GitHub response publication is busy"); - return { id: receipt.id, threadId: input.conversation.externalThreadId, raw: {}, text: input.payload.text }; - } - const endpointRuntime = await runtimeFor(input.endpoint); - const thread = endpointRuntime.thread(input.conversation.externalThreadId); - const card = safeCardForPublication(input.payload, input.endpoint.provider); - let attachments: Attachment[] = []; - let files: FileUpload[] = []; - let text = renderPublicationTransportText(input.payload); - if (input.payload.attachmentIds?.length) { - if ( - input.endpoint.provider === "microsoft-teams" && - !input.conversation.isDirectMessage - ) { - // The staged personal-file lane remains authoritative for consent. - // Channel/group pictures need no OneDrive upload or public asset URL: - // send only verified original picture bytes through the native adapter. - const candidates = await publicationFiles( - input.publication, - input.payload, - { teamsInlineImagesOnly: true }, - ); - const pictures = await Promise.all( - candidates.map(prepareTeamsInlineImage), - ); - if (pictures.length && pictures.every((picture) => picture !== null)) { - files = pictures; - } - } - const nativeFileSurface = - CAPABILITIES[input.endpoint.provider].files && - input.endpoint.provider !== "microsoft-teams"; - if (nativeFileSurface) { - const uploads = await publicationFiles( - input.publication, - input.payload, - ); - if (input.endpoint.provider === "telegram") { - attachments = await Promise.all( - uploads.map(telegramAttachmentForUpload), - ); - } else { - files = uploads; - } - } else if (files.length === 0) { - const taskUrl = safeChatTaskUrl( - getTaskBaseUrl(), - input.publication.issueId, - ); - if ( - input.endpoint.provider === "github" || - input.endpoint.provider === "microsoft-teams" - ) { - // GitHub lacks a bot upload contract. Teams reaches this fallback - // only without an admissible personal-consent destination. Replace - // our generated caption, preserving any future custom text. - const generatedFileLabel = /^Shared (.+)\.$/s.exec(text)?.[1] ?? null; - const limitation = - input.endpoint.provider === "github" - ? "This GitHub App connection cannot upload file bytes into comments." - : "Direct file delivery isn't available for this Teams conversation."; - if (generatedFileLabel) { - const saved = taskUrl - ? `File saved on the Paperclip task: ${generatedFileLabel}.` - : `File saved on the private Paperclip task: ${generatedFileLabel}.`; - text = taskUrl - ? `${saved} ${limitation} Download it: ${taskUrl}` - : `${saved} ${limitation}`; - } else { - const handoff = taskUrl - ? `Open the file on its Paperclip task: ${taskUrl}` - : "The file remains available only on the private Paperclip task."; - text = `${text}\n\n${limitation} ${handoff}`; - } - } else { - const attachmentFallback = taskUrl - ? `Open the task in Paperclip: ${taskUrl}` - : "Open the task in Paperclip to download the attachment."; - text = `${text}\n\n${attachmentFallback}`; - } - } - } - if (input.endpoint.provider === "imessage-photon") { - const adapter = endpointRuntime.getProviderAdapter(); - if (!(adapter instanceof PhotonChatAdapter)) - throw new Error("Photon publication runtime unavailable"); - const assertCurrent = async () => { - await input.beforePhotonWrite?.(); - const current = await endpointRecord(input.endpoint.id); - if ( - !current || - !["verifying", "active"].includes(current.endpoint.status) || - runtimeGeneration(current.endpoint.setup) !== - runtimeGeneration(input.endpoint.setup) || - runtime.get(input.endpoint.id) !== endpointRuntime - ) - throw new PhotonError( - "rejected", - "Photon publication authority changed", - ); - const resource = await db - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.companyId, current.endpoint.companyId), - eq(chatEndpointResources.endpointId, current.endpoint.id), - eq(chatEndpointResources.id, input.conversation.resourceId!), - ), - ) - .then((rows) => rows[0]); - if ( - resource?.availability !== "available" || - (!input.conversation.isDirectMessage && !resource.enabled) || - (input.conversation.isDirectMessage && - !current.endpoint.allowDirectMessages) - ) - throw new PhotonError( - "rejected", - "Photon conversation is no longer enabled", - ); - }; - const retry = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.endpointId, input.endpoint.id), - eq(chatActions.kind, "photon_publication_retry"), - eq( - chatActions.providerActionId, - `photon-retry:${input.publication.id}:${input.publication.attempts + 1}`, - ), - eq(chatActions.status, "processed"), - ), - ) - .then((rows) => rows[0]); - const retryUnknown = Boolean(retry); - const nextQuestion = - /^photon-question:([a-zA-Z0-9_-]{8,24}):(\d{1,2})$/.exec( - input.publication.idempotencyKey, - ); - if ( - input.payload.interactionId && - (input.publication.idempotencyKey === - `interaction:${input.payload.interactionId}:${input.endpoint.id}` || - nextQuestion) - ) { - const action = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, input.endpoint.companyId), - eq(chatActions.endpointId, input.endpoint.id), - eq(chatActions.kind, "photon_interaction"), - nextQuestion - ? eq(chatActions.providerActionId, `photon:${nextQuestion[1]}`) - : eq( - sql`${chatActions.payload}->>'publicationId'`, - input.publication.id, - ), - eq(chatActions.status, "issued"), - ), - ) - .then((rows) => rows[0]); - const interaction = ( - await issueThreadInteractionService(db).listForIssue( - input.publication.issueId, - ) - ).find((candidate) => candidate.id === input.payload.interactionId); - if ( - action && - interaction?.status === "pending" && - nativePhotonInteraction(interaction) - ) { - const promptGuard = async () => { - await assertCurrent(); - const current = await db - .select({ status: issueThreadInteractions.status }) - .from(issueThreadInteractions) - .where( - and( - eq( - issueThreadInteractions.companyId, - input.endpoint.companyId, - ), - eq(issueThreadInteractions.id, interaction.id), - ), - ) - .then((rows) => rows[0]); - if (current?.status !== "pending") - throw new PhotonError( - "rejected", - "This interaction has already been resolved", - ); - }; - const receipt = await publishPhotonPrompt({ - adapter, - threadId: thread.id, - binding: action.payload as unknown as PhotonInteractionBinding, - interaction, - questionIndex: nextQuestion ? Number(nextQuestion[2]) : 0, - taskUrl: safeChatTaskUrl( - getTaskBaseUrl(), - input.publication.issueId, - ), - assertCurrent: promptGuard, - retryUnknown, - }); - return { id: receipt.promptMessageGuid }; - } - if (nextQuestion) - throw new PhotonError( - "rejected", - "This question is no longer available", - ); - } - const reply = await adapter.state.read<{ guid: string | null }>( - `publication-reply:${input.publication.id}`, - ); - let replyTo = reply?.guid ?? null; - if (!reply) { - const sourceRunId = await receiptReactionCompletionRunId( - db, - input.publication, - input.payload, - ); - if (sourceRunId) { - const source = await db - .select({ guid: chatMessageLinks.providerMessageId }) - .from(heartbeatRuns) - .innerJoin( - chatMessageLinks, - and( - eq(chatMessageLinks.companyId, input.endpoint.companyId), - eq(chatMessageLinks.endpointId, input.endpoint.id), - eq(chatMessageLinks.conversationId, input.conversation.id), - eq(chatMessageLinks.direction, "inbound"), - or( - sql`${chatMessageLinks.commentId}::text = ${heartbeatRuns.contextSnapshot}->>'wakeCommentId'`, - sql`coalesce(${heartbeatRuns.contextSnapshot}->'wakeCommentIds', '[]'::jsonb) ? ${chatMessageLinks.commentId}::text`, - ), - ), - ) - .where( - and( - eq(heartbeatRuns.id, sourceRunId), - eq(heartbeatRuns.companyId, input.endpoint.companyId), - ), - ) - .orderBy(desc(chatMessageLinks.createdAt)) - .limit(1) - .then((rows) => rows[0]); - replyTo = source?.guid ?? null; - } - replyTo = ( - await adapter.state.update<{ guid: string | null }>( - `publication-reply:${input.publication.id}`, - (current) => current ?? { guid: replyTo }, - ) - ).guid; - } - const sent = await adapter.publish( - thread.id, - input.publication.id, - { markdown: text, files }, - { assertCurrent, retryUnknown, replyTo: replyTo ?? undefined }, - ); - if (input.publication.idempotencyKey.startsWith("photon-response:")) { - const notice = await db - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, input.endpoint.companyId), - eq(chatActions.endpointId, input.endpoint.id), - eq(chatActions.kind, "photon_response_notice"), - eq( - chatActions.providerActionId, - `photon-notice:${input.publication.idempotencyKey}`, - ), - ), - ) - .then((rows) => rows[0]); - if ( - notice && - typeof notice.payload.reference === "string" && - typeof notice.payload.questionIndex === "number" && - typeof notice.payload.publicationId === "string" - ) { - const receipt: PhotonPromptReceipt = { - schema: 1, - reference: notice.payload.reference, - questionIndex: notice.payload.questionIndex, - publicationId: notice.payload.publicationId, - promptMessageGuid: sent.id, - promptMessageGuids: sent.messageIds, - options: {}, - }; - for (const guid of sent.messageIds) - await adapter.state.update( - `prompt-message:${guid}`, - (current) => current ?? receipt, - ); - } - } - return sent; - } - if ( - input.endpoint.provider === "discord" && - input.payload.transportPart?.mode === "discord_markdown_attachment" - ) { - files.push(discordMarkdownAttachment(text)); - text = - input.payload.transportPart.count > 1 - ? "Complete response attached." - : "Paperclip attached the complete response because it exceeds Discord’s message limit."; - } - if ( - input.endpoint.provider === "telegram" && - input.payload.transportPart?.mode === "telegram_markdown_attachment" - ) { - attachments.push(telegramMarkdownAttachment(text)); - text = - input.payload.transportPart.count > 1 - ? "Complete response attached." - : "Paperclip attached the complete response to preserve its Markdown formatting."; - } - if (card && CAPABILITIES[input.endpoint.provider].cards) { - return await attemptProviderPublication(async () => - input.replaceProviderMessageId - ? await editOrPostProviderPublication( - () => - thread.adapter.editMessage( - thread.id, - input.replaceProviderMessageId!, - { - card, - fallbackText: text, - ...(files.length ? { files } : {}), - }, - ), - () => - thread.post({ - card, - fallbackText: text, - ...(files.length ? { files } : {}), - }), - ) - : await thread.post({ - card, - fallbackText: text, - ...(files.length ? { files } : {}), - }), - ); - } - if (attachments.length) { - const attachmentMessage: AdapterPostableMessage = { - attachments, - markdown: text, - }; - return await attemptProviderPublication(async () => - input.replaceProviderMessageId - ? await editOrPostProviderPublication( - () => - thread.adapter.editMessage( - thread.id, - input.replaceProviderMessageId!, - attachmentMessage, - ), - () => thread.post(attachmentMessage), - ) - : await thread.post(attachmentMessage), - ); - } - if (files.length) { - // The pinned Slack adapter performs file upload and text publication as - // two provider calls when both fields are present. A rate limit or - // network failure on the second call would make an automatic retry - // duplicate the already-shared file. File publications have their own - // ordered outbox row, so Slack receives a single native upload here. - const fileMessage: AdapterPostableMessage = { - // Empty Slack markdown keeps the SDK call type-safe while its pinned - // adapter treats this as a file-only upload and skips chat.postMessage. - markdown: input.endpoint.provider === "slack" ? "" : text, - files, - }; - if ( - input.endpoint.provider === "slack" && - !input.replaceProviderMessageId && - !input.payload.card && - !input.payload.interactionId && - !input.publication.idempotencyKey.startsWith("control:") && - input.payload.attachmentIds?.length && - input.onSlackFileUploadAccepted - ) { - return await attemptProviderPublication( - async () => - await endpointRuntime.postSlackFilePublication( - thread.id, - fileMessage, - input.onSlackFileUploadAccepted!, - ), - ); - } - return await attemptProviderPublication(async () => - input.replaceProviderMessageId - ? await editOrPostProviderPublication( - () => - thread.adapter.editMessage( - thread.id, - input.replaceProviderMessageId!, - fileMessage, - ), - () => thread.post(fileMessage), - ) - : await thread.post(fileMessage), - ); - } - if ( - !input.replaceProviderMessageId && - !input.payload.transportPart && - CAPABILITIES[input.endpoint.provider].nativeStreaming && - (input.endpoint.provider !== "telegram" || - !input.conversation.isDirectMessage || - input.telegramDraftControl !== undefined) && - shouldStreamSafePublicationText(text) - ) { - return await attemptProviderPublication(async () => - input.telegramDraftControl - ? endpointRuntime.streamTelegramDraft( - input.conversation.externalThreadId, - streamSafePublicationText(text), - input.telegramDraftControl, - ) - : thread.post(streamSafePublicationText(text)), - ); - } - return await attemptProviderPublication(async () => - input.replaceProviderMessageId - ? await editOrPostProviderPublication( - () => - thread.adapter.editMessage( - thread.id, - input.replaceProviderMessageId!, - { markdown: text }, - ), - () => thread.post({ markdown: text }), - ) - : await thread.post({ markdown: text }), - ); + async function stageTerminalReceiptReactionRemovals(...args: Parameters["stageTerminalReceiptReactionRemovals"]>) { + return publicationWorker.stageTerminalReceiptReactionRemovals(...args); } - async function claimPublicationTransportAuthorization(input: { - credentialLease: CredentialMutationLeaseGuard; - endpoint: EndpointRow; - publication: typeof chatPublications.$inferSelect; - runtimeContext: LifecycleRuntimeFence; - }): Promise<{ - authorizationActionId: string | null; - conversation: ConversationRow; - endpoint: EndpointRow; - preparedText: string | null; - } | null> { - return db.transaction(async (tx) => { - await input.credentialLease.assertOwned(tx); - const currentPublication = await tx - .select({ id: chatPublications.id }) - .from(chatPublications) - .where( - and( - eq(chatPublications.id, input.publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, input.publication.attempts + 1), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null); - if (!currentPublication) return null; - try { - if ( - !(await authorizeRetainedChatSourcePublication(tx, input.publication)) - ) - return null; - } catch (error) { - if (error instanceof NativeChatReviewPresentationContentionError) - throw error; - throw new FailedChatRetryPublicationReadError(); - } - const attachmentFailureFence = attachmentFailureNoticeFence( - input.publication.idempotencyKey, - ); - if ( - attachmentFailureFence && - (attachmentFailureFence.generation !== - input.runtimeContext.generation || - attachmentFailureFence.credentialFingerprint !== - input.runtimeContext.credentialFingerprint) - ) { - return null; - } - const endpoint = await runtimeCallbackEndpoint( - tx, - input.publication.endpointId, - input.runtimeContext, - ["verifying", "active"], - ); - const conversation = endpoint - ? await tx - .select() - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, endpoint.companyId), - eq(chatConversations.endpointId, endpoint.id), - eq(chatConversations.id, input.publication.conversationId), - inArray(chatConversations.state, [ - "active", - "waiting", - "completed", - ]), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const resource = - endpoint && conversation?.resourceId && !conversation.isDirectMessage - ? await tx - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.companyId, endpoint.companyId), - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.id, conversation.resourceId), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const attachmentFailureNoticeSuperseded = - attachmentFailureFence && conversation - ? await tx - .select({ id: chatConversations.id }) - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, conversation.companyId), - eq(chatConversations.endpointId, conversation.endpointId), - eq( - chatConversations.externalThreadId, - conversation.externalThreadId, - ), - gt( - chatConversations.sessionGeneration, - conversation.sessionGeneration, - ), - ), - ) - .limit(1) - .then((rows) => rows.length > 0) - : false; - const isTaskControl = - input.publication.idempotencyKey.startsWith("control:"); - const providerVisibleCompletion = - conversation?.state === "completed" && - !isExplicitOperatorPublication(input.publication) && - (await hasCommittedTaskControlCompletion(conversation.id, tx)); - const authorizationAction = - endpoint && isTaskControl - ? await tx - .select() - .from(chatActions) - .where( - and( - eq(chatActions.companyId, endpoint.companyId), - eq(chatActions.endpointId, endpoint.id), - eq(chatActions.conversationId, conversation?.id ?? ""), - eq(chatActions.kind, "task_control_authorization"), - eq( - chatActions.providerActionId, - `task-control-authorization:${input.publication.id}`, - ), - eq(chatActions.status, "issued"), - ), - ) - .for("update") - .then((rows) => rows[0] ?? null) - : null; - const destinationAllowed = - endpoint !== null && - conversation !== null && - (endpoint.provider !== "slack" || await slackPublicationAllowed(tx, endpoint.companyId, endpoint.id, input.publication.issueId, conversation.externalConversationId, null)) && - (conversation.isDirectMessage - ? endpoint.allowDirectMessages - : nonDirectDestinationAllowed(endpoint, resource)); - if ( - !endpoint || - !conversation || - (attachmentFailureFence && - ((conversation.state !== "active" && - conversation.state !== "waiting") || - attachmentFailureNoticeSuperseded)) || - providerVisibleCompletion || - (isTaskControl && !authorizationAction?.principalId) || - !destinationAllowed - ) { - return null; - } - let authorizationActionId: string | null = null; - if (endpoint.provider === "slack") { - const credentials = await resolveCredentials(endpoint, tx); - if (!await authorizeSlackBoardPublication(tx, endpoint, conversation, input.publication, credentials.botToken ?? "", fetchImpl)) return null; - } - if ( - input.publication.idempotencyKey.startsWith("wake:") && - !(await authorizeInboundWakePublication(tx, input.publication)) - ) - return null; - // A native response alongside an older completion review has its own - // narrow authorization. Recheck that exact proof, review and original - // requester at the transport claim, including file publications whose - // comment is the run's earlier internal attachment-selection comment. - if ( - input.publication.commentId && - !isExplicitOperatorPublication(input.publication) - ) { - const [origin] = await tx - .select({ - runId: heartbeatRuns.id, - resultJson: heartbeatRuns.resultJson, - }) - .from(issueComments) - .innerJoin( - heartbeatRuns, - and( - eq(heartbeatRuns.id, issueComments.createdByRunId), - eq(heartbeatRuns.companyId, issueComments.companyId), - ), - ) - .where( - and( - eq(issueComments.id, input.publication.commentId), - eq(issueComments.companyId, endpoint.companyId), - eq(issueComments.issueId, input.publication.issueId), - ), - ) - .limit(1); - if ( - origin?.resultJson?.finalizationReasonCode === - "governed_response_waiting" - ) { - try { - if ( - !(await authorizeNativeChatReviewPresentation( - tx as unknown as Db, - { - companyId: endpoint.companyId, - issueId: input.publication.issueId, - runId: origin.runId, - resultJson: origin.resultJson, - destination: { - endpointId: endpoint.id, - conversationId: conversation.id, - }, - }, - "nonblocking", - )) - ) - return null; - } catch (error) { - if (isExternalChatWaitAuthorizationContention(error)) - throw new NativeChatReviewPresentationContentionError(); - throw error; - } - } - } - const preparedText = await githubCurrentOmissionPreparedText( - tx, - input.publication, - endpoint, - ); - if (preparedText === false) return null; - if (authorizationAction?.principalId) { - const authorization = await lockCurrentPrincipalAuthorization( - tx, - endpoint, - authorizationAction.principalId, - ); - if (!authorization.allowed) return null; - const [claimedAuthorization] = await tx - .update(chatActions) - .set({ - status: "processing", - result: { - attempts: input.publication.attempts + 1, - authorizedUserId: authorization.userId, - credentialFingerprint: input.runtimeContext.credentialFingerprint, - runtimeGeneration: input.runtimeContext.generation, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.id, authorizationAction.id), - eq(chatActions.status, "issued"), - ), - ) - .returning({ id: chatActions.id }); - if (!claimedAuthorization) return null; - authorizationActionId = claimedAuthorization.id; - } - if (preparedText !== null) { - // Persist the exact provider text before I/O. A retry must reuse this - // body, not accumulate links or regenerate it from changed settings. - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ - payload: { ...input.publication.payload, text: preparedText }, - }) - .where(eq(chatPublications.id, input.publication.id)); - } - await input.credentialLease.assertOwned(tx); - // The publication's streaming row plus the optional authorization action - // are the durable transport claim. Their database locks are released - // before provider I/O. The renewable credential lease intentionally - // remains held so reconnect and reach mutations linearize after this - // committed authorization snapshot and transport attempt. - return { - authorizationActionId, - conversation, - endpoint, - preparedText, - }; - }); + async function processPendingPublications(...args: Parameters["processPendingPublications"]>) { + return publicationQueue.processPendingPublications(...args); } - async function stageTerminalReceiptReactionRemovals( - tx: Db, - input: { - endpoint: EndpointRow; - publication: typeof chatPublications.$inferSelect; - payload: SafeChatPublicationPayload; - runtimeContext: RuntimeContext; - closedProgressRunId?: string; - }, - ): Promise { - if ( - !["discord", "slack", "telegram", "github"].includes(input.endpoint.provider) - ) return []; - const runId = input.closedProgressRunId ?? - (await receiptReactionCompletionRunId(tx, input.publication, input.payload)); - if (!runId) return []; - return stageReceiptReactionRemovals(tx, { - endpoint: input.endpoint, - binding: input.publication, - runId, - runtimeContext: input.runtimeContext, - }); - } - - async function currentTaskControlPayload( - publication: typeof chatPublications.$inferSelect, - ): Promise { - const persisted = publication.payload as SafeChatPublicationPayload; - if (!publication.idempotencyKey.startsWith("control:status:")) { - return persisted; - } - const issue = await db - .select({ - identifier: issues.identifier, - status: issues.status, - title: issues.title, - }) - .from(issues) - .where( - and( - eq(issues.companyId, publication.companyId), - eq(issues.id, publication.issueId), - ), - ) - .then((rows) => rows[0] ?? null); - if (!issue) return persisted; - return projectSafeChatPublication({ - classification: "external", - source: "task_control", - text: `${issue.identifier}: ${issue.title} — ${issue.status}`, - }); - } - - function taskControlCompletesConversation( - publication: Pick, - ): boolean { - return /^control:(?:new|close):/.test(publication.idempotencyKey); - } - - function isExplicitOperatorPublication( - publication: Pick, - ): boolean { - return /^(?:explicit:|explicit-board:)/.test(publication.idempotencyKey); - } - - async function hasCommittedTaskControlCompletion( - conversationId: string, - database: DbOrTransaction = db, - ): Promise { - return database - .select({ id: chatPublications.id }) - .from(chatPublications) - .where( - and( - eq(chatPublications.conversationId, conversationId), - eq(chatPublications.state, "published"), - or( - like(chatPublications.idempotencyKey, "control:new:%"), - like(chatPublications.idempotencyKey, "control:close:%"), - ), - ), - ) - .limit(1) - .then((rows) => rows.length > 0); - } - - async function commitTaskControlCompletion( - tx: Db, - publication: Pick< - typeof chatPublications.$inferSelect, - | "companyId" - | "conversationId" - | "createdAt" - | "endpointId" - | "id" - | "idempotencyKey" - | "payload" - >, - committedAt: Date, - ): Promise { - if (!taskControlCompletesConversation(publication)) return; - await tx - .update(chatConversations) - .set({ state: "completed", updatedAt: committedAt }) - .where( - and( - eq(chatConversations.companyId, publication.companyId), - eq(chatConversations.endpointId, publication.endpointId), - eq(chatConversations.id, publication.conversationId), - inArray(chatConversations.state, ["active", "waiting"]), - ), - ); - // The provider-visible completion is the external boundary. Anything - // enqueued later for this old generation stays internal even if its run - // finishes after /new or /close. A racing insert that commits after this - // transaction is rejected by the completed-conversation send guard. - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ - state: "cancelled", - nextAttemptAt: null, - redactedError: - "Conversation was completed by a task control before delivery", - updatedAt: committedAt, - }) - .where( - and( - eq(chatPublications.companyId, publication.companyId), - eq(chatPublications.endpointId, publication.endpointId), - eq(chatPublications.conversationId, publication.conversationId), - or( - gt(chatPublications.createdAt, publication.createdAt), - and( - eq(chatPublications.createdAt, publication.createdAt), - gt( - publicationTransportOrderKey(chatPublications), - publication.payload.transportPart?.orderKey ?? publication.id, - ), - ), - ), - inArray(chatPublications.state, ["pending", "retry", "streaming"]), - sql`not (${chatPublications.idempotencyKey} like 'explicit:%' or ${chatPublications.idempotencyKey} like 'explicit-board:%')`, - ), - ); - } - - async function finalizePublicationFailure( - publication: typeof chatPublications.$inferSelect, - error: unknown, - providerAccepted = false, - runtimeContext?: LifecycleRuntimeFence | null, - credentialLease?: CredentialMutationLeaseGuard, - ): Promise { - const attempts = publication.attempts + 1; - const disposition = providerAccepted - ? { - kind: "delivery_unknown" as const, - reason: - "Provider accepted the publication, but Paperclip could not confirm its durable result", - } - : error instanceof NativeChatReviewPresentationContentionError - ? { kind: "retry" as const, retryAfterMs: 250, reason: error.message } - : error instanceof FailedChatRetryPublicationReadError - ? { - kind: "retry" as const, - retryAfterMs: 1_000, - reason: error.message, - } - : isOutboundAttachmentValidationError(error) - ? { - kind: "failed" as const, - reason: error.message, - } - : isOutboundAttachmentHydrationError(error) - ? { - kind: "retry" as const, - retryAfterMs: Math.min( - 60_000, - 2 ** Math.max(0, attempts) * 1_000, - ), - reason: error.message, - } - : classifyChatPublicationError(error, attempts); - const failure = redactSensitiveText(disposition.reason).slice( - 0, - MAX_ERROR_TEXT, - ); - // Provider-directed rate limits are not delivery failures. Keep the - // durable publication retryable for the provider's full interval instead - // of exhausting the five-attempt transport budget while flood control is - // still active. - const terminalRetry = - disposition.kind === "retry" && - disposition.providerRateLimit !== true && - attempts >= 5; - const attachmentFailure = attachmentFailureKind(publication.payload); - const attachmentFailureRuntimeContext = - !providerAccepted && - disposition.kind === "failed" && - runtimeContext && - attachmentFailure && - definiteAttachmentProviderRejection(error) - ? runtimeContext - : null; - let invalidateCurrentRuntime = false; - const finalized = await db.transaction(async (tx) => { - await credentialLease?.assertOwned(tx); - const state = - disposition.kind === "retry" && !terminalRetry - ? "retry" - : disposition.kind === "delivery_unknown" - ? "delivery_unknown" - : disposition.kind === "resource_unavailable" - ? "cancelled" - : "failed"; - await notifyChatPublicationWork(tx); - const [ownedPublication] = await tx - .update(chatPublications) - .set({ - state, - nextAttemptAt: - disposition.kind === "retry" && !terminalRetry - ? new Date(Date.now() + disposition.retryAfterMs) - : null, - redactedError: failure, - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, attempts), - ), - ) - .returning({ id: chatPublications.id }); - if (!ownedPublication) return false; - - if (publication.idempotencyKey.startsWith("control:")) { - await tx - .update(chatActions) - .set({ - status: - state === "delivery_unknown" - ? "delivery_unknown" - : state === "cancelled" - ? "cancelled" - : "issued", - result: { - code: - state === "delivery_unknown" - ? "task_control_delivery_unknown" - : state === "cancelled" - ? "task_control_destination_unavailable" - : "task_control_transport_failed", - attempts, - retryable: state === "retry", - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.endpointId, publication.endpointId), - eq( - chatActions.providerActionId, - `task-control-authorization:${publication.id}`, - ), - eq(chatActions.status, "processing"), - sql`(${chatActions.result}->>'attempts')::int = ${attempts}`, - ), - ); - } - - const currentEndpoint = - runtimeContext && - (disposition.kind === "endpoint_attention" || - disposition.kind === "resource_unavailable") - ? await runtimeCallbackEndpoint( - tx, - publication.endpointId, - runtimeContext, - ["verifying", "active"], - ) - : null; - - if (disposition.kind === "endpoint_attention" && currentEndpoint) { - await tx - .update(chatEndpoints) - .set({ - status: "attention", - healthMessage: "Provider credentials or permissions need attention", - lastError: failure, - updatedAt: new Date(), - }) - .where(eq(chatEndpoints.id, currentEndpoint.id)); - await tx - .update(toolConnections) - .set({ - status: "disabled", - enabled: false, - healthStatus: "degraded", - healthMessage: "Provider credentials or permissions need attention", - lastError: failure, - healthCheckedAt: new Date(), - updatedAt: new Date(), - }) - .where(eq(toolConnections.id, currentEndpoint.connectionId)); - invalidateCurrentRuntime = true; - } else if ( - disposition.kind === "resource_unavailable" && - currentEndpoint - ) { - const binding = await tx - .select({ resourceId: chatConversations.resourceId }) - .from(chatConversations) - .where(eq(chatConversations.id, publication.conversationId)) - .then((result) => result[0] ?? null); - await tx - .update(chatConversations) - .set({ state: "unavailable", updatedAt: new Date() }) - .where( - and( - eq(chatConversations.id, publication.conversationId), - eq(chatConversations.endpointId, currentEndpoint.id), - ), - ); - if (binding?.resourceId) { - await tx - .update(chatEndpointResources) - .set({ availability: "unavailable", updatedAt: new Date() }) - .where( - and( - eq(chatEndpointResources.id, binding.resourceId), - eq(chatEndpointResources.endpointId, currentEndpoint.id), - ), - ); - } - } - - if (attachmentFailureRuntimeContext && attachmentFailure) { - const noticeEndpoint = await runtimeCallbackEndpoint( - tx, - publication.endpointId, - attachmentFailureRuntimeContext, - ["verifying", "active"], - ); - const noticeConversation = noticeEndpoint - ? await tx - .select({ - id: chatConversations.id, - state: chatConversations.state, - }) - .from(chatConversations) - .where( - and( - eq(chatConversations.companyId, publication.companyId), - eq(chatConversations.endpointId, publication.endpointId), - eq(chatConversations.id, publication.conversationId), - inArray(chatConversations.state, ["active", "waiting"]), - ), - ) - .for("no key update") - .then((rows) => rows[0] ?? null) - : null; - const providerCanSendTextNotice = - noticeEndpoint && - ["slack", "discord", "telegram"].includes(noticeEndpoint.provider) && - (attachmentFailure.provider === null || - noticeEndpoint.provider === attachmentFailure.provider); - if (providerCanSendTextNotice && noticeConversation) { - const taskUrl = safeChatTaskUrl( - getTaskBaseUrl(), - publication.issueId, - ); - const noticeText = - attachmentFailure.kind === "generated_response" - ? "Paperclip could not send the response attachment. The complete response remains on its Paperclip task for an operator to retry." - : "Paperclip could not send an attachment. The file remains on its Paperclip task for an operator to retry."; - const idempotencyKey = `${ATTACHMENT_FAILURE_NOTICE_PREFIX}${publication.id}:${attachmentFailureRuntimeContext.generation}:${attachmentFailureRuntimeContext.credentialFingerprint}`; - await notifyChatPublicationWork(tx); - await tx - .insert(chatPublications) - .values({ - companyId: publication.companyId, - endpointId: publication.endpointId, - conversationId: publication.conversationId, - issueId: publication.issueId, - commentId: null, - idempotencyKey, - payload: projectSafeChatPublication({ - classification: "external", - source: "safe_milestone", - text: taskUrl - ? `${noticeText} Open task: ${taskUrl}` - : noticeText, - }), - state: "pending", - }) - .onConflictDoNothing(); - } - } - - await credentialLease?.assertOwned(tx); - return true; - }); - if (finalized && invalidateCurrentRuntime) { - await credentialLease?.assertOwned(); - await invalidateRuntime(publication.endpointId).catch(() => undefined); - } - if (finalized) { - logger.warn( - { - endpointId: publication.endpointId, - publicationId: publication.id, - error: failure, - disposition: disposition.kind, - }, - "chat publication failed", - ); - } - return finalized; - } - - async function ensureDurablePublicationTransport( - publication: typeof chatPublications.$inferSelect, - ): Promise { - const persisted = publication.payload as SafeChatPublicationPayload; - if ( - !persisted || - typeof persisted !== "object" || - typeof persisted.text !== "string" - ) { - throw Object.assign( - new Error("Durable chat publication payload is malformed"), - { code: "VALIDATION_ERROR", name: "ValidationError" }, - ); - } - // Validate closed persisted wrappers even on retries that are already split. - renderPublicationTransportText(persisted); - await options.publicationTransportPreparationBarrier?.({ - publicationId: publication.id, - }); - const provider = await db - .select({ provider: chatEndpoints.provider }) - .from(chatEndpoints) - .where(eq(chatEndpoints.id, publication.endpointId)) - .then((rows) => rows[0]?.provider ?? null); - if ( - provider === "slack" || - provider === "github" || - provider === "microsoft-teams" - ) { - if ( - persisted.transportPart || - persisted.card || - persisted.interactionId || - persisted.attachmentIds?.length - ) - return publication; - const parts = splitNativePublicationText(provider, persisted.text); - if (parts.length === 1) return publication; - return db.transaction(async (tx) => { - const current = await tx - .select() - .from(chatPublications) - .where(eq(chatPublications.id, publication.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) return publication; - const currentPayload = current.payload as SafeChatPublicationPayload; - renderPublicationTransportText(currentPayload); - if ( - currentPayload.transportPart || - currentPayload.card || - currentPayload.interactionId || - currentPayload.attachmentIds?.length || - !["pending", "retry"].includes(current.state) - ) - return current; - const currentParts = - currentPayload.text === persisted.text - ? parts - : splitNativePublicationText(provider, currentPayload.text); - if (currentParts.length === 1) return current; - const batchId = current.id; - const updatedAt = new Date(); - const payloadFor = (index: number): SafeChatPublicationPayload => { - const part = currentParts[index]!; - const payload = { - ...currentPayload, - text: part.text, - transportPart: { - batchId, - index, - count: currentParts.length, - mode: "inline" as const, - orderKey: `${batchId}:${String(index).padStart(4, "0")}`, - ...(part.prefix ? { prefix: part.prefix } : {}), - ...(part.suffix ? { suffix: part.suffix } : {}), - }, - }; - if (index > 0) delete payload.progressState; - return payload; - }; - const rootPayload = payloadFor(0); - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: rootPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - await notifyChatPublicationWork(tx); - await tx.insert(chatPublications).values( - currentParts.slice(1).map((_part, offset) => ({ - companyId: current.companyId, - endpointId: current.endpointId, - conversationId: current.conversationId, - issueId: current.issueId, - commentId: current.commentId, - idempotencyKey: `${current.idempotencyKey}:${provider}-part:${offset + 1}`, - payload: payloadFor(offset + 1), - state: "pending" as const, - createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, - updatedAt, - })), - ); - return { ...current, payload: rootPayload, updatedAt }; - }); - } - if (provider === "discord") { - if ( - persisted.transportPart || - persisted.card || - persisted.interactionId || - !discordMarkdownRequiresAttachment(persisted.text) - ) { - return publication; - } - - // A normal response can replace its run's working placeholder. Discord's - // adapter cannot add a file while editing, so first turn that placeholder - // into a durable hand-off and then publish the complete Markdown as its - // own ordered attachment row. Publications without a replacement lane - // remain one provider call: a short caption and the lossless file. - const replacesProgressMessage = Boolean( - await runPublicationToReplace(publication, persisted), - ); - return db.transaction(async (tx) => { - const current = await tx - .select() - .from(chatPublications) - .where(eq(chatPublications.id, publication.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) return publication; - const currentPayload = current.payload as SafeChatPublicationPayload; - if ( - currentPayload.transportPart || - currentPayload.card || - currentPayload.interactionId || - !["pending", "retry"].includes(current.state) || - !discordMarkdownRequiresAttachment(currentPayload.text) - ) { - return current; - } - - const batchId = current.id; - const updatedAt = new Date(); - const transportPart = ( - index: number, - count: number, - mode: "inline" | "discord_markdown_attachment", - ) => ({ - batchId, - count, - index, - mode, - orderKey: `${batchId}:${String(index).padStart(4, "0")}`, - }); - if (!replacesProgressMessage) { - const attachmentPayload: SafeChatPublicationPayload = { - ...currentPayload, - transportPart: transportPart(0, 1, "discord_markdown_attachment"), - }; - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: attachmentPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - return { ...current, payload: attachmentPayload, updatedAt }; - } - - const handoffPayload: SafeChatPublicationPayload = { - ...currentPayload, - text: "This response needs a separate attachment because it exceeds the message limit.", - transportPart: transportPart(0, 2, "inline"), - }; - delete handoffPayload.attachmentIds; - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: handoffPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - - const attachmentPayload: SafeChatPublicationPayload = { - ...currentPayload, - transportPart: transportPart(1, 2, "discord_markdown_attachment"), - }; - delete attachmentPayload.progressState; - await notifyChatPublicationWork(tx); - await tx.insert(chatPublications).values({ - companyId: current.companyId, - endpointId: current.endpointId, - conversationId: current.conversationId, - issueId: current.issueId, - commentId: current.commentId, - idempotencyKey: `discord-markdown-attachment:${current.id}`, - payload: attachmentPayload, - state: "pending", - createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, - updatedAt, - }); - return { ...current, payload: handoffPayload, updatedAt }; - }); - } - - if (provider !== "telegram") return publication; - if ( - persisted.transportPart || - persisted.card || - persisted.interactionId || - persisted.attachmentIds?.length - ) { - return publication; - } - if (telegramMarkdownRequiresAttachment(persisted.text)) { - // Telegram parses each message independently. A fixed-size split can - // turn the second half of a code fence, link, or list into unrelated - // plain text even though concatenating the source parts is lossless. - // Preserve structured long Markdown as one document. When this is a run - // final, first complete its existing progress placeholder, then send the - // file through a separate ordered outbox row because Telegram cannot add - // an attachment while editing that placeholder. - const replacesProgressMessage = Boolean( - await runPublicationToReplace(publication, persisted), - ); - return db.transaction(async (tx) => { - const current = await tx - .select() - .from(chatPublications) - .where(eq(chatPublications.id, publication.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) return publication; - const currentPayload = current.payload as SafeChatPublicationPayload; - if ( - currentPayload.transportPart || - currentPayload.card || - currentPayload.interactionId || - currentPayload.attachmentIds?.length || - !["pending", "retry"].includes(current.state) || - !telegramMarkdownRequiresAttachment(currentPayload.text) - ) { - return current; - } - - const batchId = current.id; - const updatedAt = new Date(); - const transportPart = ( - index: number, - count: number, - mode: "inline" | "telegram_markdown_attachment", - ) => ({ - batchId, - count, - index, - mode, - orderKey: `${batchId}:${String(index).padStart(4, "0")}`, - }); - if (!replacesProgressMessage) { - const attachmentPayload: SafeChatPublicationPayload = { - ...currentPayload, - transportPart: transportPart(0, 1, "telegram_markdown_attachment"), - }; - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: attachmentPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - return { ...current, payload: attachmentPayload, updatedAt }; - } - - const handoffPayload: SafeChatPublicationPayload = { - ...currentPayload, - text: "This response needs a separate attachment because it exceeds the message limit.", - transportPart: transportPart(0, 2, "inline"), - }; - delete handoffPayload.attachmentIds; - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: handoffPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - - const attachmentPayload: SafeChatPublicationPayload = { - ...currentPayload, - transportPart: transportPart(1, 2, "telegram_markdown_attachment"), - }; - delete attachmentPayload.progressState; - await notifyChatPublicationWork(tx); - await tx.insert(chatPublications).values({ - companyId: current.companyId, - endpointId: current.endpointId, - conversationId: current.conversationId, - issueId: current.issueId, - commentId: current.commentId, - idempotencyKey: `telegram-markdown-attachment:${current.id}`, - payload: attachmentPayload, - state: "pending", - createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, - updatedAt, - }); - return { ...current, payload: handoffPayload, updatedAt }; - }); - } - const parts = splitTelegramPublicationText(persisted.text); - if (parts.length === 1) return publication; - - return db.transaction(async (tx) => { - const current = await tx - .select() - .from(chatPublications) - .where(eq(chatPublications.id, publication.id)) - .for("update") - .then((rows) => rows[0] ?? null); - if (!current) return publication; - const currentPayload = current.payload as SafeChatPublicationPayload; - if ( - currentPayload.transportPart || - currentPayload.card || - currentPayload.interactionId || - currentPayload.attachmentIds?.length || - !["pending", "retry"].includes(current.state) - ) { - return current; - } - const currentParts = splitTelegramPublicationText(currentPayload.text); - if (currentParts.length === 1) return current; - - const batchId = current.id; - const updatedAt = new Date(); - const transportPart = (index: number) => ({ - batchId, - count: currentParts.length, - index, - orderKey: `${batchId}:${String(index).padStart(4, "0")}`, - }); - const rootPayload: SafeChatPublicationPayload = { - ...currentPayload, - text: currentParts[0]!, - transportPart: transportPart(0), - }; - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ payload: rootPayload, updatedAt }) - .where(eq(chatPublications.id, current.id)); - - const { progressState: _progressState, ...nonProgressPayload } = - currentPayload; - await notifyChatPublicationWork(tx); - await tx.insert(chatPublications).values( - currentParts.slice(1).map((text, offset) => { - const index = offset + 1; - return { - companyId: current.companyId, - endpointId: current.endpointId, - conversationId: current.conversationId, - issueId: current.issueId, - commentId: current.commentId, - idempotencyKey: `${current.idempotencyKey}:telegram-part:${index}`, - payload: { - ...nonProgressPayload, - text, - transportPart: transportPart(index), - }, - state: "pending" as const, - createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, - updatedAt, - }; - }), - ); - return { ...current, payload: rootPayload, updatedAt }; - }); - } - - async function settlePublicationPreparationFailure( - publication: typeof chatPublications.$inferSelect, - error: unknown, - ): Promise { - const attempts = publication.attempts + 1; - const classified = classifyChatPublicationError(error, attempts); - // No provider I/O has started at this boundary. Unknown transport or - // database failures are therefore safe to retry; only a definite local - // validation rejection (or repeated exhaustion) is terminal. - const terminal = classified.kind === "failed" || attempts >= 5; - const retryAfterMs = - classified.kind === "retry" - ? classified.retryAfterMs - : Math.min(60_000, 2 ** Math.max(0, attempts) * 1_000); - const result = await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - const [settled] = await tx - .update(chatPublications) - .set({ - state: terminal ? "failed" : "retry", - attempts, - nextAttemptAt: terminal ? null : new Date(Date.now() + retryAfterMs), - redactedError: terminal - ? "Publication payload could not be prepared for provider delivery" - : "Publication preparation failed before provider delivery", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, publication.state), - eq(chatPublications.attempts, publication.attempts), - ), - ) - .returning({ id: chatPublications.id }); - if (!settled) return false; - if (terminal && publication.idempotencyKey.startsWith("control:")) { - await tx - .update(chatActions) - .set({ - status: "cancelled", - result: { - code: "task_control_publication_invalid", - attempts, - retryable: false, - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.endpointId, publication.endpointId), - eq( - chatActions.providerActionId, - `task-control-authorization:${publication.id}`, - ), - eq(chatActions.status, "issued"), - ), - ); - } - return true; - }); - if (result) { - logger.warn( - { - endpointId: publication.endpointId, - publicationId: publication.id, - error: redactError(error), - terminal, - }, - "chat publication preparation failed", - ); - } - return result; - } - - async function withPublicationAttemptLease( - publication: typeof chatPublications.$inferSelect, - operation: (lease: CredentialMutationLeaseGuard) => Promise, - ): Promise { - // A scheduled sweep may overlap a slow provider call (also on a standby - // server). Claim and exact-attempt ownership must become visible together: - // endpoint-wide activity is not evidence that this publication is alive. - const token = randomUUID(); - const leaseKey = `publication:${publication.id}:${publication.attempts + 1}`; - const now = new Date(); - const owned = and( - eq(chatEndpointLeases.companyId, publication.companyId), - eq(chatEndpointLeases.endpointId, publication.endpointId), - eq(chatEndpointLeases.leaseKey, leaseKey), - eq(chatEndpointLeases.token, token), - ); - const claimed = await db.transaction(async (tx) => { - // Transfer issuance locks this same publication. Check after acquiring - // that lock, not through a pre-lock snapshot that could miss a new lane. - const current = await tx - .select({ id: chatPublications.id }) - .from(chatPublications) - .where( - and( - eq(chatPublications.companyId, publication.companyId), - eq(chatPublications.id, publication.id), - eq(chatPublications.state, publication.state), - eq(chatPublications.attempts, publication.attempts), - ), - ) - .for("update") - .then((rows) => rows[0]); - if (!current) return false; - const transfer = await tx - .select({ id: chatTeamsFileTransfers.id }) - .from(chatTeamsFileTransfers) - .where( - and( - eq(chatTeamsFileTransfers.companyId, publication.companyId), - eq(chatTeamsFileTransfers.publicationId, current.id), - ), - ) - .then((rows) => rows[0]); - if (transfer) return false; - await notifyChatPublicationWork(tx); - const [row] = await tx - .update(chatPublications) - .set({ - state: "streaming", - attempts: publication.attempts + 1, - updatedAt: now, - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, publication.state), - eq(chatPublications.attempts, publication.attempts), - ), - ) - .returning({ id: chatPublications.id }); - if (!row) return false; - await tx.insert(chatEndpointLeases).values({ - companyId: publication.companyId, - endpointId: publication.endpointId, - leaseKey, - token, - expiresAt: new Date(now.getTime() + CREDENTIAL_MUTATION_LEASE_TTL_MS), - }); - return true; - }); - if (!claimed) return; - let leaseLoss: Error | null = null; - const assertOwned: CredentialMutationLeaseGuard["assertOwned"] = async ( - database = db, - ) => { - if (leaseLoss) throw leaseLoss; - try { - // Use a savepoint when the caller already owns a transaction. The - // decision clock is sampled only after the lease row lock resolves; - // an UPDATE predicate using a pre-lock clock can revive expired work. - await database.transaction(async (tx) => { - const current = await tx - .select({ expiresAt: chatEndpointLeases.expiresAt }) - .from(chatEndpointLeases) - .where(owned) - .for("update") - .then((rows) => rows[0] ?? null); - const checkedAt = new Date(); - if ( - leaseLoss || - !current || - current.expiresAt.getTime() <= checkedAt.getTime() - ) { - throw new Error("Publication attempt lease is no longer owned"); - } - await tx - .update(chatEndpointLeases) - .set({ - expiresAt: new Date( - checkedAt.getTime() + CREDENTIAL_MUTATION_LEASE_TTL_MS, - ), - updatedAt: checkedAt, - }) - .where(owned); - }); - } catch (error) { - leaseLoss = Object.assign( - new Error( - "Chat publication attempt ownership was lost before completion", - { cause: error }, - ), - { code: "CHAT_PUBLICATION_LEASE_LOST" }, - ); - throw leaseLoss; - } - }; - let renewal: Promise | null = null; - const timer = setInterval(() => { - if (renewal || leaseLoss) return; - renewal = assertOwned() - .catch(() => undefined) - .finally(() => { - renewal = null; - }); - }, CREDENTIAL_MUTATION_LEASE_TTL_MS / 3); - timer.unref?.(); - try { - await operation({ assertOwned }); - } finally { - clearInterval(timer); - await renewal; - await db.delete(chatEndpointLeases).where(owned); - } - } - - async function processSelectedPublication( - selectedPublication: typeof chatPublications.$inferSelect, - ): Promise { - if (await settleGitHubAutomaticPublication(selectedPublication)) return; - let publication: typeof chatPublications.$inferSelect; - try { - publication = - await ensureDurablePublicationTransport(selectedPublication); - } catch (error) { - try { - await settlePublicationPreparationFailure(selectedPublication, error); - } catch (settlementError) { - logger.error( - { - endpointId: selectedPublication.endpointId, - publicationId: selectedPublication.id, - error: redactError(settlementError), - }, - "could not durably settle chat publication preparation failure", - ); - } - return; - } - const explicitSlackDelivery = await slackExplicitPublicationDuplicate(db, publication); - if (explicitSlackDelivery === "unresolved") { - // Keep the durable publication pending until the explicit send is settled; - // a missing provider receipt does not authorize a duplicate. - return; - } - if (explicitSlackDelivery === "delivered") { - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx.update(chatPublications).set({ state: "cancelled", redactedError: "Identical response already delivered by Slack tool", updatedAt: new Date() }).where(and(eq(chatPublications.id, publication.id), inArray(chatPublications.state, ["pending", "retry"]))) -; - }); - return; - } - const earlierOpenPublication = await db - .select({ id: chatPublications.id }) - .from(chatPublications) - .where( - and( - eq(chatPublications.conversationId, publication.conversationId), - or( - lt(chatPublications.createdAt, publication.createdAt), - and( - eq(chatPublications.createdAt, publication.createdAt), - lt( - publicationTransportOrderKey(chatPublications), - publication.payload.transportPart?.orderKey ?? publication.id, - ), - ), - ), - inArray(chatPublications.state, [ - "pending", - "retry", - "streaming", - "delivery_unknown", - ]), - ), - ) - .limit(1) - .then((result) => result[0] ?? null); - if (earlierOpenPublication) return; - if (await tryProcessTeamsFilePublication(publication)) return; - await withPublicationAttemptLease(publication, async (publicationLease) => { - let providerAccepted = false; - let publicationRuntimeContext: RuntimeContext | null = null; - let failureHandledWithinCredentialLease = false; - let slackFileReceiptActionId: string | null = null; - const settlePublicationFailure = async ( - error: unknown, - credentialLease?: CredentialMutationLeaseGuard, - ) => { - try { - await finalizePublicationFailure( - publication, - error, - providerAccepted, - publicationRuntimeContext, - credentialLease ?? publicationLease, - ); - } catch (finalizationError) { - // If the provider accepted the message, a failed receipt write - // must remain streaming until the stale scan quarantines it. A - // later lifecycle mutation must not turn this into an implicit - // replay or allow a stale failure to overwrite the new runtime. - logger.error( - { - endpointId: publication.endpointId, - publicationId: publication.id, - error: redactError(finalizationError), - }, - "could not durably finalize chat publication failure", - ); - } - }; - try { - const recordForLease = await endpointRecord(publication.endpointId); - if (!recordForLease) - throw new Error("Chat publication binding is unavailable"); - const currentPublicationRuntimeContext = - runtimeContextForRecord(recordForLease); - publicationRuntimeContext = currentPublicationRuntimeContext; - const [endpoint, conversation] = await Promise.all([ - db - .select() - .from(chatEndpoints) - .where(eq(chatEndpoints.id, publication.endpointId)) - .then((result) => result[0] ?? null), - db - .select() - .from(chatConversations) - .where(eq(chatConversations.id, publication.conversationId)) - .then((result) => result[0] ?? null), - ]); - if (!endpoint || !conversation) - throw new Error("Chat publication binding is unavailable"); - const providerVisibleCompletion = - conversation.state === "completed" && - !isExplicitOperatorPublication(publication) && - (await hasCommittedTaskControlCompletion(conversation.id)); - if (endpoint.status === "paused" || endpoint.status === "attention") { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "pending", - attempts: publication.attempts, - // Pause can race the eligibility read. The query excludes - // inactive endpoints on the next pass, so retain only the - // original provider deadline rather than delaying Resume. - nextAttemptAt: publication.nextAttemptAt, - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - // The setup conversation is a real end-to-end test: once provider - // credentials are verified, its safe agent response must be able to - // reach the provider before the operator confirms the final wizard - // step. Draft, paused, revoked, and archived endpoints remain closed. - if ( - !["active", "verifying"].includes(endpoint.status) || - ["unavailable", "endpoint_removed"].includes(conversation.state) || - providerVisibleCompletion - ) { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "cancelled", - redactedError: providerVisibleCompletion - ? "Conversation was completed before delivery" - : "External destination is no longer active", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - if (conversation.isDirectMessage) { - if (!endpoint.allowDirectMessages) { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "cancelled", - redactedError: "Direct messages are disabled in Paperclip", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - } else { - const resource = conversation.resourceId - ? await db - .select() - .from(chatEndpointResources) - .where( - and( - eq(chatEndpointResources.id, conversation.resourceId), - eq(chatEndpointResources.endpointId, endpoint.id), - eq(chatEndpointResources.companyId, endpoint.companyId), - ), - ) - .then((result) => result[0] ?? null) - : null; - if (!nonDirectDestinationAllowed(endpoint, resource)) { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "cancelled", - redactedError: "Destination is disabled in Paperclip", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - } - const interactionId = publication.payload.interactionId; - const isInteractionPrompt = - Boolean(interactionId) && - publication.idempotencyKey === - `interaction:${interactionId}:${publication.endpointId}`; - if (isInteractionPrompt && publication.issueId) { - const currentInteraction = await db - .select({ status: issueThreadInteractions.status }) - .from(issueThreadInteractions) - .where( - and( - eq(issueThreadInteractions.id, interactionId!), - eq(issueThreadInteractions.companyId, publication.companyId), - eq(issueThreadInteractions.issueId, publication.issueId), - ), - ) - .then((result) => result[0] ?? null); - if (!currentInteraction || currentInteraction.status !== "pending") { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "cancelled", - attempts: publication.attempts, - nextAttemptAt: null, - redactedError: "Interaction resolved before provider delivery", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - } - // Milestone selection and insertion can overlap across sweeps. An old - // snapshot inserted after newer attention must not overwrite it simply - // because it now occupies the next FIFO slot. - const progressSupersessionReason = - (await runOwnershipMilestoneSupersessionReason(publication)) ?? - ((await progressSupersededByFastResponse(publication)) ? "Acknowledged by contextual fast response" : null) ?? - ((await runProgressSupersededByPublishedInteraction(publication)) - ? "Run progress was superseded by its provider interaction" - : null); - if (progressSupersessionReason) { - await publicationLease.assertOwned(); - await db.transaction(async (tx) => { - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "cancelled", - attempts: publication.attempts, - nextAttemptAt: null, - redactedError: progressSupersessionReason, - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - }); - return; - } - // Status is sampled when it reaches the head of the provider lane, - // not when the command was admitted. If an already-streaming final - // publication won the race, this reply reflects Paperclip's latest - // authoritative task state after that earlier send commits. - let payload = await currentTaskControlPayload(publication); - await withCredentialMutationLease( - recordForLease.endpoint, - async (credentialGuard) => { - const credentialLease: CredentialMutationLeaseGuard = { - assertOwned: async (database = db) => { - await credentialGuard.assertOwned(database); - await publicationLease.assertOwned(database); - }, - }; - try { - if (await settleGitHubAutomaticPublication(publication, credentialLease)) return; - // A prior failure's provider ID may have been reused while this - // worker waited for the endpoint lane. Select against current - // outbound links only after owning that lane, before the final - // source authorization claim and provider I/O. - const closedProgress = await closeProgressToReplace( - publication, - endpoint, - ); - const replaceProviderMessageId = endpoint.provider !== "imessage-photon" && CAPABILITIES[endpoint.provider] - .messageEdits - ? (closedProgress?.providerMessageId ?? - (await interactionResolutionPublicationToReplace( - publication, - payload, - )) ?? - (await interactionPromptPublicationToReplace( - publication, - payload, - )) ?? - (await runPublicationToReplace(publication, payload)) ?? - (await inboundWakePublicationToReplace( - publication, - payload, - )) ?? - (await taskStatusPublicationToReplace(publication, payload))) - : null; - const authorizationClaim = - await claimPublicationTransportAuthorization({ - credentialLease, - endpoint: recordForLease.endpoint, - publication, - runtimeContext: currentPublicationRuntimeContext, - }); - if (!authorizationClaim) { - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - await notifyChatPublicationWork(tx); - await tx - .update(chatPublications) - .set({ - state: "cancelled", - nextAttemptAt: null, - redactedError: - "Task control requester or destination is no longer authorized", - updatedAt: new Date(), - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ); - await tx - .update(chatActions) - .set({ - status: "cancelled", - result: { - code: "task_control_authorization_changed", - }, - updatedAt: new Date(), - }) - .where( - and( - eq(chatActions.endpointId, publication.endpointId), - eq( - chatActions.providerActionId, - `task-control-authorization:${publication.id}`, - ), - eq(chatActions.status, "issued"), - ), - ); - await credentialLease.assertOwned(tx); - }); - return; - } - // The durable claim above is the authorization linearization - // point. Provider I/O runs without a database transaction or - // row lock. The renewable lease preserves credential/runtime - // identity until conditional settlement. - if (authorizationClaim.preparedText !== null) - payload = { ...payload, text: authorizationClaim.preparedText }; - const telegramDraft = - authorizationClaim.endpoint.provider === "telegram" && - authorizationClaim.conversation.isDirectMessage && - !replaceProviderMessageId && - !payload.transportPart && - !payload.card && - !payload.attachmentIds?.length && - shouldStreamSafePublicationText( - renderPublicationTransportText(payload), - ) && - (await hasTelegramStopSubscription( - authorizationClaim.endpoint, - currentPublicationRuntimeContext, - )) - ? await reserveTelegramPublicationDraft({ - endpoint: authorizationClaim.endpoint, - conversation: authorizationClaim.conversation, - publication, - text: renderPublicationTransportText(payload), - credentialLease, - runtimeContext: currentPublicationRuntimeContext, - }) - : null; - const sent = await postSafePublication({ - endpoint: authorizationClaim.endpoint, - conversation: authorizationClaim.conversation, - publication, - payload, - replaceProviderMessageId, - telegramDraftControl: telegramDraft?.control, - beforePhotonWrite: () => db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - if (!(await authorizeRetainedChatSourcePublication(tx, publication))) throw new PhotonError("rejected", "Publication source authorization changed"); - }), - onSlackFileUploadAccepted: async (receipt) => { - // uploadV2 has completed at this point. Mark acceptance - // before the durable callback so a local write failure is - // still quarantined and can never trigger an implicit upload. - providerAccepted = true; - slackFileReceiptActionId = await recordSlackFileUploadReceipt( - { - credentialLease, - endpoint: authorizationClaim.endpoint, - conversation: authorizationClaim.conversation, - publication, - receipt, - runtimeContext: currentPublicationRuntimeContext, - }, - ); - }, - }); - if (isTelegramDraftStopped(sent)) { - // Only the durable stop transaction can produce this outcome. - // It settled this exact publication; never fabricate a provider - // receipt, complete its task, or enqueue a retry from this path. - if (!telegramDraft) - throw new Error("Unexpected Telegram draft stop result"); - const stopped = await db - .select({ id: chatActions.id }) - .from(chatActions) - .innerJoin( - chatPublications, - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "cancelled"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ) - .where( - and( - eq(chatActions.id, telegramDraft.actionId), - eq(chatActions.status, "cancelled"), - sql`${chatActions.result}->>'phase' = 'stopped'`, - ), - ); - if (stopped.length !== 1) - throw new Error( - "Telegram draft stop was not durably committed", - ); - return; - } - providerAccepted = true; - // A resolved SDK call is not a usable delivery receipt. Teams, - // for example, returns an empty ID for an empty HTTP success - // body. Preserve uncertainty instead of publishing a broken - // message link or automatically sending the content again. - if ( - typeof sent?.id !== "string" || - !sent.id || - sent.id.length > 2048 || - sent.id.trim() !== sent.id || - /[\u0000-\u001f\u007f]/.test(sent.id) - ) { - throw new Error( - "Provider did not return a usable message receipt", - ); - } - const { authorizationActionId } = authorizationClaim; - const receiptRemovalActionIds: string[] = []; - await db.transaction(async (tx) => { - await credentialLease.assertOwned(tx); - const committedAt = new Date(); - await notifyChatPublicationWork(tx); - const [completedPublication] = await tx - .update(chatPublications) - .set({ - state: "published", - payload, - providerMessageId: sent.id, - publishedAt: committedAt, - redactedError: null, - updatedAt: committedAt, - }) - .where( - and( - eq(chatPublications.id, publication.id), - eq(chatPublications.state, "streaming"), - eq(chatPublications.attempts, publication.attempts + 1), - ), - ) - .returning({ id: chatPublications.id }); - if (!completedPublication) { - throw new Error( - "Chat publication ownership changed before commit", - ); - } - if (telegramDraft) { - const completedDraft = await tx - .update(chatActions) - .set({ - status: "processed", - result: { - phase: "published", - providerMessageId: sent.id, - }, - updatedAt: committedAt, - }) - .where( - and( - eq(chatActions.id, telegramDraft.actionId), - eq(chatActions.status, "processing"), - sql`${chatActions.result}->>'phase' = 'final_sending'`, - ), - ) - .returning({ id: chatActions.id }); - if (completedDraft.length !== 1) - throw new Error( - "Telegram final draft ownership changed before commit", - ); - } - if (slackFileReceiptActionId) { - const [completedReceipt] = await tx - .update(chatActions) - .set({ - status: "processed", - result: { - code: "slack_file_upload_identity_confirmed", - attempts: 1, - }, - updatedAt: committedAt, - }) - .where( - and( - eq(chatActions.id, slackFileReceiptActionId), - eq(chatActions.kind, "slack_file_upload_receipt"), - eq(chatActions.status, "received"), - sql`(${chatActions.payload}->>'publicationId')::uuid = ${publication.id}`, - sql`(${chatActions.payload}->>'publicationAttempt')::int = ${publication.attempts + 1}`, - ), - ) - .returning({ id: chatActions.id }); - if (!completedReceipt) { - throw new Error( - "Slack file upload receipt ownership changed before commit", - ); - } - } - if (slackFileReceiptActionId) { - await stageSlackFilePublicationSuccess(tx, { - committedAt, - endpoint: authorizationClaim.endpoint, - publication, - providerMessageId: sent.id, - runtimeContext: currentPublicationRuntimeContext, - stageSessionSync: - !isExplicitOperatorPublication(publication) && - !publication.idempotencyKey.startsWith("control:status:"), - }); - } else { - // A queue notice references its causal inbound comment for - // authorization, but is not an authored copy of that comment. - // Reactions/readers must not attribute bot status to the user. - const linkedCommentId = parseInboundWakePublicationKey( - publication.idempotencyKey, - ) - ? null - : publication.commentId; - const messageLinkInsert = tx.insert(chatMessageLinks).values({ - companyId: publication.companyId, - endpointId: publication.endpointId, - conversationId: publication.conversationId, - publicationId: publication.id, - commentId: linkedCommentId, - providerMessageId: sent.id, - direction: "outbound", - }); - if (replaceProviderMessageId) { - await messageLinkInsert.onConflictDoUpdate({ - target: [ - chatMessageLinks.endpointId, - chatMessageLinks.conversationId, - chatMessageLinks.providerMessageId, - ], - set: { - publicationId: publication.id, - commentId: linkedCommentId, - }, - }); - } else { - await messageLinkInsert.onConflictDoNothing(); - } - await tx - .update(chatEndpoints) - .set({ - lastPublicationAt: committedAt, - updatedAt: committedAt, - }) - .where( - and( - eq(chatEndpoints.id, authorizationClaim.endpoint.id), - inArray(chatEndpoints.status, ["verifying", "active"]), - sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer, 0) = ${currentPublicationRuntimeContext.generation}`, - ), - ); - } - if (authorizationActionId) { - const processedAuthorization = await tx - .update(chatActions) - .set({ - status: "processed", - result: { code: "task_control_authorized_and_sent" }, - updatedAt: committedAt, - }) - .where( - and( - eq(chatActions.id, authorizationActionId), - eq(chatActions.status, "processing"), - sql`(${chatActions.result}->>'attempts')::int = ${publication.attempts + 1}`, - ), - ) - .returning({ id: chatActions.id }); - if (!processedAuthorization.length) { - throw new Error( - "Task-control authorization ownership changed before commit", - ); - } - } - await commitTaskControlCompletion( - tx as unknown as Db, - publication, - committedAt, - ); - if ( - !slackFileReceiptActionId && - authorizationClaim.endpoint.provider === "slack" && - !isExplicitOperatorPublication(publication) && - !parseInboundWakePublicationKey(publication.idempotencyKey) && - !publication.idempotencyKey.startsWith("control:status:") - ) { - await stageSlackSessionSync(tx, { - companyId: publication.companyId, - endpointId: publication.endpointId, - conversationId: publication.conversationId, - runtimeGeneration: - currentPublicationRuntimeContext.generation, - credentialFingerprint: - currentPublicationRuntimeContext.credentialFingerprint, - }); - } - receiptRemovalActionIds.push( - ...(await stageTerminalReceiptReactionRemovals( - tx as unknown as Db, - { - endpoint: authorizationClaim.endpoint, - publication, - payload, - runtimeContext: currentPublicationRuntimeContext, - closedProgressRunId: closedProgress?.runId, - }, - )), - ); - await credentialLease.assertOwned(tx); - }); - // Receipt cleanup is non-critical provider I/O. Dispatch it - // after the terminal reply and durable action commit so a - // slow reaction endpoint cannot hold the publication lane. - // If this process stops first, the action outbox resumes the - // idempotent removal on a later reconciliation sweep. - for (const actionId of receiptRemovalActionIds) { - scheduleMessageProcessing(() => - processReceiptReaction(actionId), - ); - } - } catch (error) { - // Authentication, membership, and destination failures mutate - // endpoint/runtime state. Settle them before releasing the - // credential lease so a concurrent reconnect or provider - // recovery cannot be overwritten by this stale attempt. - failureHandledWithinCredentialLease = true; - // If ownership was already lost, leave the publication in its - // conservative streaming state for quarantine. Mutating health - // from this obsolete runtime would overwrite the new owner. - await settlePublicationFailure(error, credentialLease); - } - }, - ); - } catch (error) { - if (!failureHandledWithinCredentialLease) - await settlePublicationFailure(error); - } - }); - } - - function publicationEligibility({ now, busyEndpointIds = [], coolingTeamsPublicationIds = [], attemptedIds = [] }: { now?: Date; busyEndpointIds?: string[]; coolingTeamsPublicationIds?: string[]; attemptedIds?: string[] } = {}) { - const earlierPublication = alias(chatPublications, "earlier_chat_publications"); - return and( - sql`not (${unresolvedSlackPublicationCondition()})`, - // An explicit Board send promises both delivery and work. Keep - // its text/files pending until the durable wake has been accepted. - sql`not exists (select 1 from chat_actions a where a.company_id = ${chatPublications.companyId} and a.endpoint_id = ${chatPublications.endpointId} and a.conversation_id = ${chatPublications.conversationId} and a.kind = 'slack_board_message' and a.status = 'received' and a.payload->>'commentId' = ${chatPublications.commentId}::text)`, - or( - and( - sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`, - inArray(chatPublications.state, ["pending", "retry"]), - notExists( - db - .select({ id: chatTeamsFileTransfers.id }) - .from(chatTeamsFileTransfers) - .where( - and( - eq( - chatTeamsFileTransfers.companyId, - chatPublications.companyId, - ), - eq( - chatTeamsFileTransfers.publicationId, - chatPublications.id, - ), - ), - ), - ), - ), - // Dedicated protocol work shares the endpoint concurrency cap, - // never the ordinary publication claim or resend implementation. - sql`exists (${db - .select({ id: chatTeamsFileTransfers.id }) - .from(chatTeamsFileTransfers) - .where( - and( - eq( - chatTeamsFileTransfers.companyId, - chatPublications.companyId, - ), - eq( - chatTeamsFileTransfers.endpointId, - chatPublications.endpointId, - ), - eq( - chatTeamsFileTransfers.conversationId, - chatPublications.conversationId, - ), - eq( - chatTeamsFileTransfers.publicationId, - chatPublications.id, - ), - sql`exists (${db - .select({ id: chatEndpoints.id }) - .from(chatEndpoints) - .where( - and( - eq(chatEndpoints.id, chatPublications.endpointId), - eq( - chatEndpoints.companyId, - chatPublications.companyId, - ), - eq(chatEndpoints.provider, "microsoft-teams"), - eq(chatEndpoints.status, "active"), - ), - )})`, - or( - inArray(chatTeamsFileTransfers.phase, [ - "consent_pending", - "upload_pending", - "file_info_pending", - ]), - and( - eq(chatTeamsFileTransfers.phase, "consent_unknown"), - isNotNull(chatTeamsFileTransfers.responseActivityId), - sql`${chatTeamsFileTransfers.privateState}->'response' is not null`, - ), - ), - ), - )})`, - ), - busyEndpointIds.length > 0 - ? notInArray(chatPublications.endpointId, busyEndpointIds) - : undefined, - coolingTeamsPublicationIds.length > 0 - ? notInArray(chatPublications.id, coolingTeamsPublicationIds) - : undefined, - // Inactive endpoints must not consume the eligibility page or - // acquire an artificial retry deadline. When an operator resumes - // them, due work is immediately eligible; genuine provider retry - // deadlines below remain unchanged. - notExists( - db - .select({ id: chatEndpoints.id }) - .from(chatEndpoints) - .where( - and( - eq(chatEndpoints.id, chatPublications.endpointId), - inArray(chatEndpoints.status, ["paused", "attention"]), - ), - ), - ), - now ? or(isNull(chatPublications.nextAttemptAt), lte(chatPublications.nextAttemptAt, now)) : undefined, - attemptedIds.length > 0 - ? notInArray(chatPublications.id, attemptedIds) - : undefined, - notExists( - db - .select({ id: earlierPublication.id }) - .from(earlierPublication) - .where( - and( - eq( - earlierPublication.conversationId, - chatPublications.conversationId, - ), - publicationPrecedes(earlierPublication, chatPublications), - inArray(earlierPublication.state, [ - "pending", - "retry", - "streaming", - "delivery_unknown", - ]), - ), - ), - ), - ); - } - - async function processPendingPublications( - limit = 25, - { waitForCompletion = true, maintenance = true }: { waitForCompletion?: boolean; maintenance?: boolean } = {}, - ) { - if (shuttingDown) return 0; - if (maintenance) { - await enqueueInboundWakeupPublications(); - await reconcileTerminalConfirmationActions(limit); - } - const now = new Date(); - const staleBefore = new Date(now.getTime() - 60_000); - // A process that disappears after the provider accepted a post but before - // Paperclip persisted its message id leaves an ambiguous delivery. Never - // resend it automatically; an operator can explicitly replay after - // checking the provider conversation. - const quarantinedPublications = await db.transaction(async tx => { - const condition = and( - sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`, - eq(chatPublications.state, "streaming"), - lte(chatPublications.updatedAt, staleBefore), - // Teams owns separate staged I/O intents and a longer attempt lease. - // Only that protocol may recover/quarantine its in-flight stages. - notExists( - db - .select({ id: chatTeamsFileTransfers.id }) - .from(chatTeamsFileTransfers) - .where( - and( - eq( - chatTeamsFileTransfers.companyId, - chatPublications.companyId, - ), - eq(chatTeamsFileTransfers.publicationId, chatPublications.id), - ), - ), - ), - notExists( - db - .select({ id: chatEndpointLeases.id }) - .from(chatEndpointLeases) - .where( - and( - eq(chatEndpointLeases.companyId, chatPublications.companyId), - eq( - chatEndpointLeases.endpointId, - chatPublications.endpointId, - ), - sql`${chatEndpointLeases.leaseKey} = 'publication:' || ${chatPublications.id}::text || ':' || ${chatPublications.attempts}::text`, - gt(chatEndpointLeases.expiresAt, now), - ), - ), - ), - ); - const candidates = await tx.select({ id: chatPublications.id }).from(chatPublications).where(condition).for("update", { skipLocked: true }); - if (!candidates.length) return []; - await notifyChatPublicationWork(tx); - return tx - .update(chatPublications) - .set({ - state: "delivery_unknown", - nextAttemptAt: null, - redactedError: - "Provider delivery could not be confirmed after the worker stopped. Check the external conversation before replaying.", - updatedAt: now, - }) - .where( - and(condition, inArray(chatPublications.id, candidates.map(row => row.id))), - ) - .returning({ id: chatPublications.id }); - }); - if (quarantinedPublications.length > 0) { - await db - .update(chatActions) - .set({ - status: "delivery_unknown", - result: { - code: "task_control_delivery_unknown", - retryable: false, - }, - updatedAt: now, - }) - .where( - and( - eq(chatActions.kind, "task_control_authorization"), - eq(chatActions.status, "processing"), - inArray( - chatActions.providerActionId, - quarantinedPublications.map( - (publication) => `task-control-authorization:${publication.id}`, - ), - ), - ), - ); - } - const attemptedIds: string[] = []; - const ownedTasks = new Set>(); - let failed = false; - let firstError: unknown; - try { - while (attemptedIds.length < limit && !failed && !shuttingDown) { - if (publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY) { - if (!waitForCompletion) break; - await Promise.race(publicationEndpointTasks.values()); - continue; - } - const busyEndpointIds = [...publicationEndpointTasks.keys()]; - for (const [id, retryAfter] of teamsFileRetryAfter) - if (retryAfter <= Date.now()) teamsFileRetryAfter.delete(id); - const coolingTeamsPublicationIds = [...teamsFileRetryAfter.keys()]; - // Select only each conversation's current head before applying the - // global limit. Re-query after every batch so one invocation can still - // drain a conversation's newly unblocked milestones in FIFO order. - const rows = await db - .select() - .from(chatPublications) - .where( - publicationEligibility({ now, busyEndpointIds, coolingTeamsPublicationIds, attemptedIds }), - ) - .orderBy( - asc(chatPublications.createdAt), - asc(publicationTransportOrderKey(chatPublications)), - ) - .limit(limit - attemptedIds.length); - if (rows.length === 0) { - // The query excluded the busy-endpoint snapshot above. A worker - // may have completed while PostgreSQL selected this empty page, - // exposing a new head that must be queried before declaring idle. - if (busyEndpointIds.some((id) => !publicationEndpointTasks.has(id))) - continue; - if (!waitForCompletion || publicationEndpointTasks.size === 0) break; - await Promise.race(publicationEndpointTasks.values()); - continue; - } - let started = false; - for (const selectedPublication of rows) { - if ( - publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY || - attemptedIds.length >= limit || - failed || - shuttingDown - ) - break; - if (publicationEndpointTasks.has(selectedPublication.endpointId)) - continue; - attemptedIds.push(selectedPublication.id); - started = true; - // Reserve synchronously before provider work; overlapping drains share - // this cap. Different conversations for one bot keep the existing - // endpoint-exclusive credential fence instead of timing out behind it. - const task = Promise.resolve() - .then(async () => { - await processSelectedPublication(selectedPublication); - await settleExternalConversation(db, selectedPublication.companyId, selectedPublication.issueId).catch((err) => { - logger.warn({ err, publicationId: selectedPublication.id }, "Slack conversation settlement deferred to reconciliation"); - }); - }) - .catch((error: unknown) => { - if (!failed) { - failed = true; - firstError = error; - } - if (!waitForCompletion) { - logger.error( - { - endpointId: selectedPublication.endpointId, - publicationId: selectedPublication.id, - error: redactError(error), - }, - "scheduled chat publication worker failed", - ); - } - }) - .finally(() => { - if ( - publicationEndpointTasks.get(selectedPublication.endpointId) === - task - ) { - publicationEndpointTasks.delete(selectedPublication.endpointId); - } - ownedTasks.delete(task); - wakePublications(); - }); - publicationEndpointTasks.set(selectedPublication.endpointId, task); - ownedTasks.add(task); - } - // Re-query while slots remain: one busy endpoint must not fill the - // candidate page and hide later eligible endpoints behind its backlog. - if (!started && publicationEndpointTasks.size > 0) { - if (!waitForCompletion) break; - await Promise.race(publicationEndpointTasks.values()); - } - } - } catch (error) { - if (!failed) { - failed = true; - firstError = error; - } - } finally { - // Observe rejection immediately above and join every started worker - // before this invocation returns or propagates its original failure. - if (waitForCompletion) await Promise.all(ownedTasks); - } - if (failed) throw firstError; - if (maintenance) await processPendingInteractionWakeups(limit); - return attemptedIds.length; - } let githubMaintenancePending = false; function scheduleGitHubMaintenance() { @@ -18879,60 +7582,6 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { let wakePublications = () => {}; - async function nextInboundDeliveryAt(): Promise { - const rows = await db.select({ delivery: { - id: chatDeliveries.id, endpointId: chatDeliveries.endpointId, providerEventId: chatDeliveries.providerEventId, - state: chatDeliveries.state, eventKind: chatDeliveries.eventKind, - // Retain scheduling metadata only, not message bodies or attachment data. - normalizedEvent: sql`jsonb_build_object( - 'conversation', ${chatDeliveries.normalizedEvent}->'conversation', - 'lifecycle', ${chatDeliveries.normalizedEvent}->'lifecycle', - 'message', jsonb_build_object( - 'targetProviderEventId', ${chatDeliveries.normalizedEvent}->'message'->'targetProviderEventId', - 'admissionDependencyEventId', ${chatDeliveries.normalizedEvent}->'message'->'admissionDependencyEventId'))`, - }, at: sql`case - when ${chatDeliveries.state} = 'processing' then ${chatDeliveries.updatedAt} + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond' - when ${chatDeliveries.state} = 'processed' then ( - select min(case when a.status = 'processing' then a.updated_at + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond' - else coalesce((a.result->>'retryAt')::timestamptz, now()) end) - from chat_actions a where a.delivery_id = chat_deliveries.id - and a.company_id = chat_deliveries.company_id and a.endpoint_id = chat_deliveries.endpoint_id - and a.kind = 'inbound_wakeup' and a.status in ('issued', 'processing')) - else coalesce(${chatDeliveries.nextAttemptAt}, now()) end` }) - .from(chatDeliveries).where(and( - sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.provider = 'agentmail')`, - // Provider lifecycle receipts can restore an unavailable endpoint. - // The ordinary conversation drain still waits for operator resume. - or(sql`${chatDeliveries.normalizedEvent}->'lifecycle' is not null`, - sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.status in ('paused', 'attention'))`), - or(inArray(chatDeliveries.state, ["received", "retry", "processing"]), - and(eq(chatDeliveries.state, "processed"), pendingInboundWakeupCondition())), - )).orderBy(...conversationDeliveryOrder()); - // Only an ordered conversation head can run. Reaction and provider - // lifecycle receipts are independent; lifecycle edits may select their - // exact admission dependency before the chronologically earliest row. - const byEvent = new Map(rows.map(row => [`${row.delivery.endpointId}:${row.delivery.providerEventId}`, row])); - const conversations = new Set(); - let next: number | null = null; - for (let row of rows) { - const delivery = row.delivery; - const threadId = normalizedDeliveryThreadId(delivery); - if (threadId && !normalizedLifecycleEffect(delivery) && !["reaction_added", "reaction_removed"].includes(delivery.eventKind)) { - const key = conversationDrainKey(delivery.endpointId, threadId); - if (conversations.has(key)) continue; - conversations.add(key); - const dependency = normalizedLifecycleTargetEventId(delivery); - const target = dependency ? byEvent.get(`${delivery.endpointId}:${dependency}`) : undefined; - if (target && ["received", "retry", "processing"].includes(target.delivery.state)) row = target; - } - if (row.at !== null) { - const at = new Date(row.at).getTime(); - next = next === null ? at : Math.min(next, at); - } - } - return next; - } - async function nextPublicationAt(): Promise { const cooldowns = Object.fromEntries([...teamsFileRetryAfter].map(([id, at]) => [id, new Date(at).toISOString()])); const [queued, streaming] = await Promise.all([ @@ -18966,6 +7615,20 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { : null; } + const { configure, configureWithCredentialLease, runtimeConfig } = createChatConfiguration(db, { + endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime, + nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, + legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials, + persistCredentials, clearCredentials, resolveCredentials, prepareProviderInventory, + reconcileProviderResourceRows, reconcileDiscordCommands, runtimeFor, runtimeContextForRecord, + stageTelegramMaintenance, processTelegramMaintenance, recordTelegramStopSubscription, + fetchImpl, redactError, runtimeGeneration, leasedChatProvider, PROVIDER_LABELS, SUPPLIED_CREDENTIAL_KEYS, + PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, isSupportedTelegramWebhookBaseUrl, + get slackRegistration() { return slackRegistration; }, + get githubWizard() { return githubWizard; }, + get nativeBotIdentityClaimBarrier() { return options.nativeBotIdentityClaimBarrier; }, + get allowLocalVoiceBoard() { return options.allowLocalVoiceBoard; }, + }); const linkSlackAccount: SlackSetupOptions["linkInstaller"] = async (endpointId, user, actor, lease) => { const record = await endpointRecord(endpointId); if (!record || record.endpoint.provider !== "slack" || record.endpoint.providerAccountId !== user.team_id) throw conflict("Slack setup changed"); @@ -19058,6 +7721,79 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) { prepareProviderInventory, reconcileProviderResourceRows, listResources, replaceResources, update, configure, test, handleWebhook, get githubWizardOrigin() { return options.githubWizardOrigin; }, }); + const { prepareInboundEnvelope } = createInboundEnvelope(db, { runtimeFor, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages }); + const { admitInboundMessage } = createInboundAdmission(db, { runtimeGeneration, credentialFingerprint, leasedChatProvider, renewDiscordGatewayOwnershipForMessageAdmission, ensureResource, DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS }); + const { admitMessageResource } = createMessageResourceAdmission(db, { ensureResource }); + const messageLifecycle = createChatMessageLifecycle(db, { + DELIVERY_PROCESSING_STALE_MS, ORPHAN_FOLLOW_UP_GRACE_MS, ORPHAN_FOLLOW_UP_MAX_ATTEMPTS, + INGRESS_REORDER_WINDOW_MS, issuesSvc, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed, + redactError, runtimeCallbackEndpoint, sameTeamsThreadIdentity, durableExternalThreadIdentity, + recordCurrentMicrosoftTeamsRoute, conversationDrainKey, scheduleConversationDrain, scheduledConversationDrains, + drainConversationDeliveries, ensureDiscordGatewayRuntimeIsCurrent, + get deferWebhookProcessing() { return options.deferWebhookProcessing; }, + }); + const { recordLifecycleDelivery, processLifecycleDelivery, conversationForThread } = messageLifecycle; + const conversationDeliveryQueue = createChatConversationDeliveryQueue(db, { + DELIVERY_DRAIN_LIMIT, DELIVERY_LEASE_TTL_MS, discordRootThreadPending, reconcileDiscordRootThread, + durableExternalThreadIdentity, liveInboundMessages, processInboundWakeup, processLifecycleDelivery, processMessage, + redactError, runtimeContexts, runtimeFor, settleRejectedInboundWakeups, sanitizeFilename, + githubManualMessages, githubAutomaticMessages, githubIssueMessages, conversationDrainKey, + get conversationLeaseRenewalIntervalMs() { return options.conversationLeaseRenewalIntervalMs; }, + get renewConversationDeliveryLease() { return options.renewConversationDeliveryLease; }, + }); + const { nextInboundDeliveryAt } = conversationDeliveryQueue; + const { currentTaskControlPayload, hasCommittedTaskControlCompletion, commitTaskControlCompletion } = createChatPublicationTaskControl(db); + const publicationTransport = createChatPublicationTransport(db, { + endpointRecord, fetchImpl, getTaskBaseUrl, publicationFiles, receiptReactionCompletionRunId, + runIdFromMilestonePublication, runtime, runtimeFor, runtimeGeneration, CAPABILITIES, + voice, voiceStore, scheduleMessageProcessing, + }); + const { postSafePublication } = publicationTransport; + const { claimPublicationTransportAuthorization } = createChatPublicationAuthorization(db, { + authorizeInboundWakePublication, authorizeRetainedChatSourcePublication, fetchImpl, + githubCurrentOmissionPreparedText, hasCommittedTaskControlCompletion, lockCurrentPrincipalAuthorization, + nonDirectDestinationAllowed, resolveCredentials, runtimeCallbackEndpoint, FailedChatRetryPublicationReadError, + }); + const { finalizePublicationFailure } = createChatPublicationFailures(db, { + FailedChatRetryPublicationReadError, MAX_ERROR_TEXT, getTaskBaseUrl, invalidateRuntime, runtimeCallbackEndpoint, + }); + const { ensureDurablePublicationTransport } = createChatPublicationPreparation(db, { + runPublicationToReplace, + get publicationTransportPreparationBarrier() { return options.publicationTransportPreparationBarrier; }, + }); + const publicationWorker = createChatPublicationWorker(db, { + CAPABILITIES, authorizeRetainedChatSourcePublication, claimPublicationTransportAuthorization, + closeProgressToReplace, commitTaskControlCompletion, currentTaskControlPayload, endpointRecord, + ensureDurablePublicationTransport, finalizePublicationFailure, hasCommittedTaskControlCompletion, + hasTelegramStopSubscription, inboundWakePublicationToReplace, interactionPromptPublicationToReplace, + interactionResolutionPublicationToReplace, nonDirectDestinationAllowed, postSafePublication, + processReceiptReaction, progressSupersededByFastResponse, recordSlackFileUploadReceipt, redactError, + reserveTelegramPublicationDraft, runOwnershipMilestoneSupersessionReason, runProgressSupersededByPublishedInteraction, + runPublicationToReplace, runtimeContextForRecord, scheduleMessageProcessing, settleGitHubAutomaticPublication, + stageSlackFilePublicationSuccess, taskStatusPublicationToReplace, tryProcessTeamsFilePublication, + withCredentialMutationLease, receiptReactionCompletionRunId, + }); + const { processSelectedPublication } = publicationWorker; + const { processPendingInteractionWakeups } = createChatInteractionContinuations(db, { + redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, get heartbeat() { return options.heartbeat; }, + }); + const { handleAction, handleModalSubmit, reconcileTerminalConfirmationActions } = createChatInteractionCallbacks(db, { + CAPABILITIES, MAX_ERROR_TEXT, actionThreadMatchesConversation, nonDirectDestinationAllowed, sameTeamsThreadIdentity, + ensurePrincipal, endpointRecord, runtimeCallbackRecord, runtimeCallbackEndpoint, + requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, + recordCurrentMicrosoftTeamsRoute, lockTelegramCallbackNoticeAuthority, stageProviderEffect, + scheduleProviderEffect, invalidateRuntime, redactError, persistence, questionResponses, + scheduleMessageProcessing, processPendingPublications, + get confirmationResolutionPersistBarrier() { return options.confirmationResolutionPersistBarrier; }, + get questionFormOpenAuthorizationBarrier() { return options.questionFormOpenAuthorizationBarrier; }, + get questionResolutionPersistBarrier() { return options.questionResolutionPersistBarrier; }, + }); + const publicationQueue = createChatPublicationQueue(db, { + PUBLICATION_ENDPOINT_CONCURRENCY, enqueueInboundWakeupPublications, processPendingInteractionWakeups, + reconcileTerminalConfirmationActions, processSelectedPublication, publicationEndpointTasks, + teamsFileRetryAfter, redactError, isShuttingDown: () => shuttingDown, wakePublications: () => wakePublications(), + }); + const { publicationEligibility } = publicationQueue; const { unregisterSlackTaskAuthority } = createSlackTaskAuthorityRegistration(db, { authorizeInboundWakeup, resolveCredentials }); return { diff --git a/server/src/services/chat-channels/configuration.ts b/server/src/services/chat-channels/configuration.ts new file mode 100644 index 00000000000..b7cd9e4c22e --- /dev/null +++ b/server/src/services/chat-channels/configuration.ts @@ -0,0 +1,968 @@ +import { preparePhotonConfiguration, createPhotonConfiguration } from "./imessage-photon/setup.js"; +import { slackRuntimeConfig } from "./slack/setup.js"; +import { createTelegramSetup } from "./telegram/setup.js"; +import { createGitHubSetup } from "./github/setup.js"; +import { createSpekoSetup } from "./speko/setup.js"; +import { createGitHubAppSetup } from "./github/app-setup.js"; + +import type { VerifiedProviderIdentity } from "./runtime-types.js"; + +import { isProvisionedSlackSetup } from "@paperclipai/shared"; + +import { slackCallbackMatchesPublicUrl } from "./slack/callback-verification.js"; + +import { createDiscordCommands } from "./discord/commands.js"; + +import { createTelegramMaintenance } from "./telegram/maintenance.js"; +import type { EndpointRow, RuntimeContext, InternalSetupState } from "./runtime-types.js"; + +import { createChatResourceReconciliation } from "./resource-reconciliation.js"; + +import { notifyChatDeliveryWork, notifyChatEndpointWork } from "../chat-work-notifications.js"; + +import type { ChatEndpointRuntime } from "../voice/voice-runtime.js"; + +import { chatCredentialMutationLease, type CredentialMutationLeaseGuard } from "../chat-credential-mutation-lease.js"; + +import { slackChatRegistrationService } from "../chat-slack-registration.js"; + +import { + createHash, +} from "node:crypto"; +import { + and, + eq, + inArray, + isNull, + ne, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatConversations, + chatDeliveries, + chatEndpoints, + toolApplications, + toolConnections, +} from "@paperclipai/db"; +import type { + ChatEndpoint, + ChatEndpointSetupState, + ChatProvider, + ConfigureChatEndpointInput, +} from "@paperclipai/shared"; + +import { + MAX_ATTACHMENT_BYTES, +} from "../../attachment-types.js"; + +import { + badRequest, + conflict, + notFound, + unprocessable, +} from "../../errors.js"; + +import { + type ResolvedChatSdkProviderConfig, +} from "../chat-sdk-runtime.js"; +import { + logActivity, +} from "../activity-log.js"; + +import type { EndpointRecord } from "./runtime-types.js"; +import type { createChatCredentials } from "./credentials.js"; +type ChatConfigurationOptions = Pick, "normalizedCredentials" | "verifyCredentials" | "nativeBotIdentityMatches" | "legacyGitHubLabelsMatchVerifiedCredentials" | "assertNativeBotIdentityAvailable" | "isNativeBotIdentityUniqueViolation" | "nativeBotIdentityConflict" | "persistCredentials" | "clearCredentials" | "resolveCredentials" | "prepareProviderInventory"> & { + endpointRecord: (endpointId: string) => Promise; + get: (endpointId: string) => Promise; + withCredentialMutationLease: ReturnType; + getWebhookPublicBaseUrl: () => string | null; + invalidateRuntime: (endpointId: string) => Promise; + reconcileProviderResourceRows: ReturnType["reconcileProviderResourceRows"]; + reconcileDiscordCommands: ReturnType["reconcileDiscordCommands"]; + runtimeFor: (endpoint: EndpointRow, input?: { requireDiscordOwnership?: boolean; waitForDiscordOwnership?: boolean }) => Promise; + runtimeContextForRecord: (record: EndpointRecord) => RuntimeContext; + stageTelegramMaintenance: ReturnType["stageTelegramMaintenance"]; + processTelegramMaintenance: ReturnType["processTelegramMaintenance"]; + recordTelegramStopSubscription: ReturnType["recordTelegramStopSubscription"]; + fetchImpl: typeof globalThis.fetch; + redactError: (error: unknown) => string; + runtimeGeneration: (setup: ChatEndpointSetupState) => number; + leasedChatProvider: (provider: string) => boolean; + PROVIDER_LABELS: Record; + SUPPLIED_CREDENTIAL_KEYS: Record; + PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number; + isSupportedTelegramWebhookBaseUrl: (url: string | null) => boolean; + slackRegistration: Pick, "cleanup">; + githubWizard: Pick["githubWizard"], "repairCloud">; + nativeBotIdentityClaimBarrier?: () => Promise; + allowLocalVoiceBoard?: boolean; +}; +export function createChatConfiguration(db: Db, options: ChatConfigurationOptions) { + const { endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime, nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials, persistCredentials, clearCredentials, resolveCredentials, prepareProviderInventory, reconcileProviderResourceRows, reconcileDiscordCommands, runtimeFor, runtimeContextForRecord, stageTelegramMaintenance, processTelegramMaintenance, recordTelegramStopSubscription, fetchImpl, redactError, runtimeGeneration, leasedChatProvider, PROVIDER_LABELS, SUPPLIED_CREDENTIAL_KEYS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, isSupportedTelegramWebhookBaseUrl } = options; + const { persistPhotonConfiguration } = createPhotonConfiguration(db, { invalidateRuntime }); + const { registerTelegramWebhook } = createTelegramSetup(db, { fetchImpl, getWebhookPublicBaseUrl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, runtimeContextForRecord, recordTelegramStopSubscription, stageTelegramMaintenance, processTelegramMaintenance }); + const { repairGitHubCloudSetup, repairGitHubWebhookSetup } = createGitHubSetup(db, { fetchImpl, getWebhookPublicBaseUrl, get githubWizard() { return options.githubWizard; } }); + const { installSpekoVoiceTools, configureSpekoTools } = createSpekoSetup(db, { fetchImpl, getWebhookPublicBaseUrl, get allowLocalVoiceBoard() { return options.allowLocalVoiceBoard; } }); + + + + async function configure( + endpointId: string, + input: ConfigureChatEndpointInput, + actorUserId?: string | null, + ) { + const record = await endpointRecord(endpointId); + if (!record) throw notFound("Chat endpoint not found"); + if (record.endpoint.provider === "agentmail") throw badRequest("Use the email inbox API for AgentMail"); + if (input.photon && record.endpoint.provider !== "imessage-photon") throw badRequest("Photon configuration is only valid for iMessage Photon"); + const suppliedCredentialKeys = Object.keys(input.credentials ?? {}); + if (suppliedCredentialKeys.length > 0) { + const credentialAction = + input.action === "configure" || input.action === "reconnect"; + const allowedKeys = new Set( + credentialAction + ? SUPPLIED_CREDENTIAL_KEYS[record.endpoint.provider] + : [], + ); + const unsupportedKeys = suppliedCredentialKeys.filter( + (key) => !allowedKeys.has(key), + ); + if (unsupportedKeys.length > 0) { + throw unprocessable( + `Unsupported ${PROVIDER_LABELS[record.endpoint.provider]} credential fields for ${input.action}: ${unsupportedKeys.join(", ")}`, + { + code: "chat_endpoint_credentials_invalid", + provider: record.endpoint.provider, + action: input.action, + unsupportedKeys, + }, + ); + } + } + return withCredentialMutationLease(record.endpoint, (credentialLease) => + configureWithCredentialLease( + endpointId, + input, + credentialLease, + actorUserId, + ), + ); + } + + async function configureWithCredentialLease( + endpointId: string, + input: ConfigureChatEndpointInput, + credentialLease: CredentialMutationLeaseGuard, + actorUserId?: string | null, + ) { + const record = await endpointRecord(endpointId); + if (!record) throw notFound("Chat endpoint not found"); + const endpoint = record.endpoint; + if (input.action === "pause") { + if (endpoint.status !== "active") { + throw conflict("Only an active chat connection can be paused", { + code: "chat_endpoint_not_active", + }); + } + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const pausedAt = new Date(); + const current = await tx + .select({ setup: chatEndpoints.setup, status: chatEndpoints.status }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current || current.status !== "active") { + throw conflict("Only an active chat connection can be paused", { + code: "chat_endpoint_not_active", + }); + } + await notifyChatEndpointWork(tx); + await tx + .update(chatEndpoints) + .set({ + status: "paused", + setup: { + ...current.setup, + runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1), + } as InternalSetupState, + updatedAt: pausedAt, + }) + .where(eq(chatEndpoints.id, endpoint.id)); + await tx + .update(toolConnections) + .set({ status: "disabled", enabled: false, updatedAt: new Date() }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + nextAttemptAt: null, + redactedError: "Connection was paused before processing", + processedAt: pausedAt, + updatedAt: pausedAt, + }) + .where( + and( + endpoint.provider === "imessage-photon" ? sql`false` : undefined, + eq(chatDeliveries.endpointId, endpoint.id), + inArray(chatDeliveries.state, ["received", "retry"]), + ), + ); + await credentialLease.assertOwned(tx); + }); + await invalidateRuntime(endpoint.id); + await logActivity(db, { + companyId: endpoint.companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action: "chat_endpoint.paused", + entityType: "tool_connection", + entityId: endpoint.connectionId, + details: { endpointId: endpoint.id }, + }); + return get(endpoint.id); + } + if (input.action === "resume") { + if (endpoint.status !== "paused" || endpoint.setup.step !== "complete") { + throw conflict( + "Only a previously active chat connection can be resumed", + { + code: "chat_endpoint_not_resumable", + }, + ); + } + let credentials = await resolveCredentials(endpoint).catch(() => { + throw conflict("Reconnect the provider credentials before resuming", { + code: "chat_endpoint_credentials_missing", + }); + }); + const identity = await verifyCredentials(endpoint.provider, credentials); + const upgradingLegacyGitHubIdentity = + endpoint.provider === "github" && + !endpoint.botExternalId && + legacyGitHubLabelsMatchVerifiedCredentials(endpoint, identity); + if ( + !upgradingLegacyGitHubIdentity && + !nativeBotIdentityMatches(endpoint.provider, endpoint, identity) + ) { + throw conflict( + "The provider credentials now identify a different bot; reconnect this connection instead", + { + code: "chat_bot_identity_changed", + }, + ); + } + await assertNativeBotIdentityAvailable(endpoint, identity); + const prepared = await prepareProviderInventory(endpoint, credentials); + if ( + endpoint.provider === "github" && + prepared.credentials.installationId !== credentials.installationId + ) { + await persistCredentials( + endpoint, + prepared.credentials, + credentialLease, + actorUserId, + ); + } + credentials = prepared.credentials; + if (prepared.inventory) + await reconcileProviderResourceRows( + endpoint, + prepared.inventory, + credentialLease, + ); + let activatedEndpoint!: EndpointRow; + try { + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const resumedAt = new Date(); + const current = await tx + .select() + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current || current.status !== "paused") { + throw conflict( + "Only a previously active chat connection can be resumed", + { + code: "chat_endpoint_not_resumable", + }, + ); + } + // Fail closed for any legacy or racing delivery that remained open + // while this endpoint was paused. Resume must never execute traffic + // that Paperclip acknowledged during the inactive interval. + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + nextAttemptAt: null, + redactedError: "Connection was paused when this event arrived", + processedAt: resumedAt, + updatedAt: resumedAt, + }) + .where( + and( + endpoint.provider === "imessage-photon" ? sql`false` : undefined, + eq(chatDeliveries.endpointId, endpoint.id), + inArray(chatDeliveries.state, ["received", "retry"]), + ), + ); + await notifyChatEndpointWork(tx); + [activatedEndpoint] = await tx + .update(chatEndpoints) + .set({ + status: "active", + ...(upgradingLegacyGitHubIdentity + ? { botExternalId: identity.botExternalId } + : {}), + healthMessage: "Connected", + lastError: null, + setup: { + ...current.setup, + ...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: resumedAt.toISOString() } : {}), + runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1), + } as InternalSetupState, + updatedAt: resumedAt, + }) + .where(eq(chatEndpoints.id, endpoint.id)) + .returning(); + await tx + .update(toolConnections) + .set({ + status: "active", + enabled: true, + healthStatus: "healthy", + healthMessage: "Connected", + lastError: null, + healthCheckedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await credentialLease.assertOwned(tx); + }); + await invalidateRuntime(endpoint.id); + if (activatedEndpoint.provider === "discord") + await reconcileDiscordCommands(endpoint.id, credentialLease, true); + await runtimeFor(activatedEndpoint, { + requireDiscordOwnership: leasedChatProvider(activatedEndpoint.provider), + waitForDiscordOwnership: leasedChatProvider(activatedEndpoint.provider), + }); + } catch (error) { + await invalidateRuntime(endpoint.id).catch(() => undefined); + if (activatedEndpoint) { + const failure = redactError(error); + const failedAt = new Date(); + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await notifyChatEndpointWork(tx); + await tx + .update(chatEndpoints) + .set({ + status: "attention", + healthMessage: "Provider resume needs attention", + lastError: failure, + updatedAt: failedAt, + }) + .where( + and( + eq(chatEndpoints.id, endpoint.id), + eq(chatEndpoints.status, "active"), + sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer, 0) = ${runtimeGeneration(activatedEndpoint.setup)}`, + ), + ); + await tx + .update(toolConnections) + .set({ + status: "disabled", + enabled: false, + healthStatus: "degraded", + healthMessage: "Provider resume failed", + lastError: failure, + healthCheckedAt: failedAt, + updatedAt: failedAt, + }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + nextAttemptAt: null, + redactedError: "Connection activation failed", + processedAt: failedAt, + updatedAt: failedAt, + }) + .where( + and( + eq(chatDeliveries.endpointId, endpoint.id), + inArray(chatDeliveries.state, ["received", "retry"]), + ), + ); + await credentialLease.assertOwned(tx); + }); + } + throw error; + } + await logActivity(db, { + companyId: endpoint.companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action: "chat_endpoint.resumed", + entityType: "tool_connection", + entityId: endpoint.connectionId, + details: { endpointId: endpoint.id }, + }); + return get(endpoint.id); + } + if (input.action === "remove") { + if (endpoint.status === "archived") { + if (endpoint.provider === "slack") await options.slackRegistration.cleanup(endpoint.id, credentialLease); + // Archival is the durable ingress fence and intentionally commits + // before secret-store cleanup. If that cleanup failed, a repeated + // remove is the recovery operation; once refs are empty, retain the + // normal already-removed conflict contract. + if (record.credentialSecretRefs.length > 0) { + await invalidateRuntime(endpoint.id).catch(() => undefined); + if (endpoint.provider === "telegram") { + await credentialLease.assertOwned(); + const action = await stageTelegramMaintenance( + endpoint, + "remove_endpoint", + ); + if (action) + await processTelegramMaintenance( + action.id, + true, + credentialLease, + ); + } else { + await clearCredentials(endpoint, credentialLease); + } + return get(endpoint.id); + } + throw conflict("This chat connection has already been removed", { + code: "chat_endpoint_already_removed", + }); + } + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const current = await tx + .select({ setup: chatEndpoints.setup, status: chatEndpoints.status }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current || current.status === "archived") { + throw conflict("This chat connection has already been removed", { + code: "chat_endpoint_already_removed", + }); + } + await notifyChatEndpointWork(tx); + await tx + .update(chatEndpoints) + .set({ + status: "archived", + setup: { + ...current.setup, + runtimeGeneration: runtimeGeneration(current.setup) + 1, + } as InternalSetupState, + archivedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(chatEndpoints.id, endpoint.id)); + await tx + .update(chatConversations) + .set({ state: "endpoint_removed", updatedAt: new Date() }) + .where(eq(chatConversations.endpointId, endpoint.id)); + await tx + .update(toolConnections) + .set({ status: "archived", enabled: false, updatedAt: new Date() }) + .where(eq(toolConnections.id, endpoint.connectionId)); + const otherConnection = await tx + .select({ id: toolConnections.id }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.applicationId, record.applicationId), + ne(toolConnections.id, endpoint.connectionId), + ne(toolConnections.status, "archived"), + ), + ) + .limit(1) + .then((rows) => rows[0] ?? null); + if (!otherConnection) { + await tx + .update(toolApplications) + .set({ + status: "archived", + archivedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(toolApplications.id, record.applicationId)); + } + await credentialLease.assertOwned(tx); + }); + if (endpoint.provider === "slack") await options.slackRegistration.cleanup(endpoint.id, credentialLease); + await invalidateRuntime(endpoint.id).catch(() => undefined); + if (endpoint.provider === "telegram") { + await credentialLease.assertOwned(); + const action = await stageTelegramMaintenance( + endpoint, + "remove_endpoint", + ); + if (action) { + await processTelegramMaintenance(action.id, true, credentialLease); + } + } else { + await clearCredentials(endpoint, credentialLease); + } + return get(endpoint.id); + } + if ( + input.action !== "configure" && + input.action !== "reconnect" && + input.action !== "verify" + ) { + throw unprocessable("Unsupported chat endpoint setup action"); + } + if (!getWebhookPublicBaseUrl() && endpoint.provider !== "discord" && endpoint.provider !== "imessage-photon" && !(endpoint.provider === "github" && endpoint.setup.github?.cloudRegistrationId)) { + throw unprocessable( + `A public HTTPS Paperclip URL is required before connecting ${PROVIDER_LABELS[endpoint.provider]}`, + ); + } + if ( + endpoint.provider === "telegram" && + (input.action === "configure" || input.action === "reconnect") && + !isSupportedTelegramWebhookBaseUrl(getWebhookPublicBaseUrl()) + ) { + throw unprocessable( + "Telegram webhooks require PAPERCLIP_CHAT_WEBHOOK_PUBLIC_URL to use HTTPS on port 443, 80, 88, or 8443", + { + code: "chat_telegram_webhook_url_unsupported", + provider: "telegram", + supportedPorts: [443, 80, 88, 8443], + }, + ); + } + if (input.action === "verify") { + if ( + endpoint.provider !== "slack" || + endpoint.status !== "verifying" || + endpoint.setup.step !== "provider_setup" + ) { + throw conflict( + "Provider verification is not available at this setup step", + { + code: "chat_endpoint_invalid_setup_step", + }, + ); + } + if (!endpoint.setup.webhookVerifiedAt) { + throw conflict("Paperclip has not received a verified Slack callback yet", { + code: "chat_webhook_not_verified", + }); + } + } else if ( + input.action === "configure" && + !["draft", "attention", "revoked"].includes(endpoint.status) && + // GitHub installation discovery can advance the status before the + // wizard has configured its runtime. Finish that exact setup stage; + // configured connections still require the explicit reconnect flow. + !( + endpoint.provider === "github" && + endpoint.status === "verifying" && + endpoint.setup.step === "provider_setup" + ) + ) { + throw conflict( + "This connection is already configured; use its reconnect flow if credentials need repair", + { + code: "chat_endpoint_already_configured", + }, + ); + } else if ( + input.action === "reconnect" && + !["verifying", "active", "attention", "revoked", "paused"].includes( + endpoint.status, + ) + ) { + throw conflict( + "This chat connection does not currently need reconnecting", + { + code: "chat_endpoint_not_reconnectable", + }, + ); + } + if ( + endpoint.provider === "github" && + (input.action === "configure" || input.action === "reconnect") && + !endpoint.setup.webhookVerifiedAt + ) { + throw conflict( + "GitHub has not delivered a signed webhook ping for this secret yet", + { + code: "chat_webhook_not_verified", + }, + ); + } + + let verifiedCurrentIdentity: VerifiedProviderIdentity = endpoint; + if ( + input.action === "reconnect" && + endpoint.provider === "github" && + !endpoint.botExternalId + ) { + const storedCredentials = await resolveCredentials(endpoint).catch(() => { + throw conflict( + "Reconnect the existing GitHub App credentials before changing them", + { + code: "chat_bot_identity_unverifiable", + }, + ); + }); + const storedIdentity = await verifyCredentials( + "github", + storedCredentials, + ); + if ( + !legacyGitHubLabelsMatchVerifiedCredentials(endpoint, storedIdentity) + ) { + throw conflict( + "The stored credentials no longer verify the GitHub App that owns this connection", + { + code: "chat_bot_identity_unverifiable", + }, + ); + } + verifiedCurrentIdentity = storedIdentity; + } + + const previousSpekoSigningSecret = endpoint.provider === "speko" + ? (await resolveCredentials(endpoint).catch(() => ({} as Record))).signingSecret : undefined; + let credentials = + input.credentials && Object.keys(input.credentials).length > 0 + ? await normalizedCredentials(endpoint, input.credentials) + : await resolveCredentials(endpoint).catch(() => { + throw unprocessable( + `${PROVIDER_LABELS[endpoint.provider]} credentials are required`, + ); + }); + if (endpoint.provider === "imessage-photon") { + credentials = preparePhotonConfiguration(endpoint, input, credentials); + } + const identity = await verifyCredentials(endpoint.provider, credentials); + // Once setup has claimed a provider bot identity, every credential repair + // must prove that same identity before secrets can be replaced. A process + // interruption can leave the endpoint in `attention` after the claim but + // before credential persistence; a repeated first-setup request must not + // turn that recovery state into an identity-swap escape hatch. + if (input.action === "reconnect" || Boolean(endpoint.botExternalId)) { + if ( + !nativeBotIdentityMatches( + endpoint.provider, + verifiedCurrentIdentity, + identity, + ) + ) { + throw conflict( + "The provider credentials identify a different bot; create a new connection for that bot instead", + { code: "chat_bot_identity_changed" }, + ); + } + } + await assertNativeBotIdentityAvailable(endpoint, identity); + const prepared = await prepareProviderInventory(endpoint, credentials); + const credentialsChangedByDiscovery = + endpoint.provider === "github" && + prepared.credentials.installationId !== credentials.installationId; + credentials = prepared.credentials; + await options.nativeBotIdentityClaimBarrier?.(); + // Claim a fresh provider identity before any newly supplied secrets are + // persisted. The provider-global unique indexes are the multi-process + // arbitration boundary when two setup requests both pass the optimistic + // preflight. A process that stops after this small claim leaves a + // fail-closed attention endpoint that the same operator can reconnect; + // it never leaves credentials bound to the losing endpoint. + if (!endpoint.botExternalId) { + try { + const claimed = await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await notifyChatEndpointWork(tx); + const rows = await tx + .update(chatEndpoints) + .set({ + status: "attention", + providerAccountId: identity.providerAccountId ?? null, + providerAccountLabel: identity.providerAccountLabel ?? null, + botExternalId: identity.botExternalId ?? null, + botUsername: identity.botUsername ?? null, + botDisplayName: + identity.botLabel ?? + endpoint.botDisplayName ?? + record.assignedAgentName, + healthMessage: "Provider setup must be completed", + lastError: null, + updatedAt: new Date(), + }) + .where( + and( + eq(chatEndpoints.id, endpoint.id), + eq(chatEndpoints.status, endpoint.status), + isNull(chatEndpoints.botExternalId), + ), + ) + .returning({ id: chatEndpoints.id }); + await credentialLease.assertOwned(tx); + return rows; + }); + if (claimed.length !== 1) { + throw conflict("This chat connection changed during setup", { + code: "chat_endpoint_setup_changed", + }); + } + } catch (error) { + if (isNativeBotIdentityUniqueViolation(error)) { + throw nativeBotIdentityConflict(endpoint.provider); + } + throw error; + } + } + if (endpoint.provider === "imessage-photon") { + await persistPhotonConfiguration(endpoint, credentials, credentialLease); + } + if ( + (input.credentials && Object.keys(input.credentials).length > 0) || + credentialsChangedByDiscovery + ) + await persistCredentials( + endpoint, + credentials, + credentialLease, + actorUserId, + ); + if (prepared.inventory) + await reconcileProviderResourceRows( + endpoint, + prepared.inventory, + credentialLease, + ); + const updatedAt = new Date(); + const waitingForSlackConfiguration = + endpoint.provider === "slack" && + (input.action === "configure" || input.action === "reconnect"); + try { + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const current = await tx + .select({ setup: chatEndpoints.setup }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) throw notFound("Chat endpoint not found"); + if (endpoint.provider === "speko") { + await installSpekoVoiceTools(tx, endpoint, actorUserId); + } + const observedSlackUrl = (current.setup as InternalSetupState).slackCallbackSurfaces?.events?.url; + const preserveSlackVerification = isProvisionedSlackSetup(current.setup.slackSetupMethod) + && (current.setup as InternalSetupState).slackVerificationSigningFingerprint === createHash("sha256").update(credentials.signingSecret ?? "").digest("hex") + && Boolean(observedSlackUrl && slackCallbackMatchesPublicUrl(observedSlackUrl, `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/slack`)); + await notifyChatEndpointWork(tx); + await tx + .update(chatEndpoints) + .set({ + status: "verifying", + providerAccountId: identity.providerAccountId ?? null, + providerAccountLabel: identity.providerAccountLabel ?? null, + botExternalId: identity.botExternalId ?? null, + botUsername: identity.botUsername ?? null, + botDisplayName: + identity.botLabel ?? + endpoint.botDisplayName ?? + record.assignedAgentName, + healthMessage: waitingForSlackConfiguration + ? "Finish provider webhook configuration" + : "Waiting for a test conversation", + lastError: null, + lastEventAt: null, + setup: { + ...current.setup, + ...(endpoint.provider === "github" ? { github: { ...current.setup.github, stage: current.setup.github?.stage ?? "connect", appOwnerType: identity.githubOwnerType, appOwnerLogin: identity.githubOwnerLogin } } : {}), + step: waitingForSlackConfiguration ? "provider_setup" : "test", + ...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: (current.setup as InternalSetupState).photonIntakeAfter ?? updatedAt.toISOString() } : {}), + testStartedAt: waitingForSlackConfiguration + ? null + : updatedAt.toISOString(), + webhookVerifiedAt: waitingForSlackConfiguration + ? preserveSlackVerification ? (current.setup.webhookVerifiedAt ?? null) : null + : (current.setup.webhookVerifiedAt ?? null), + runtimeGeneration: runtimeGeneration(current.setup) + 1, + } as InternalSetupState, + updatedAt, + }) + .where(eq(chatEndpoints.id, endpoint.id)); + await tx + .update(toolConnections) + .set({ + status: "active", + enabled: true, + healthStatus: "healthy", + healthMessage: "Provider credentials verified", + healthCheckedAt: updatedAt, + updatedAt, + }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await tx + .update(toolApplications) + .set({ status: "active", archivedAt: null, updatedAt }) + .where(eq(toolApplications.id, record.applicationId)); + await credentialLease.assertOwned(tx); + }); + } catch (error) { + if (isNativeBotIdentityUniqueViolation(error)) { + throw nativeBotIdentityConflict(endpoint.provider); + } + throw error; + } + try { + const next = await endpointRecord(endpoint.id); + if (!next) throw notFound("Chat endpoint not found"); + await invalidateRuntime(endpoint.id); + if (endpoint.provider === "github" && input.action === "reconnect" && endpoint.setup.github?.cloudRegistrationId) { + await repairGitHubCloudSetup(endpoint, credentialLease, actorUserId); + } + if (endpoint.provider === "github" && input.action === "reconnect" && !endpoint.setup.github?.cloudRegistrationId) { + await repairGitHubWebhookSetup(endpoint, credentials, credentialLease, actorUserId); + } + if (next.endpoint.provider === "speko") { + await configureSpekoTools(endpoint, credentials, credentialLease, previousSpekoSigningSecret); + } + if (next.endpoint.provider === "discord") + await reconcileDiscordCommands(endpoint.id, credentialLease, true); + await runtimeFor(next.endpoint, { + requireDiscordOwnership: next.endpoint.provider === "discord", + waitForDiscordOwnership: next.endpoint.provider === "discord", + }); + + if (endpoint.provider === "telegram" && getWebhookPublicBaseUrl()) { + await registerTelegramWebhook(endpoint, input, credentials, credentialLease, next); + } + } catch (error) { + const failure = redactError(error); + await invalidateRuntime(endpoint.id).catch(() => undefined); + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await notifyChatEndpointWork(tx); + await tx + .update(chatEndpoints) + .set({ + status: "attention", + healthMessage: "Provider setup needs attention", + lastError: failure, + updatedAt: new Date(), + }) + .where(eq(chatEndpoints.id, endpoint.id)); + await tx + .update(toolConnections) + .set({ + healthStatus: "degraded", + healthMessage: "Provider setup failed", + lastError: failure, + healthCheckedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await credentialLease.assertOwned(tx); + }); + throw unprocessable(failure, { code: "chat_provider_setup_failed" }); + } + + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await logActivity(tx as unknown as Db, { + companyId: endpoint.companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action: + input.action === "reconnect" + ? "chat_endpoint.reconnected" + : "chat_endpoint.credentials_verified", + entityType: "tool_connection", + entityId: endpoint.connectionId, + details: { endpointId: endpoint.id, provider: endpoint.provider }, + }); + await credentialLease.assertOwned(tx); + }); + return get(endpoint.id); + } + + function runtimeConfig( + endpoint: EndpointRow, + userName: string, + credentials: Record, + ): ResolvedChatSdkProviderConfig { + if (endpoint.provider === "speko") throw conflict("Speko uses the voice runtime"); + if (endpoint.provider === "imessage-photon") return { + provider: "imessage-photon", userName, + intakeAfter: Date.parse(String((endpoint.setup as InternalSetupState).photonIntakeAfter ?? endpoint.setup.testStartedAt ?? endpoint.createdAt.toISOString())), + credentials: { allocation: credentials.allocation === "shared" ? "shared" : "dedicated", projectId: credentials.projectId, lineId: credentials.lineId, projectSecret: credentials.projectSecret, phoneNumber: endpoint.botExternalId! }, + }; + if (endpoint.provider === "slack") + return slackRuntimeConfig(endpoint, userName, credentials); + if (endpoint.provider === "telegram") + return { + provider: "telegram", + userName, + maxDownloadBytes: MAX_ATTACHMENT_BYTES, + credentials: { + botToken: credentials.botToken, + secretToken: credentials.webhookSecret, + }, + }; + if (endpoint.provider === "discord") + return { + provider: "discord", + userName, + credentials: { + applicationId: credentials.applicationId, + botToken: credentials.botToken, + guildId: credentials.guildId, + }, + }; + if (endpoint.provider === "microsoft-teams") + return { + provider: "microsoft-teams", + userName, + credentials: { + appId: credentials.clientId, + appPassword: credentials.clientSecret, + appTenantId: credentials.tenantId, + appType: "SingleTenant", + }, + }; + return { + provider: "github", + userName, + credentials: { + appId: credentials.appId, + privateKey: credentials.privateKey, + installationId: credentials.installationId + ? Number(credentials.installationId) + : undefined, + webhookSecret: credentials.webhookSecret, + }, + }; + } + return { configure, configureWithCredentialLease, runtimeConfig }; +} diff --git a/server/src/services/chat-channels/credentials.ts b/server/src/services/chat-channels/credentials.ts new file mode 100644 index 00000000000..09f6c4525b3 --- /dev/null +++ b/server/src/services/chat-channels/credentials.ts @@ -0,0 +1,507 @@ +import { prepareSlackInventory } from "./slack/inventory.js"; +import { prepareGitHubInventory } from "./github/inventory.js"; +import { prepareDiscordInventory } from "./discord/inventory.js"; +import type { VerifiedProviderIdentity } from "./runtime-types.js"; +import { verifySlackCredentials } from "./slack/credentials.js"; +import { verifyDiscordCredentials } from "./discord/credentials.js"; +import { verifyTelegramCredentials } from "./telegram/credentials.js"; +import { verifyGitHubCredentials } from "./github/credentials.js"; +import { verifySpekoCredentials } from "./speko/credentials.js"; +import { verifyTeamsCredentials } from "./microsoft-teams/credentials.js"; +import { createPhotonCredentialChecks } from "./imessage-photon/credentials.js"; + +import type { EndpointRow, DbOrTransaction } from "./runtime-types.js"; + +import { type CredentialMutationLeaseGuard } from "../chat-credential-mutation-lease.js"; + +import { photonSharedScope } from "../photon/cloud.js"; + +import { photonChannelConfigurationSchema } from "@paperclipai/shared"; + +import { + randomBytes, + randomUUID, +} from "node:crypto"; +import { + and, + eq, + inArray, + ne, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatEndpoints, + companySecretBindings, + toolConnections, +} from "@paperclipai/db"; +import type { + ChatProvider, + ToolCredentialSecretRef, +} from "@paperclipai/shared"; + +import { isUniqueViolation } from "../../db-errors.js"; + +import { + conflict, + notFound, + unprocessable, +} from "../../errors.js"; +import { logger } from "../../middleware/logger.js"; + +import { type ChatProviderInventoryResult } from "../chat-provider-inventory.js"; + +import { + normalizeMicrosoftTeamsCredentialIds, +} from "../chat-teams-credentials.js"; + +import { secretService } from "../secrets.js"; + +type ChatCredentialsOptions = { + secrets: ReturnType; + PROVIDER_LABELS: Record; + REQUIRED_CREDENTIALS: Record, readonly string[]>; + PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number; + fetchImpl: typeof globalThis.fetch; + redactError: (error: unknown) => string; + credentialFingerprint: (refs: ToolCredentialSecretRef[]) => string; + verifyPhotonCredentials: ReturnType["verifyPhotonCredentials"]; +}; +export function createChatCredentials(db: Db, options: ChatCredentialsOptions) { + const { secrets, PROVIDER_LABELS, REQUIRED_CREDENTIALS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, fetchImpl, redactError, credentialFingerprint, verifyPhotonCredentials } = options; + + + async function normalizedCredentials( + endpoint: EndpointRow, + supplied: Record | undefined, + ) { + const values = await resolveCredentials(endpoint).catch( + () => ({}) as Record, + ); + for (const [key, rawValue] of Object.entries(supplied ?? {})) { + if (typeof rawValue === "string" && rawValue.trim()) + values[key] = key === "privateKey" ? rawValue : rawValue.trim(); + } + if (endpoint.provider === "telegram" && !values.webhookSecret) { + values.webhookSecret = randomBytes(32).toString("hex"); + } + const required = + endpoint.provider === "github" + ? ["appId", "privateKey", "webhookSecret"] + : REQUIRED_CREDENTIALS[endpoint.provider]; + const missing = required.filter((key) => !values[key]); + if (missing.length > 0) { + throw unprocessable( + `Missing required ${PROVIDER_LABELS[endpoint.provider]} credentials: ${missing.join(", ")}`, + ); + } + return endpoint.provider === "microsoft-teams" + ? normalizeMicrosoftTeamsCredentialIds(values) + : values; + } + + async function verifyCredentials( + provider: ChatProvider, + credentials: Record, + ): Promise { + if (provider === "imessage-photon") return verifyPhotonCredentials(credentials); + if (provider === "slack") return verifySlackCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); + if (provider === "discord") return verifyDiscordCredentials(credentials, fetchImpl, redactError); + if (provider === "telegram") return verifyTelegramCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); + if (provider === "github") return verifyGitHubCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); + if (provider === "speko") return verifySpekoCredentials(credentials, fetchImpl); + return verifyTeamsCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS); + } + + function nativeBotIdentityKey( + provider: ChatProvider, + identity: VerifiedProviderIdentity, + ): string | null { + const account = identity.providerAccountId?.trim().toLowerCase(); + const bot = identity.botExternalId?.trim().toLowerCase(); + if (provider === "github" || provider === "discord") { + if (bot) return `${provider}:app:${bot}`; + return null; + } + if (!account || !bot) return null; + return `${provider}:${account}:${bot}`; + } + + function providerUsesGlobalApplicationIdentity( + provider: ChatProvider, + ): boolean { + return ( + provider === "github" || + provider === "discord" || + provider === "microsoft-teams" || provider === "imessage-photon" + ); + } + + function nativeBotIdentityMatches( + provider: ChatProvider, + current: VerifiedProviderIdentity, + incoming: VerifiedProviderIdentity, + ): boolean { + const currentKey = nativeBotIdentityKey(provider, current); + return ( + currentKey !== null && + currentKey === nativeBotIdentityKey(provider, incoming) + ); + } + + function legacyGitHubLabelsMatchVerifiedCredentials( + current: VerifiedProviderIdentity, + verified: VerifiedProviderIdentity, + ): boolean { + return ( + !current.botExternalId && + Boolean(verified.botExternalId) && + current.providerAccountId?.trim().toLowerCase() === + verified.providerAccountId?.trim().toLowerCase() && + current.botUsername?.trim().toLowerCase() === + verified.botUsername?.trim().toLowerCase() + ); + } + + async function assertNativeBotIdentityAvailable( + endpoint: EndpointRow, + identity: VerifiedProviderIdentity, + ): Promise { + const key = nativeBotIdentityKey(endpoint.provider, identity); + if (!key) { + throw unprocessable( + `${PROVIDER_LABELS[endpoint.provider]} did not return a stable native bot identity`, + ); + } + const candidates = await db + .select({ + id: chatEndpoints.id, + companyId: chatEndpoints.companyId, + assignedAgentId: chatEndpoints.assignedAgentId, + providerAccountId: chatEndpoints.providerAccountId, + botExternalId: chatEndpoints.botExternalId, + botUsername: chatEndpoints.botUsername, + }) + .from(chatEndpoints) + .where( + and( + providerUsesGlobalApplicationIdentity(endpoint.provider) + ? undefined + : eq(chatEndpoints.companyId, endpoint.companyId), + eq(chatEndpoints.provider, endpoint.provider), + ne(chatEndpoints.id, endpoint.id), + endpoint.provider === "imessage-photon" ? ne(chatEndpoints.status, "archived") : inArray(chatEndpoints.status, [ + "verifying", + "active", + "paused", + "attention", + ]), + ), + ); + const conflictEndpoint = candidates.find((candidate) => { + if (!providerUsesGlobalApplicationIdentity(endpoint.provider)) { + return nativeBotIdentityKey(endpoint.provider, candidate) === key; + } + const candidateAppId = candidate.botExternalId?.trim().toLowerCase(); + const incomingAppId = identity.botExternalId?.trim().toLowerCase(); + if (candidateAppId && incomingAppId) + return candidateAppId === incomingAppId; + return ( + candidate.providerAccountId?.trim().toLowerCase() === + identity.providerAccountId?.trim().toLowerCase() && + candidate.botUsername?.trim().toLowerCase() === + identity.botUsername?.trim().toLowerCase() + ); + }); + if (conflictEndpoint) { + throw conflict( + `This ${PROVIDER_LABELS[endpoint.provider]} bot already represents another Paperclip agent connection`, + conflictEndpoint.companyId === endpoint.companyId + ? { + code: "chat_bot_identity_in_use", + endpointId: conflictEndpoint.id, + assignedAgentId: conflictEndpoint.assignedAgentId, + } + : { code: "chat_bot_identity_in_use" }, + ); + } + } + + function isNativeBotIdentityUniqueViolation(error: unknown): boolean { + return ( + isUniqueViolation(error, "chat_endpoints_photon_number_uq") || + isUniqueViolation(error, "chat_endpoints_live_bot_external_uq") || + isUniqueViolation(error, "chat_endpoints_live_discord_bot_external_uq") || + isUniqueViolation( + error, + "chat_endpoints_live_global_app_bot_external_uq", + ) || + isUniqueViolation(error, "chat_endpoints_live_bot_username_uq") + ); + } + + function nativeBotIdentityConflict(provider: ChatProvider) { + return conflict( + `This ${PROVIDER_LABELS[provider]} bot already represents another Paperclip agent connection`, + { code: "chat_bot_identity_in_use" }, + ); + } + + async function persistCredentials( + endpoint: EndpointRow, + credentials: Record, + credentialLease: CredentialMutationLeaseGuard, + actorUserId?: string | null, + ) { + await credentialLease.assertOwned(); + const previousRefs = await db + .select({ refs: toolConnections.credentialSecretRefs }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.id, endpoint.connectionId), + ), + ) + .then((rows) => rows[0]?.refs ?? []); + const refs: ToolCredentialSecretRef[] = []; + try { + for (const [key, value] of Object.entries(credentials)) { + if (endpoint.provider === "imessage-photon" && key !== "projectSecret") continue; + await credentialLease.assertOwned(); + const suffix = randomUUID(); + const secret = await secrets.create( + endpoint.companyId, + { + name: `chat.${endpoint.provider}.${endpoint.id.slice(0, 8)}.${key}.${suffix.slice(0, 8)}`, + key: `CHAT_${endpoint.provider.replaceAll("-", "_")}_${endpoint.id.replaceAll("-", "_")}_${key}_${suffix}`.toUpperCase(), + provider: "local_encrypted", + managedMode: "paperclip_managed", + value, + description: `${PROVIDER_LABELS[endpoint.provider]} channel credential for endpoint ${endpoint.id}`, + }, + { userId: actorUserId ?? undefined }, + ); + refs.push({ + secretId: secret.id, + versionSelector: "latest", + configPath: `credentials.${key}`, + required: true, + label: key, + projectionClass: "unclassified", + }); + await credentialLease.assertOwned(); + } + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const current = await tx + .select({ refs: toolConnections.credentialSecretRefs }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.id, endpoint.connectionId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) throw notFound("Chat connection not found"); + if ( + credentialFingerprint(current.refs) !== + credentialFingerprint(previousRefs) + ) { + throw conflict("Chat credentials changed during rotation", { + code: "chat_endpoint_credentials_changed", + }); + } + await tx + .delete(companySecretBindings) + .where( + and( + eq(companySecretBindings.companyId, endpoint.companyId), + eq(companySecretBindings.targetType, "tool_connection"), + eq(companySecretBindings.targetId, endpoint.connectionId), + ), + ); + if (refs.length > 0) { + await tx.insert(companySecretBindings).values( + refs.map((ref) => ({ + companyId: endpoint.companyId, + secretId: ref.secretId, + targetType: "tool_connection", + targetId: endpoint.connectionId, + configPath: ref.configPath, + versionSelector: String(ref.versionSelector ?? "latest"), + required: ref.required, + label: ref.label ?? null, + projectionClass: ref.projectionClass ?? "unclassified", + projectionAllowlistKey: ref.projectionAllowlistKey ?? null, + })), + ); + } + await tx + .update(toolConnections) + .set({ credentialSecretRefs: refs, updatedAt: new Date() }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await credentialLease.assertOwned(tx); + }); + } catch (error) { + // A lost database acknowledgement can be ambiguous even when the + // transaction committed. Delete only secrets proven unbound after the + // failure; when that proof cannot be read, leak safely for later cleanup + // instead of deleting a possibly-authoritative credential. + const boundSecretIds = + refs.length === 0 + ? new Set() + : await db + .select({ secretId: companySecretBindings.secretId }) + .from(companySecretBindings) + .where( + and( + eq(companySecretBindings.companyId, endpoint.companyId), + inArray( + companySecretBindings.secretId, + refs.map((ref) => ref.secretId), + ), + ), + ) + .then((rows) => new Set(rows.map((row) => row.secretId))) + .catch(() => null); + if (boundSecretIds) { + await Promise.allSettled( + refs + .filter((ref) => !boundSecretIds.has(ref.secretId)) + .map((ref) => secrets.remove(ref.secretId)), + ); + } + throw error; + } + const currentIds = new Set(refs.map((ref) => ref.secretId)); + const retired = previousRefs.filter((ref) => !currentIds.has(ref.secretId)); + const results = await Promise.allSettled( + retired.map((ref) => secrets.remove(ref.secretId)), + ); + if (results.some((result) => result.status === "rejected")) { + logger.warn( + { endpointId: endpoint.id }, + "chat credential rotation left an unbound retired secret for later cleanup", + ); + } + return refs; + } + + async function clearCredentials( + endpoint: EndpointRow, + credentialLease: CredentialMutationLeaseGuard, + ) { + let refs: ToolCredentialSecretRef[] = []; + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const current = await tx + .select({ refs: toolConnections.credentialSecretRefs }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.id, endpoint.connectionId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) throw notFound("Chat connection not found"); + refs = current.refs; + await tx + .delete(companySecretBindings) + .where( + and( + eq(companySecretBindings.companyId, endpoint.companyId), + eq(companySecretBindings.targetType, "tool_connection"), + eq(companySecretBindings.targetId, endpoint.connectionId), + ), + ); + await tx + .update(toolConnections) + .set({ credentialSecretRefs: [], updatedAt: new Date() }) + .where(eq(toolConnections.id, endpoint.connectionId)); + await credentialLease.assertOwned(tx); + }); + const results = await Promise.allSettled( + refs.map((ref) => secrets.remove(ref.secretId)), + ); + if (results.some((result) => result.status === "rejected")) { + logger.warn( + { endpointId: endpoint.id }, + "chat endpoint removal left an unbound retired secret for later cleanup", + ); + } + } + + async function resolveCredentialRefs( + endpoint: EndpointRow, + refs: ToolCredentialSecretRef[], + database: DbOrTransaction = db, + ): Promise> { + const values: Record = {}; + for (const ref of refs) { + const key = ref.configPath.replace(/^credentials\./, ""); + values[key] = await (database === db ? secrets : secretService(database as Db)).resolveSecretValue( + endpoint.companyId, + ref.secretId, + ref.versionSelector ?? "latest", + { + consumerType: "tool_connection", + consumerId: endpoint.connectionId, + configPath: ref.configPath, + actorType: "system", + actorId: null, + }, + ); + } + return values; + } + + async function resolveCredentials( + endpoint: EndpointRow, + database: DbOrTransaction = db, + ): Promise> { + const connection = await database + .select({ refs: toolConnections.credentialSecretRefs, config: toolConnections.config }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.id, endpoint.connectionId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!connection) throw notFound("Chat connection not found"); + const values = await resolveCredentialRefs(endpoint, connection.refs, database); + if (endpoint.provider === "imessage-photon") { + const configuration = photonChannelConfigurationSchema.parse(connection.config.photon); + return { ...values, ...configuration, lineId: configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId }; + } + return values; + } + + async function prepareProviderInventory( + endpoint: EndpointRow, + credentials: Record, + ): Promise<{ + credentials: Record; + inventory: ChatProviderInventoryResult | null; + managementUrl?: string; + }> { + try { + if (endpoint.provider === "slack") return await prepareSlackInventory(credentials, fetchImpl); + if (endpoint.provider === "github") return await prepareGitHubInventory(credentials, fetchImpl); + if (endpoint.provider === "discord") return await prepareDiscordInventory(credentials, fetchImpl); + return { credentials, inventory: null }; + } catch (error) { + throw unprocessable(redactError(error), { + code: "chat_provider_inventory_failed", + }); + } + } + return { normalizedCredentials, verifyCredentials, nativeBotIdentityMatches, legacyGitHubLabelsMatchVerifiedCredentials, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, persistCredentials, clearCredentials, resolveCredentialRefs, resolveCredentials, prepareProviderInventory }; +} diff --git a/server/src/services/chat-channels/discord/publication-preparation.ts b/server/src/services/chat-channels/discord/publication-preparation.ts new file mode 100644 index 00000000000..6c9de395c89 --- /dev/null +++ b/server/src/services/chat-channels/discord/publication-preparation.ts @@ -0,0 +1,120 @@ +import { createChatPublicationReplacement } from "../publication-replacement.js"; +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; +import { + eq, + sql, +} from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { + chatPublications, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, +} from "@paperclipai/shared"; +import { + discordMarkdownRequiresAttachment, +} from "../../chat-discord.js"; + +type PublicationPreparationOptions = { + runPublicationToReplace: ReturnType["runPublicationToReplace"]; +}; +export function createDiscordPublicationPreparation(db: Db, options: PublicationPreparationOptions) { + const { runPublicationToReplace } = options; + async function prepareDiscordPublication(publication: typeof chatPublications.$inferSelect, persisted: SafeChatPublicationPayload): Promise { + if ( + persisted.transportPart || + persisted.card || + persisted.interactionId || + !discordMarkdownRequiresAttachment(persisted.text) + ) { + return publication; + } + + // A normal response can replace its run's working placeholder. Discord's + // adapter cannot add a file while editing, so first turn that placeholder + // into a durable hand-off and then publish the complete Markdown as its + // own ordered attachment row. Publications without a replacement lane + // remain one provider call: a short caption and the lossless file. + const replacesProgressMessage = Boolean( + await runPublicationToReplace(publication, persisted), + ); + return db.transaction(async (tx) => { + const current = await tx + .select() + .from(chatPublications) + .where(eq(chatPublications.id, publication.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) return publication; + const currentPayload = current.payload as SafeChatPublicationPayload; + if ( + currentPayload.transportPart || + currentPayload.card || + currentPayload.interactionId || + !["pending", "retry"].includes(current.state) || + !discordMarkdownRequiresAttachment(currentPayload.text) + ) { + return current; + } + + const batchId = current.id; + const updatedAt = new Date(); + const transportPart = ( + index: number, + count: number, + mode: "inline" | "discord_markdown_attachment", + ) => ({ + batchId, + count, + index, + mode, + orderKey: `${batchId}:${String(index).padStart(4, "0")}`, + }); + if (!replacesProgressMessage) { + const attachmentPayload: SafeChatPublicationPayload = { + ...currentPayload, + transportPart: transportPart(0, 1, "discord_markdown_attachment"), + }; + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: attachmentPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + return { ...current, payload: attachmentPayload, updatedAt }; + } + + const handoffPayload: SafeChatPublicationPayload = { + ...currentPayload, + text: "This response needs a separate attachment because it exceeds the message limit.", + transportPart: transportPart(0, 2, "inline"), + }; + delete handoffPayload.attachmentIds; + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: handoffPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + + const attachmentPayload: SafeChatPublicationPayload = { + ...currentPayload, + transportPart: transportPart(1, 2, "discord_markdown_attachment"), + }; + delete attachmentPayload.progressState; + await notifyChatPublicationWork(tx); + await tx.insert(chatPublications).values({ + companyId: current.companyId, + endpointId: current.endpointId, + conversationId: current.conversationId, + issueId: current.issueId, + commentId: current.commentId, + idempotencyKey: `discord-markdown-attachment:${current.id}`, + payload: attachmentPayload, + state: "pending", + createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, + updatedAt, + }); + return { ...current, payload: handoffPayload, updatedAt }; + }); + } + return { prepareDiscordPublication }; +} diff --git a/server/src/services/chat-channels/discord/question-forms.ts b/server/src/services/chat-channels/discord/question-forms.ts new file mode 100644 index 00000000000..fdf2d08a35b --- /dev/null +++ b/server/src/services/chat-channels/discord/question-forms.ts @@ -0,0 +1,189 @@ +import { createChatSdkStatePersistence } from "../sdk-state.js"; + +import type { DbTransaction, RuntimeContext, LifecycleRuntimeFence } from "../runtime-types.js"; + +import { + and, + eq, +} from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { + chatConversations, + chatExternalPrincipals, + chatPublications, + issueThreadInteractions, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, + AskUserQuestionsInteraction, +} from "@paperclipai/shared"; + +import { + forbidden, +} from "../../../errors.js"; +import { logger } from "../../../middleware/logger.js"; +import { + type ChatSdkCallbackEvent, +} from "../../chat-sdk-runtime.js"; +import type { + ChatSdkStatePersistence, +} from "../../chat-sdk-state.js"; + +import { + deleteDiscordQuestionFormCorrection, + retainDiscordQuestionFormCorrection, +} from "../../chat-discord-question-forms.js"; + +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; + +import { + isChatQuestionFormOpenActionId, + loadChatQuestionFormSubmissionToken, + resolveChatQuestionFormOpen, +} from "../../chat-question-forms.js"; +import type { + ModalResponse, + ModalSubmitEvent, +} from "chat"; +import type { EndpointRecord } from "../runtime-types.js"; + +import type { ChatQuestionFormValidationResult } from "../../chat-question-forms.js"; +type LoadedForm = NonNullable>>; +type DiscordFormActor = { principal: typeof chatExternalPrincipals.$inferSelect; userId: string }; +type DiscordFormCorrectionInput = { + record: EndpointRecord; event: ChatSdkCallbackEvent; runtimeContext: RuntimeContext; + conversation: typeof chatConversations.$inferSelect; principal: DiscordFormActor; loaded: LoadedForm; + originalPublication: typeof chatPublications.$inferSelect; providerMessageId: string; + interaction: AskUserQuestionsInteraction; + payload: SafeChatPublicationPayload; validation: Exclude; + deny: (code: string) => Promise; +}; +type DiscordQuestionFormsOptions = { + requireCurrentExternalActionAuthorization: (tx: DbTransaction, input: { conversationId: string; endpointId: string; expectedUserId: string; principalId: string; runtimeContext: LifecycleRuntimeFence }) => Promise; + isExternalActionAuthorizationChange: (error: unknown) => boolean; + persistence: ChatSdkStatePersistence; +}; +export function createDiscordQuestionForms(db: Db, options: DiscordQuestionFormsOptions) { + const { requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, persistence } = options; + const denialError = () => forbidden("This chat form is not a current Paperclip question"); + async function handleDiscordFormCorrection(input: DiscordFormCorrectionInput) { + const { record, event, runtimeContext, conversation, principal, loaded, originalPublication, providerMessageId, interaction, payload, validation, deny } = input; + + const openAction = payload.card?.actions?.find( + (action) => + action.type === "callback" && + isChatQuestionFormOpenActionId(action.actionId), + ); + const openActionId = + openAction?.type === "callback" ? openAction.actionId : null; + if (!openActionId) return deny("chat_modal_open_token_not_current"); + try { + return await db.transaction(async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + const pending = await tx + .select({ id: issueThreadInteractions.id }) + .from(issueThreadInteractions) + .where( + and( + eq( + issueThreadInteractions.companyId, + record.endpoint.companyId, + ), + eq(issueThreadInteractions.issueId, conversation.issueId), + eq(issueThreadInteractions.id, interaction.id), + eq(issueThreadInteractions.status, "pending"), + ), + ) + .for("update") + .then((rows) => rows[0]); + const resolved = pending + ? await resolveChatQuestionFormOpen(tx, { + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + conversationId: conversation.id, + interaction, + openActionId, + }) + : null; + if ( + !resolved || + resolved.submitActionId !== loaded.payload.formActionId || + resolved.publicationId !== originalPublication.id + ) + throw denialError(); + return retainDiscordQuestionFormCorrection( + createChatSdkStatePersistence(tx as unknown as Db), + { + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + }, + { + principalId: principal.principal.id, + userId: principal.userId!, + externalUserId: event.event.user.userId, + conversationId: conversation.id, + publicationId: originalPublication.id, + providerMessageId, + threadId: conversation.externalThreadId, + interactionId: interaction.id, + openActionId, + submitActionId: resolved.submitActionId, + parentExpiresAt: loaded.payload.expiresAt, + modal: resolved.modal, + values: event.event.values, + fieldErrors: validation.fieldErrors, + }, + ); + }); + } catch (error) { + if ( + isExternalActionAuthorizationChange(error) || + (error && + typeof error === "object" && + "status" in error && + error.status === 403) + ) + return deny("chat_modal_correction_not_authorized"); + throw error; + } + + } + async function acceptDiscordQuestionForm(input: { record: EndpointRecord; event: ChatSdkCallbackEvent; principal: DiscordFormActor; loaded: LoadedForm }): Promise { + const { record, event, principal, loaded } = input; + + try { + await deleteDiscordQuestionFormCorrection( + persistence, + { + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + }, + { + principalId: principal.principal.id, + userId: principal.userId!, + externalUserId: event.event.user.userId, + }, + loaded.payload.formActionId, + ); + } catch { + // The answer receipt is already committed. Cleanup is retryable on + // exact replay/expiry access and cannot relabel an accepted answer. + logger.warn( + { + endpointId: record.endpoint.id, + code: "discord_question_correction_cleanup_failed", + }, + "Discord correction draft cleanup will be retried", + ); + } + + return { action: "clear" }; + } + return { handleDiscordFormCorrection, acceptDiscordQuestionForm }; +} diff --git a/server/src/services/chat-channels/endpoints.ts b/server/src/services/chat-channels/endpoints.ts new file mode 100644 index 00000000000..842860e1036 --- /dev/null +++ b/server/src/services/chat-channels/endpoints.ts @@ -0,0 +1,476 @@ +import { slackSetupState } from "./slack/setup.js"; +import { assertChatExecutionDefaultsAccess } from "../chat-execution-defaults.js"; + +import type { EndpointRow } from "./runtime-types.js"; + +import { chatCredentialMutationLease } from "../chat-credential-mutation-lease.js"; + +import { slackChatRegistrationService, slackRegistrationProjection } from "../chat-slack-registration.js"; +import { chatSlackRegistrations } from "@paperclipai/db"; + +import { defaultSlackAppConfiguration } from "@paperclipai/shared"; +import { + randomBytes, + randomUUID, +} from "node:crypto"; +import { + and, + desc, + eq, + ne, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + agents, + chatEndpoints, + toolApplications, + toolConnections, +} from "@paperclipai/db"; +import type { + ChatAdapterCapabilities, + ChatEndpoint, + ChatProvider, + CreateChatEndpointInput, + UpdateChatEndpointInput, +} from "@paperclipai/shared"; +import { + isAgentStatusInvokable, +} from "@paperclipai/shared"; + +import { + badRequest, + conflict, + notFound, + unprocessable, +} from "../../errors.js"; + +import { + logActivity, +} from "../activity-log.js"; + +import type { EndpointRecord } from "./runtime-types.js"; +type ChatEndpointsOptions = { + endpointRecord: (endpointId: string) => Promise; + CAPABILITIES: Record; + PROVIDER_LABELS: Record; + withCredentialMutationLease: ReturnType; + getPublicBaseUrl: () => string | null; + getWebhookPublicBaseUrl: () => string | null; + slackRegistration: Pick, "cleanup" | "managerGrants">; +}; +export function createChatEndpoints(db: Db, options: ChatEndpointsOptions) { + const { endpointRecord, CAPABILITIES, PROVIDER_LABELS, withCredentialMutationLease, getPublicBaseUrl, getWebhookPublicBaseUrl } = options; + + +function iso(value: Date | null | undefined): string | null { + return value ? value.toISOString() : null; +} + +function providerSetupState( + endpoint: Pick< + EndpointRow, + | "provider" + | "providerAccountId" + | "publicId" + | "status" + | "setup" + | "botUsername" + >, + publicBaseUrl: string | null, + assignedAgentName?: string | null, +) { + const path = endpoint.provider === "speko" ? `/api/voice-webhooks/${endpoint.publicId}/tools` : `/api/chat-webhooks/${endpoint.publicId}/${endpoint.provider}`; + const webhookUrl = publicBaseUrl ? `${publicBaseUrl}${path}` : null; + const step = endpoint.status === "active" ? "complete" : endpoint.setup.step; + switch (endpoint.provider) { + case "speko": + return { step, providerUrl: "https://platform.speko.ai/", webhookUrl } as const; + case "imessage-photon": return { step, providerUrl: "https://photon.codes/", testStartedAt: endpoint.setup.testStartedAt } as const; + case "agentmail": return endpoint.setup; + case "slack": { + return slackSetupState({ endpoint, webhookUrl, step, assignedAgentName }); + } + case "github": + return { + step, + github: endpoint.setup.github, + testStartedAt: endpoint.setup.testStartedAt, + testSkipped: endpoint.setup.testSkipped, + authorizationUrl: "https://github.com/settings/apps/new", + providerUrl: "https://github.com/settings/installations", + webhookUrl, + webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null, + } as const; + case "discord": + return { + step, + authorizationUrl: "https://discord.com/developers/applications", + providerUrl: endpoint.providerAccountId + ? `https://discord.com/channels/${encodeURIComponent(endpoint.providerAccountId)}` + : "https://discord.com/channels/@me", + webhookUrl, + } as const; + case "microsoft-teams": + return { + step, + authorizationUrl: "https://dev.teams.microsoft.com/apps", + providerUrl: "https://teams.microsoft.com/", + messagingEndpoint: webhookUrl, + } as const; + case "telegram": + return { + step, + providerUrl: endpoint.botUsername + ? `https://t.me/${encodeURIComponent(endpoint.botUsername)}` + : "https://t.me/BotFather", + webhookUrl, + } as const; + } +} + + function serializeEndpoint( + row: Awaited>, + ): ChatEndpoint { + if (!row) throw notFound("Chat endpoint not found"); + const endpoint = row.endpoint; + return { + id: endpoint.id, + companyId: endpoint.companyId, + connectionId: endpoint.connectionId, + publicationMode: endpoint.publicationMode, + externalExecutionPolicy: endpoint.externalExecutionPolicy, + provider: endpoint.provider, + publicId: endpoint.publicId, + status: endpoint.status, + deploymentMode: endpoint.deploymentMode, + assignedAgentId: endpoint.assignedAgentId, + assignedAgentName: row.assignedAgentName, + sponsorUserId: endpoint.sponsorUserId, + providerAccountId: endpoint.providerAccountId, + providerAccountLabel: endpoint.providerAccountLabel, + botExternalId: endpoint.botExternalId, + botUsername: endpoint.botUsername, + botLabel: endpoint.botDisplayName ?? row.assignedAgentName, + botAvatarUrl: endpoint.botAvatarUrl, + communicationInstructions: endpoint.communicationInstructions, + executionDefaults: endpoint.executionDefaults, + ...(endpoint.provider === "imessage-photon" && endpoint.botExternalId ? { photonAllocation: endpoint.botExternalId.startsWith("photon-project:") ? "shared" as const : "dedicated" as const } : {}), + allowDirectMessages: endpoint.allowDirectMessages, + requireAtMention: endpoint.requireAtMention, + allowGroupChats: endpoint.allowGroupChats, + allowUnlinkedPeople: endpoint.allowUnlinkedPeople, + replyMode: "subscribed", + capabilities: endpoint.capabilities, + setup: { + ...providerSetupState( + endpoint, + getWebhookPublicBaseUrl(), + row.assignedAgentName, + ), + ...(endpoint.provider === "github" + ? { + webhookSecretConfigured: row.credentialSecretRefs.some( + (ref) => ref.configPath === "credentials.webhookSecret", + ), + } + : endpoint.provider === "speko" ? { + webhookSecretConfigured: row.credentialSecretRefs.some( + (ref) => ref.configPath === "credentials.signingSecret", + ), + } : {}), + }, + healthMessage: endpoint.healthMessage, + lastError: endpoint.lastError, + lastActivityAt: iso(endpoint.lastEventAt), + lastPublicationAt: iso(endpoint.lastPublicationAt), + activatedAt: iso(endpoint.activatedAt), + createdAt: endpoint.createdAt.toISOString(), + updatedAt: endpoint.updatedAt.toISOString(), + }; + } + + async function list(companyId: string) { + const rows = await db + .select({ + endpoint: chatEndpoints, + assignedAgentName: agents.name, + connectionName: toolConnections.name, + applicationId: toolConnections.applicationId, + credentialSecretRefs: toolConnections.credentialSecretRefs, + }) + .from(chatEndpoints) + .innerJoin( + agents, + and( + eq(agents.id, chatEndpoints.assignedAgentId), + eq(agents.companyId, chatEndpoints.companyId), + ), + ) + .innerJoin( + toolConnections, + and( + eq(toolConnections.id, chatEndpoints.connectionId), + eq(toolConnections.companyId, chatEndpoints.companyId), + ), + ) + .where( + and( + eq(chatEndpoints.companyId, companyId), + ne(chatEndpoints.status, "archived"), + ), + ) + .orderBy(desc(chatEndpoints.updatedAt)); + return rows.map((row) => serializeEndpoint(row)); + } + + async function get(endpointId: string) { + const result = serializeEndpoint(await endpointRecord(endpointId)); + if (result.provider === "slack") { + const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId)); + result.setup.slackRegistration = registration ? slackRegistrationProjection(registration) : undefined; + const publicOrigin = getPublicBaseUrl(); + result.setup.slackOAuthCallbackUri = publicOrigin ? `${publicOrigin}/api/chat-slack/oauth/callback` : null; + } + return result; + } + + async function create( + companyId: string, + input: CreateChatEndpointInput, + actorUserId?: string | null, + ) { + if ((input.provider as string) === "agentmail") throw badRequest("Use the email inbox setup API for AgentMail"); + if ((input.slackApp || input.slackSetupMethod) && input.provider !== "slack") throw unprocessable("Slack app details only apply to Slack connections"); + if (input.slackSetupMethod === "managed" && options.slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable"); + const agent = await db + .select({ id: agents.id, name: agents.name, status: agents.status }) + .from(agents) + .where( + and( + eq(agents.companyId, companyId), + eq(agents.id, input.assignedAgentId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!agent) + throw unprocessable("The selected agent does not belong to this company"); + if (!isAgentStatusInvokable(agent.status)) { + throw unprocessable( + "The selected agent must be active before it can be connected to chat", + { + code: "chat_agent_not_invokable", + }, + ); + } + + const endpointId = randomUUID(); + const publicId = randomBytes(32).toString("base64url"); + const applicationId = input.applicationId ?? randomUUID(); + const connectionId = randomUUID(); + const suffix = endpointId.slice(0, 8); + const name = ( + input.name?.trim() || `${PROVIDER_LABELS[input.provider]} — ${agent.name}` + ).slice(0, 145); + await db.transaction(async (tx) => { + if (input.applicationId) { + const application = await tx + .select({ + id: toolApplications.id, + applicationKey: toolApplications.applicationKey, + metadata: toolApplications.metadata, + status: toolApplications.status, + }) + .from(toolApplications) + .where( + and( + eq(toolApplications.companyId, companyId), + eq(toolApplications.id, input.applicationId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!application) throw notFound("App not found"); + const sourceTemplateKey = + typeof application.metadata.sourceTemplateKey === "string" + ? application.metadata.sourceTemplateKey + : null; + if ( + application.status === "archived" || + (application.applicationKey !== input.provider && + sourceTemplateKey !== input.provider) + ) { + throw unprocessable( + `The selected App is not the ${PROVIDER_LABELS[input.provider]} connector`, + ); + } + } else { + await tx.insert(toolApplications).values({ + id: applicationId, + companyId, + applicationKey: `chat:${input.provider}:${endpointId}`, + name: `${name} ${suffix}`, + description: `People talk to ${agent.name} through ${PROVIDER_LABELS[input.provider]}.`, + type: "chat", + status: "draft", + metadata: { sourceTemplateKey: input.provider, purpose: "channel" }, + ownerUserId: actorUserId ?? null, + }); + } + await tx.insert(toolConnections).values({ + id: connectionId, + companyId, + applicationId, + name, + uid: `chat-${input.provider}-${endpointId}`, + connectionKind: "managed", + connectionPurpose: "channel", + ownership: "customer", + transport: input.provider === "speko" ? "voice" : "chat_sdk", + authKind: "api_key", + credentialPolicy: "shared", + status: "draft", + enabled: false, + config: { provider: input.provider }, + transportConfig: {}, + createdByUserId: actorUserId ?? null, + }); + await tx.insert(chatEndpoints).values({ + id: endpointId, + companyId, + connectionId, + provider: input.provider, + publicId, + assignedAgentId: agent.id, + sponsorUserId: actorUserId ?? null, + // Group chat is a broad Teams reach grant rather than a discovered + // destination toggle. Keep it closed until the operator explicitly + // enables that surface, even when this process is running against a + // database created before the column default was hardened. + requireAtMention: input.provider === "slack", + allowGroupChats: !["microsoft-teams", "speko"].includes(input.provider), + allowUnlinkedPeople: !["slack", "imessage-photon", "speko"].includes(input.provider), + capabilities: CAPABILITIES[input.provider], + setup: { + step: "provider_setup", + ...(input.provider === "slack" + ? { slackApp: input.slackApp ?? defaultSlackAppConfiguration(agent.name), + command: (input.slackApp ?? defaultSlackAppConfiguration(agent.name)).command, slackSetupMethod: input.slackSetupMethod ?? options.slackRegistration.managerGrants.defaultMethod() } + : {}), + }, + }); + }); + await logActivity(db, { + companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action: "chat_endpoint.created", + entityType: "tool_connection", + entityId: connectionId, + details: { + endpointId, + provider: input.provider, + assignedAgentId: agent.id, + }, + }); + return get(endpointId); + } + + async function update( + endpointId: string, + input: UpdateChatEndpointInput, + actorUserId?: string | null, + ) { + const initial = await endpointRecord(endpointId); + if (!initial) throw notFound("Chat endpoint not found"); + if (initial.endpoint.provider === "agentmail" && Object.keys(input).some((key) => key !== "executionDefaults")) throw badRequest("Use the email inbox API for AgentMail"); + await withCredentialMutationLease( + initial.endpoint, + async (credentialLease) => { + // Reach changes share the provider-transport lane with publications. + // Once a disable returns, no send that sampled the previous grant can + // still begin or finish behind it. + const existing = await endpointRecord(endpointId); + if (!existing) throw notFound("Chat endpoint not found"); + const values: Partial = { + updatedAt: new Date(), + }; + if (input.communicationInstructions !== undefined) { + if (existing.endpoint.provider !== "slack") { + throw unprocessable("Communication instructions are currently supported for Slack connections"); + } + values.communicationInstructions = input.communicationInstructions; + } + if (input.slackSetupMethod === "managed" && options.slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable"); + if (input.slackSetupMethod && (existing.endpoint.provider !== "slack" || existing.endpoint.botExternalId || existing.endpoint.status !== "draft")) { + throw conflict("Setup method can only change before connecting the app"); + } + if (input.slackApp || input.slackSetupMethod) { + const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId)); + if (registration && (registration.managerGrantId || registration.status !== "failed")) { + if (registration.managerGrantId || input.slackSetupMethod === "managed" || input.slackApp || input.slackSetupMethod === "automatic") throw conflict("Slack app details are locked after creation starts"); + // Manual recovery fences pending authorization under the same credential lease. + await options.slackRegistration.cleanup(endpointId, credentialLease); + } + } + if (input.slackSetupMethod) { + values.setup = { ...existing.endpoint.setup, slackSetupMethod: input.slackSetupMethod }; + } + if (input.slackApp) { + if (existing.endpoint.provider !== "slack") { + throw unprocessable("Slack app details only apply to Slack connections"); + } + if (existing.endpoint.status !== "draft" || existing.endpoint.botExternalId) { + throw conflict("Slack app details can only be edited before connecting the app"); + } + values.setup = { + ...(values.setup ?? existing.endpoint.setup), + slackApp: input.slackApp, + command: input.slackApp.command, + }; + } + if (input.allowDirectMessages !== undefined) + values.allowDirectMessages = input.allowDirectMessages; + if (input.requireAtMention !== undefined) { + if (existing.endpoint.provider !== "slack") + throw unprocessable("Require at-mention only applies to Slack connections"); + values.requireAtMention = input.requireAtMention; + } + if (input.allowGroupChats && existing.endpoint.provider === "imessage-photon" && (!existing.endpoint.botExternalId || existing.endpoint.botExternalId.startsWith("photon-project:"))) + throw unprocessable("Photon shared channels support direct messages only; groups require a dedicated channel"); + if (input.allowGroupChats !== undefined) + values.allowGroupChats = input.allowGroupChats; + if (input.allowUnlinkedPeople !== undefined) + values.allowUnlinkedPeople = input.allowUnlinkedPeople; + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + if (input.executionDefaults !== undefined) { + await assertChatExecutionDefaultsAccess(db, { + companyId: existing.endpoint.companyId, + assigneeAgentId: values.assignedAgentId ?? existing.endpoint.assignedAgentId, + actor: { type: "board", userId: actorUserId, source: actorUserId ? "session" : "local_implicit" }, + defaults: input.executionDefaults, + }, tx); + values.executionDefaults = input.executionDefaults; + } + await tx + .update(chatEndpoints) + .set(values) + .where(eq(chatEndpoints.id, endpointId)); + await logActivity(tx as unknown as Db, { + companyId: existing.endpoint.companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action: "chat_endpoint.updated", + entityType: "tool_connection", + entityId: existing.endpoint.connectionId, + details: { endpointId, fields: Object.keys(input) }, + }); + await credentialLease.assertOwned(tx); + }); + }, + ); + return get(endpointId); + } + return { list, get, create, update }; +} diff --git a/server/src/services/chat-channels/github/message-provenance.ts b/server/src/services/chat-channels/github/message-provenance.ts new file mode 100644 index 00000000000..5ce5d4dcbdd --- /dev/null +++ b/server/src/services/chat-channels/github/message-provenance.ts @@ -0,0 +1,32 @@ +import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js"; +import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared"; + +import { + chatDeliveries, +} from "@paperclipai/db"; + +import type { + Message, +} from "chat"; + +type GitHubMessageProvenanceOptions = { + githubManualMessages: WeakMap; + githubAutomaticMessages: WeakMap; + githubIssueMessages: WeakMap; +}; +export function createGitHubMessageProvenance(options: GitHubMessageProvenanceOptions) { + const { githubManualMessages, githubAutomaticMessages, githubIssueMessages } = options; + function restoreGitHubMessageProvenance(delivery: typeof chatDeliveries.$inferSelect, endpointRuntime: Pick, message: Message): void { +const githubManual = delivery.normalizedEvent.githubManual; + if (endpointRuntime.provider === "github" && githubManual && typeof githubManual === "object") githubManualMessages.set(message, githubManual as { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }); + const githubAutomatic = delivery.normalizedEvent.githubAutomatic; + if (endpointRuntime.provider === "github" && githubAutomatic && typeof githubAutomatic === "object") { + githubAutomaticMessages.set(message, githubAutomatic as { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }); + } + const githubIssue = delivery.normalizedEvent.githubIssue; + if (endpointRuntime.provider === "github" && githubIssue && typeof githubIssue === "object") { + githubIssueMessages.set(message, githubIssue as { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }); + } + } + return { restoreGitHubMessageProvenance }; +} diff --git a/server/src/services/chat-channels/github/setup.ts b/server/src/services/chat-channels/github/setup.ts new file mode 100644 index 00000000000..54f21322fd3 --- /dev/null +++ b/server/src/services/chat-channels/github/setup.ts @@ -0,0 +1,69 @@ +import { createGitHubAppSetup } from "./app-setup.js"; + +import type { EndpointRow } from "../runtime-types.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import { githubAppJwt } from "../../chat-github-client.js"; + +import type { Db } from "@paperclipai/db"; + +import { + forbidden, +} from "../../../errors.js"; + +import { + logActivity, +} from "../../activity-log.js"; +import { + resyncGitHubAppWebhook, +} from "../../chat-github-webhook-config.js"; + +type GitHubSetupOptions = { + fetchImpl: typeof globalThis.fetch; + getWebhookPublicBaseUrl: () => string | null; + githubWizard: Pick["githubWizard"], "repairCloud">; +}; +export function createGitHubSetup(db: Db, options: GitHubSetupOptions) { + const { fetchImpl, getWebhookPublicBaseUrl } = options; + + async function repairGitHubCloudSetup(endpoint: EndpointRow, credentialLease: CredentialMutationLeaseGuard, actorUserId?: string | null) { + if (!actorUserId) throw forbidden("A configuring member is required to recover this App"); + await credentialLease.assertOwned(); + await options.githubWizard.repairCloud(endpoint.id, actorUserId); + await credentialLease.assertOwned(); + } + + async function repairGitHubWebhookSetup(endpoint: EndpointRow, credentials: Record, credentialLease: CredentialMutationLeaseGuard, actorUserId?: string | null) { + // The immutable App and its installation were verified above. Repair + // only its callback using the stored secret; never reinstall it or + // change repository access. Keep historical signed-ping evidence, but + // require the fresh test round trip established above before activation. + const auditWebhookSync = async (action: string) => { + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await logActivity(tx as unknown as Db, { + companyId: endpoint.companyId, + actorType: "user", + actorId: actorUserId ?? "board", + action, + entityType: "tool_connection", + entityId: endpoint.connectionId, + details: { endpointId: endpoint.id, provider: "github" }, + }); + await credentialLease.assertOwned(tx); + }); + }; + await auditWebhookSync("chat_endpoint.webhook_sync_started"); + await credentialLease.assertOwned(); + await resyncGitHubAppWebhook({ + fetch: fetchImpl, + appToken: githubAppJwt(credentials.appId, credentials.privateKey), + webhookUrl: `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/github`, + webhookSecret: credentials.webhookSecret, + }); + await auditWebhookSync("chat_endpoint.webhook_synced"); + } + + return { repairGitHubCloudSetup, repairGitHubWebhookSetup }; +} diff --git a/server/src/services/chat-channels/imessage-photon/destination-admission.ts b/server/src/services/chat-channels/imessage-photon/destination-admission.ts new file mode 100644 index 00000000000..46c2f25040d --- /dev/null +++ b/server/src/services/chat-channels/imessage-photon/destination-admission.ts @@ -0,0 +1,29 @@ +import type { Thread } from "chat"; + +import { chatEndpointResources } from "@paperclipai/db"; +import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js"; + +type DestinationAdmissionInput = { + endpoint: EndpointRow; + currentEndpoint: Pick; + thread: Thread; + +}; +type DestinationAdmissionOptions = { + ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise; +}; +export function createPhotonDestinationAdmission(options: DestinationAdmissionOptions) { + const { ensureResource } = options; + async function admitPhotonDestination(input: DestinationAdmissionInput, tx: DbTransaction) { + const { endpoint, currentEndpoint, thread } = input; + let destinationAccepting = true; + + const resource = await ensureResource(endpoint, thread, false, tx); + destinationAccepting = + resource.enabled && + resource.availability === "available" && + currentEndpoint.allowGroupChats; + return destinationAccepting; + } + return { admitPhotonDestination }; +} diff --git a/server/src/services/chat-channels/imessage-photon/publications.ts b/server/src/services/chat-channels/imessage-photon/publications.ts index d85b4410cb5..bef81fd99e7 100644 --- a/server/src/services/chat-channels/imessage-photon/publications.ts +++ b/server/src/services/chat-channels/imessage-photon/publications.ts @@ -1,4 +1,285 @@ -import type { SafeChatPublicationPayload } from "@paperclipai/shared"; +import { createChatPublicationReplacement } from "../publication-replacement.js"; + +import { type ChatRuntime } from "../../chat-runtime.js"; +import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js"; + +import { nativePhotonInteraction, publishPhotonPrompt, type PhotonPromptReceipt, type PhotonInteractionBinding } from "../../photon/interactions.js"; +import { PhotonError } from "../../photon/cloud.js"; +import { PhotonChatAdapter } from "../../photon/adapter.js"; +import { + and, + desc, + eq, + or, + sql, +} from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { + chatActions, + chatEndpointResources, + chatMessageLinks, + heartbeatRuns, + issueThreadInteractions, +} from "@paperclipai/db"; +import type { + ChatEndpointSetupState, + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { + type ChatSdkRuntime, +} from "../../chat-sdk-runtime.js"; +import { safeChatTaskUrl } from "../../chat-task-url.js"; + +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; + +import type { + FileUpload, +} from "chat"; + +import type { EndpointRecord } from "../runtime-types.js"; + +import type { PublicationTransportInput } from "../publications/types.js"; +import type { Thread } from "chat"; +type PhotonPublicationDeliveryOptions = { + endpointRecord: (endpointId: string) => Promise; + runtimeGeneration: (setup: ChatEndpointSetupState) => number; + runtime: Pick; + receiptReactionCompletionRunId: ReturnType["receiptReactionCompletionRunId"]; + getTaskBaseUrl: () => string | null | undefined; +}; +export function createPhotonPublicationDelivery(db: Db, options: PhotonPublicationDeliveryOptions) { + const { endpointRecord, runtimeGeneration, runtime, receiptReactionCompletionRunId, getTaskBaseUrl } = options; + async function deliverPhotonPublication(input: PublicationTransportInput, endpointRuntime: ChatEndpointRuntime, thread: Pick, text: string, files: FileUpload[]) { + const adapter = endpointRuntime.getProviderAdapter(); + if (!(adapter instanceof PhotonChatAdapter)) + throw new Error("Photon publication runtime unavailable"); + const assertCurrent = async () => { + await input.beforePhotonWrite?.(); + const current = await endpointRecord(input.endpoint.id); + if ( + !current || + !["verifying", "active"].includes(current.endpoint.status) || + runtimeGeneration(current.endpoint.setup) !== + runtimeGeneration(input.endpoint.setup) || + runtime.get(input.endpoint.id) !== endpointRuntime + ) + throw new PhotonError( + "rejected", + "Photon publication authority changed", + ); + const resource = await db + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.companyId, current.endpoint.companyId), + eq(chatEndpointResources.endpointId, current.endpoint.id), + eq(chatEndpointResources.id, input.conversation.resourceId!), + ), + ) + .then((rows) => rows[0]); + if ( + resource?.availability !== "available" || + (!input.conversation.isDirectMessage && !resource.enabled) || + (input.conversation.isDirectMessage && + !current.endpoint.allowDirectMessages) + ) + throw new PhotonError( + "rejected", + "Photon conversation is no longer enabled", + ); + }; + const retry = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.endpointId, input.endpoint.id), + eq(chatActions.kind, "photon_publication_retry"), + eq( + chatActions.providerActionId, + `photon-retry:${input.publication.id}:${input.publication.attempts + 1}`, + ), + eq(chatActions.status, "processed"), + ), + ) + .then((rows) => rows[0]); + const retryUnknown = Boolean(retry); + const nextQuestion = + /^photon-question:([a-zA-Z0-9_-]{8,24}):(\d{1,2})$/.exec( + input.publication.idempotencyKey, + ); + if ( + input.payload.interactionId && + (input.publication.idempotencyKey === + `interaction:${input.payload.interactionId}:${input.endpoint.id}` || + nextQuestion) + ) { + const action = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, input.endpoint.companyId), + eq(chatActions.endpointId, input.endpoint.id), + eq(chatActions.kind, "photon_interaction"), + nextQuestion + ? eq(chatActions.providerActionId, `photon:${nextQuestion[1]}`) + : eq( + sql`${chatActions.payload}->>'publicationId'`, + input.publication.id, + ), + eq(chatActions.status, "issued"), + ), + ) + .then((rows) => rows[0]); + const interaction = ( + await issueThreadInteractionService(db).listForIssue( + input.publication.issueId, + ) + ).find((candidate) => candidate.id === input.payload.interactionId); + if ( + action && + interaction?.status === "pending" && + nativePhotonInteraction(interaction) + ) { + const promptGuard = async () => { + await assertCurrent(); + const current = await db + .select({ status: issueThreadInteractions.status }) + .from(issueThreadInteractions) + .where( + and( + eq( + issueThreadInteractions.companyId, + input.endpoint.companyId, + ), + eq(issueThreadInteractions.id, interaction.id), + ), + ) + .then((rows) => rows[0]); + if (current?.status !== "pending") + throw new PhotonError( + "rejected", + "This interaction has already been resolved", + ); + }; + const receipt = await publishPhotonPrompt({ + adapter, + threadId: thread.id, + binding: action.payload as unknown as PhotonInteractionBinding, + interaction, + questionIndex: nextQuestion ? Number(nextQuestion[2]) : 0, + taskUrl: safeChatTaskUrl( + getTaskBaseUrl(), + input.publication.issueId, + ), + assertCurrent: promptGuard, + retryUnknown, + }); + return { id: receipt.promptMessageGuid }; + } + if (nextQuestion) + throw new PhotonError( + "rejected", + "This question is no longer available", + ); + } + const reply = await adapter.state.read<{ guid: string | null }>( + `publication-reply:${input.publication.id}`, + ); + let replyTo = reply?.guid ?? null; + if (!reply) { + const sourceRunId = await receiptReactionCompletionRunId( + db, + input.publication, + input.payload, + ); + if (sourceRunId) { + const source = await db + .select({ guid: chatMessageLinks.providerMessageId }) + .from(heartbeatRuns) + .innerJoin( + chatMessageLinks, + and( + eq(chatMessageLinks.companyId, input.endpoint.companyId), + eq(chatMessageLinks.endpointId, input.endpoint.id), + eq(chatMessageLinks.conversationId, input.conversation.id), + eq(chatMessageLinks.direction, "inbound"), + or( + sql`${chatMessageLinks.commentId}::text = ${heartbeatRuns.contextSnapshot}->>'wakeCommentId'`, + sql`coalesce(${heartbeatRuns.contextSnapshot}->'wakeCommentIds', '[]'::jsonb) ? ${chatMessageLinks.commentId}::text`, + ), + ), + ) + .where( + and( + eq(heartbeatRuns.id, sourceRunId), + eq(heartbeatRuns.companyId, input.endpoint.companyId), + ), + ) + .orderBy(desc(chatMessageLinks.createdAt)) + .limit(1) + .then((rows) => rows[0]); + replyTo = source?.guid ?? null; + } + replyTo = ( + await adapter.state.update<{ guid: string | null }>( + `publication-reply:${input.publication.id}`, + (current) => current ?? { guid: replyTo }, + ) + ).guid; + } + const sent = await adapter.publish( + thread.id, + input.publication.id, + { markdown: text, files }, + { assertCurrent, retryUnknown, replyTo: replyTo ?? undefined }, + ); + if (input.publication.idempotencyKey.startsWith("photon-response:")) { + const notice = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, input.endpoint.companyId), + eq(chatActions.endpointId, input.endpoint.id), + eq(chatActions.kind, "photon_response_notice"), + eq( + chatActions.providerActionId, + `photon-notice:${input.publication.idempotencyKey}`, + ), + ), + ) + .then((rows) => rows[0]); + if ( + notice && + typeof notice.payload.reference === "string" && + typeof notice.payload.questionIndex === "number" && + typeof notice.payload.publicationId === "string" + ) { + const receipt: PhotonPromptReceipt = { + schema: 1, + reference: notice.payload.reference, + questionIndex: notice.payload.questionIndex, + publicationId: notice.payload.publicationId, + promptMessageGuid: sent.id, + promptMessageGuids: sent.messageIds, + options: {}, + }; + for (const guid of sent.messageIds) + await adapter.state.update( + `prompt-message:${guid}`, + (current) => current ?? receipt, + ); + } + } + return sent; + } + return { deliverPhotonPublication }; +} + export function imessageProgressSupersessionReason( progress: SafeChatPublicationPayload["progressState"], @@ -6,4 +287,4 @@ export function imessageProgressSupersessionReason( return progress && ["queued", "working"].includes(progress) ? "iMessage uses typing instead of progress bubbles" : null; -} +} \ No newline at end of file diff --git a/server/src/services/chat-channels/imessage-photon/setup.ts b/server/src/services/chat-channels/imessage-photon/setup.ts new file mode 100644 index 00000000000..106342e9b09 --- /dev/null +++ b/server/src/services/chat-channels/imessage-photon/setup.ts @@ -0,0 +1,88 @@ +import type { createPhotonCredentialChecks } from "./credentials.js"; + +import type { EndpointRow } from "../runtime-types.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import { photonSharedScope, photonSharedIdentity } from "../../photon/cloud.js"; +import { photonChannelConfigurationSchema } from "@paperclipai/shared"; + +import { + and, + eq, + ne, +} from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { + chatEndpoints, + toolConnections, +} from "@paperclipai/db"; +import type { + ConfigureChatEndpointInput, +} from "@paperclipai/shared"; + +import { + conflict, + notFound, +} from "../../../errors.js"; + +import type { EndpointRecord } from "../runtime-types.js"; + +type PhotonSetupInspectionOptions = { + endpointRecord: (endpointId: string) => Promise; + inspectPhotonCredentials: ReturnType["inspectPhotonCredentials"]; +}; +export function createPhotonSetupInspection(db: Db, options: PhotonSetupInspectionOptions) { + const { endpointRecord, inspectPhotonCredentials } = options; + + + async function inspectPhoton( + endpointId: string, + input: { projectId: string; projectSecret: string }, + ) { + const record = await endpointRecord(endpointId); + if (!record || record.endpoint.provider !== "imessage-photon") + throw notFound("iMessage Photon endpoint not found"); + const inspection = structuredClone(await inspectPhotonCredentials( + input.projectId, + input.projectSecret, + )); + const reserved = await db + .select({ number: chatEndpoints.botExternalId }) + .from(chatEndpoints) + .where( + and( + eq(chatEndpoints.provider, "imessage-photon"), + ne(chatEndpoints.id, endpointId), + ne(chatEndpoints.status, "archived"), + ), + ); + for (const line of inspection.lines) { + if (reserved.some((row) => row.number === line.phoneNumber)) { + line.eligible = false; + line.unavailableReason = + "This number already belongs to another channel"; + } + } + inspection.eligible = inspection.allocation === "shared" ? inspection.eligible && !reserved.some((row) => row.number === photonSharedIdentity(inspection.projectId)) : inspection.lines.some((line) => line.eligible); + return inspection; + } + return { inspectPhoton }; +} +export function preparePhotonConfiguration(endpoint: EndpointRow, input: ConfigureChatEndpointInput, credentials: Record): Record { + const configuration = photonChannelConfigurationSchema.parse(input.photon ?? (credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { projectId: credentials.projectId, lineId: credentials.lineId })); + const lineId = configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId; + if (endpoint.botExternalId && (configuration.projectId !== endpoint.providerAccountId || credentials.lineId && lineId !== credentials.lineId)) throw conflict("A different Photon identity requires a new channel"); + return { ...credentials, ...configuration, lineId }; + } + +export function createPhotonConfiguration(db: Db, options: { invalidateRuntime: (endpointId: string) => Promise }) { + const { invalidateRuntime } = options; + async function persistPhotonConfiguration(endpoint: EndpointRow, credentials: Record, credentialLease: CredentialMutationLeaseGuard) { + await invalidateRuntime(endpoint.id); + await credentialLease.assertOwned(); + if (credentials.allocation === "shared") await db.update(chatEndpoints).set({ allowGroupChats: false }).where(eq(chatEndpoints.id, endpoint.id)); + await db.update(toolConnections).set({ config: { provider: endpoint.provider, photon: credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { allocation: "dedicated", projectId: credentials.projectId, lineId: credentials.lineId } } }).where(and(eq(toolConnections.companyId, endpoint.companyId), eq(toolConnections.id, endpoint.connectionId))); + } + return { persistPhotonConfiguration }; +} diff --git a/server/src/services/chat-channels/inbound/admission.ts b/server/src/services/chat-channels/inbound/admission.ts new file mode 100644 index 00000000000..13fd1c0ae61 --- /dev/null +++ b/server/src/services/chat-channels/inbound/admission.ts @@ -0,0 +1,399 @@ +import { createTeamsDestinationAdmission } from "../microsoft-teams/destination-admission.js"; +import { createTelegramDestinationAdmission } from "../telegram/destination-admission.js"; +import { createPhotonDestinationAdmission } from "../imessage-photon/destination-admission.js"; + +import { deliveryContentWasRedacted } from "./delivery-ledger.js"; + +import type { EndpointRow, DbTransaction, DbOrTransaction, InboundRuntimeContext } from "../runtime-types.js"; + +import { notifyChatDeliveryWork } from "../../chat-work-notifications.js"; + +import { + githubMessageAddressesAnotherBot, +} from "../../chat-github-events.js"; + +import { + createHash, +} from "node:crypto"; +import { + and, + eq, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatDeliveries, + chatEndpointResources, + chatEndpoints, + toolConnections, +} from "@paperclipai/db"; +import type { + ChatEndpointSetupState, + ToolCredentialSecretRef, +} from "@paperclipai/shared"; + +import type { + Message, + Thread, +} from "chat"; + +import type { DiscordGatewayOwnership } from "../runtime-types.js"; +import type { InboundEnvelope } from "./envelope.js"; +import type { ChatSdkMessageCallbackEvent } from "../../chat-sdk-runtime.js"; + +type InboundAdmissionInput = { + endpoint: EndpointRow; + thread: Thread; + message: Message; + trigger: ChatSdkMessageCallbackEvent["trigger"]; + runtimeContext?: InboundRuntimeContext; + admittedDeliveryId: string | null; + deferDrainUntilFollowup: boolean; + scheduledAt: number | null; + envelope: Pick; +}; +type GatewayAdmissionState = { + lost?: DiscordGatewayOwnership; + renewed?: { expiresAt: Date; ownership: DiscordGatewayOwnership }; +}; +type InboundAdmissionOptions = { + runtimeGeneration: (setup: ChatEndpointSetupState) => number; + credentialFingerprint: (refs: ToolCredentialSecretRef[]) => string; + leasedChatProvider: (provider: string) => boolean; + renewDiscordGatewayOwnershipForMessageAdmission: (tx: DbTransaction, endpointId: string, context: InboundRuntimeContext) => Promise<{ kind: "owned"; expiresAt: Date; ownership: DiscordGatewayOwnership } | { kind: "lost"; ownership: DiscordGatewayOwnership | null }>; + ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise; + DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS: number; +}; +export function createInboundAdmission(db: Db, options: InboundAdmissionOptions) { + const { runtimeGeneration, credentialFingerprint, leasedChatProvider, renewDiscordGatewayOwnershipForMessageAdmission, ensureResource, DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS } = options; + const { admitTeamsDestination } = createTeamsDestinationAdmission({ ensureResource }); + const { admitTelegramDestination } = createTelegramDestinationAdmission({ ensureResource }); + const { admitPhotonDestination } = createPhotonDestinationAdmission({ ensureResource }); + async function admitInboundMessage(input: InboundAdmissionInput, discordGatewayAdmissionState: GatewayAdmissionState) { + const { endpoint, thread, message, trigger, runtimeContext, admittedDeliveryId, deferDrainUntilFollowup, scheduledAt, envelope } = input; + const { providerEventId, addressed, eventKind, normalized, redactedDestinationNormalized, githubAutomatic, githubIssue } = envelope; + return await db.transaction(async (tx) => { + const currentEndpoint = await tx + .select({ + status: chatEndpoints.status, + setup: chatEndpoints.setup, + allowDirectMessages: chatEndpoints.allowDirectMessages, + requireAtMention: chatEndpoints.requireAtMention, + allowGroupChats: chatEndpoints.allowGroupChats, + botUsername: chatEndpoints.botUsername, + }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!currentEndpoint) return null; + // An SDK callback can finish parsing after a pause/resume cycle. Each + // runtime captures the durable endpoint generation at registration; + // resume advances it under the same row lock used by admission. This + // rejects stale callbacks without relying on provider/server clock sync. + const currentRuntimeGeneration = runtimeGeneration(currentEndpoint.setup); + const currentCredentialFingerprint = runtimeContext + ? await tx + .select({ refs: toolConnections.credentialSecretRefs }) + .from(toolConnections) + .where( + and( + eq(toolConnections.companyId, endpoint.companyId), + eq(toolConnections.id, endpoint.connectionId), + ), + ) + .then((rows) => credentialFingerprint(rows[0]?.refs ?? [])) + : null; + const staleActivation = + runtimeContext !== undefined && + (runtimeContext.generation !== currentRuntimeGeneration || + runtimeContext.credentialFingerprint !== + currentCredentialFingerprint); + const endpointAccepting = + ["verifying", "active"].includes(currentEndpoint.status) && + !staleActivation; + if ( + endpointAccepting && + leasedChatProvider(endpoint.provider) && + runtimeContext && + !admittedDeliveryId + ) { + // The first process-local ownership check can precede a host pause. + // Reclaim the same durable token while the endpoint admission row is + // locked so a standby takeover cannot be followed by an obsolete + // callback write. This renews only the exact token that authenticated + // the callback; generation and credential checks above remain intact. + const gatewayOwnership = + await renewDiscordGatewayOwnershipForMessageAdmission( + tx, + endpoint.id, + runtimeContext, + ); + if (gatewayOwnership.kind === "lost") { + if (gatewayOwnership.ownership) { + discordGatewayAdmissionState.lost = gatewayOwnership.ownership; + } + return null; + } + discordGatewayAdmissionState.renewed = gatewayOwnership; + } + let provisionalTeamsSetupReply = false; + let destinationAccepting = true; + if (endpointAccepting && thread.isDM) { + destinationAccepting = currentEndpoint.allowDirectMessages; + } + if ( + endpointAccepting && + endpoint.provider === "microsoft-teams" && + !thread.isDM + ) { + ({ destinationAccepting, provisionalTeamsSetupReply } = await admitTeamsDestination({ endpoint, currentEndpoint, thread, message, addressed }, tx)); + } + if ( + endpointAccepting && + endpoint.provider === "telegram" && + !thread.isDM + ) { + destinationAccepting = await admitTelegramDestination({ endpoint, currentEndpoint, thread, addressed }, tx); + } + if ( + endpointAccepting && + endpoint.provider === "imessage-photon" && + !thread.isDM + ) { + destinationAccepting = await admitPhotonDestination({ endpoint, currentEndpoint, thread }, tx); + } + const mentionRequired = + endpoint.provider === "slack" && + currentEndpoint.requireAtMention && + trigger !== "mention" && + message.isMention !== true; + // GitHub sends the same comment to every installed App. Subscribed + // conversations are a fallback for unaddressed replies, not permission + // to wake every bot when the person names a different connected App. + // Recheck inside durable admission so recovered deliveries follow the + // same company-scoped routing decision before any task or reaction. + const githubAddressedElsewhere = + endpointAccepting && + endpoint.provider === "github" && + !githubAutomatic && + !githubIssue && + (await githubMessageAddressesAnotherBot(tx, { + id: endpoint.id, + companyId: endpoint.companyId, + botUsername: currentEndpoint.botUsername, + }, message.text)); + const accepting = endpointAccepting && destinationAccepting && !githubAddressedElsewhere && !mentionRequired; + const redactDestinationDelivery = + githubAddressedElsewhere || mentionRequired || (!accepting && thread.isDM) || + + (!thread.isDM && + (endpoint.provider === "microsoft-teams" || + endpoint.provider === "telegram" || + endpoint.provider === "imessage-photon") && + (!accepting || provisionalTeamsSetupReply)); + const ignoredAt = accepting ? null : new Date(); + const inactiveReason = !endpointAccepting + ? staleActivation + ? "Connection activation changed before admission" + : "Connection is not active" + : githubAddressedElsewhere + ? "GitHub message explicitly mentions a different connected bot" + : mentionRequired + ? "Message did not @mention the bot" + : endpoint.provider === "telegram" && !thread.isDM && !addressed + ? "Message did not address the agent" + : "Destination is not enabled in Paperclip"; + let candidate = admittedDeliveryId + ? await tx + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.id, admittedDeliveryId), + eq(chatDeliveries.endpointId, endpoint.id), + eq(chatDeliveries.providerEventId, providerEventId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + if (!admittedDeliveryId) { + await notifyChatDeliveryWork(tx); + const [delivery] = await tx + .insert(chatDeliveries) + .values({ + companyId: endpoint.companyId, + endpointId: endpoint.id, + providerEventId, + deduplicationKey: createHash("sha256") + .update(providerEventId) + .digest("hex"), + eventKind, + normalizedEvent: redactDestinationDelivery + ? redactedDestinationNormalized + : deferDrainUntilFollowup && accepting + ? { + ...normalized, + providerThreadPending: true, + } + : normalized, + state: accepting + ? deferDrainUntilFollowup + ? "retry" + : "received" + : "filtered", + nextAttemptAt: + accepting && deferDrainUntilFollowup + ? new Date( + Date.now() + DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS, + ) + : accepting && scheduledAt + ? new Date(scheduledAt) + : null, + redactedError: + accepting && deferDrainUntilFollowup + ? "Waiting for Discord thread creation confirmation" + : accepting + ? null + : inactiveReason, + processedAt: ignoredAt, + }) + .onConflictDoNothing() + .returning(); + candidate = delivery ?? null; + } + if (!candidate && !admittedDeliveryId) { + const existingDelivery = await tx + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.endpointId, endpoint.id), + eq(chatDeliveries.providerEventId, providerEventId), + ), + ) + .then((rows) => rows[0] ?? null); + if (existingDelivery) { + const hydrateTeamsDelivery = + endpoint.provider === "microsoft-teams" && + !thread.isDM && + accepting && + !provisionalTeamsSetupReply && + ["received", "retry", "processing"].includes( + existingDelivery.state, + ) && + deliveryContentWasRedacted(existingDelivery.normalizedEvent); + await notifyChatDeliveryWork(tx); + [candidate] = await tx + .update(chatDeliveries) + .set({ + // Increment under PostgreSQL's row lock so simultaneous handler + // fanout and provider retries cannot lose duplicate telemetry. + normalizedEvent: redactDestinationDelivery + ? sql`${JSON.stringify(redactedDestinationNormalized)}::jsonb + || jsonb_build_object( + 'deduplication', + jsonb_build_object( + 'duplicateCount', + coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, + 'lastDuplicateAt', + ${new Date().toISOString()}::text + ) + )` + : hydrateTeamsDelivery + ? sql`${JSON.stringify(normalized)}::jsonb + || jsonb_build_object( + 'deduplication', + coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb) + || jsonb_build_object( + 'duplicateCount', + coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, + 'lastDuplicateAt', + ${new Date().toISOString()}::text + ) + )` + : sql`coalesce(${chatDeliveries.normalizedEvent}, '{}'::jsonb) + || jsonb_build_object( + 'deduplication', + coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb) + || jsonb_build_object( + 'duplicateCount', + coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1, + 'lastDuplicateAt', + ${new Date().toISOString()}::text + ) + )`, + ...(!accepting && + ["received", "retry"].includes(existingDelivery.state) + ? { + state: "filtered" as const, + nextAttemptAt: null, + redactedError: inactiveReason, + processedAt: ignoredAt, + } + : {}), + updatedAt: new Date(), + }) + .where(eq(chatDeliveries.id, existingDelivery.id)) + .returning(); + } + } + if ( + candidate && + admittedDeliveryId && + accepting && + !provisionalTeamsSetupReply && + endpoint.provider === "microsoft-teams" && + !thread.isDM && + ["received", "retry", "processing"].includes(candidate.state) && + deliveryContentWasRedacted(candidate.normalizedEvent) + ) { + [candidate] = await tx + .update(chatDeliveries) + .set({ normalizedEvent: normalized, updatedAt: new Date() }) + .where(eq(chatDeliveries.id, candidate.id)) + .returning(); + } + if ( + candidate && + admittedDeliveryId && + !accepting && + ["received", "retry", "processing"].includes(candidate.state) + ) { + await notifyChatDeliveryWork(tx); + [candidate] = await tx + .update(chatDeliveries) + .set({ + state: "filtered", + ...(redactDestinationDelivery + ? { + normalizedEvent: redactedDestinationNormalized, + principalId: null, + } + : {}), + nextAttemptAt: null, + redactedError: inactiveReason, + processedAt: ignoredAt, + updatedAt: new Date(), + }) + .where(eq(chatDeliveries.id, candidate.id)) + .returning(); + } + if (!accepting && ignoredAt) { + await tx + .update(chatEndpoints) + .set({ lastEventAt: ignoredAt, updatedAt: ignoredAt }) + .where(eq(chatEndpoints.id, endpoint.id)); + } + return { + accepting, + candidate: candidate ?? null, + provisionalTeamsSetupReply, + }; + }); + } + return { admitInboundMessage }; +} diff --git a/server/src/services/chat-channels/inbound/authorization.ts b/server/src/services/chat-channels/inbound/authorization.ts new file mode 100644 index 00000000000..cb3f330a3d3 --- /dev/null +++ b/server/src/services/chat-channels/inbound/authorization.ts @@ -0,0 +1,429 @@ +import type { EndpointRow, DbTransaction, DbOrTransaction, LifecycleRuntimeFence } from "../runtime-types.js"; + +import { + stableExternalPrincipalId, +} from "../message-lifecycle.js"; + +import { voiceSessionStore } from "../../voice/voice-session-store.js"; + +import { githubChatPrincipalAccess } from "../../chat-github-access.js"; + +import { + and, + eq, + inArray, + ne, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatConversations, + chatEndpointResources, + chatExternalPrincipals, + chatIdentityLinks, + companyMemberships, +} from "@paperclipai/db"; + +import { + HttpError, + forbidden, +} from "../../../errors.js"; + +import type { + Author, + Thread, +} from "chat"; +import { canonicalProviderResourceId, chatSurfaceKind, providerResourceLabelFromThread, providerResourceType, nonDirectDestinationAllowed } from "./surfaces.js"; +type ResourceRow = typeof chatEndpointResources.$inferSelect; +type InboundAuthorityOptions = { + voiceStore: Pick, "authorizePrincipal">; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; +}; +export function createChatInboundAuthority(db: Db, options: InboundAuthorityOptions) { + const { voiceStore, runtimeCallbackEndpoint } = options; + + + async function findAuthorizedIdentityLink(endpoint: EndpointRow, userId: string) { + return db.select({ id: chatIdentityLinks.id }).from(chatIdentityLinks) + .innerJoin(companyMemberships, and( + eq(companyMemberships.companyId, chatIdentityLinks.companyId), eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, userId), eq(companyMemberships.status, "active"), ne(companyMemberships.membershipRole, "viewer"), + )) + .where(and(eq(chatIdentityLinks.companyId, endpoint.companyId), eq(chatIdentityLinks.endpointId, endpoint.id), + eq(chatIdentityLinks.paperclipUserId, userId), eq(chatIdentityLinks.status, "linked"))) + .limit(1).then((rows) => rows[0] ?? null); + } + + async function ensurePrincipal( + endpoint: EndpointRow, + author: Author, + raw?: unknown, + ) { + const providerAccountId = endpoint.providerAccountId ?? "unknown"; + const externalId = stableExternalPrincipalId( + endpoint.provider, + author, + raw, + ); + const [principal] = await db + .insert(chatExternalPrincipals) + .values({ + companyId: endpoint.companyId, + provider: endpoint.provider, + providerAccountId, + externalId, + kind: + author.isBot === true ? "bot" : author.isSystem ? "system" : "user", + displayName: author.fullName, + handle: author.userName, + isBot: author.isBot === true || author.isMe, + lastSeenAt: new Date(), + }) + .onConflictDoUpdate({ + target: [ + chatExternalPrincipals.companyId, + chatExternalPrincipals.provider, + chatExternalPrincipals.providerAccountId, + chatExternalPrincipals.externalId, + ], + set: { + displayName: author.fullName, + handle: author.userName, + isBot: author.isBot === true || author.isMe, + lastSeenAt: new Date(), + updatedAt: new Date(), + }, + }) + .returning(); + if (endpoint.provider === "speko") { + try { + const authorization = await db.transaction((tx) => voiceStore.authorizePrincipal(tx, endpoint.companyId, endpoint.id, externalId)); + return { principal, userId: authorization.userId, linkedDenied: false }; + } catch (error) { + if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error; + return { principal, userId: null, linkedDenied: true }; + } + } + const link = await db + .select({ + userId: chatIdentityLinks.paperclipUserId, + status: chatIdentityLinks.status, + }) + .from(chatIdentityLinks) + .where( + and( + eq(chatIdentityLinks.endpointId, endpoint.id), + eq(chatIdentityLinks.principalId, principal.id), + ), + ) + .then((rows) => rows[0] ?? null); + if (link?.status === "linked" && link.userId) { + const membership = await db + .select({ + status: companyMemberships.status, + membershipRole: companyMemberships.membershipRole, + }) + .from(companyMemberships) + .where( + and( + eq(companyMemberships.companyId, endpoint.companyId), + eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, link.userId), + ), + ) + .then((rows) => rows[0] ?? null); + // External chat performs the same task mutations as the board. A linked + // viewer therefore remains read-only instead of gaining write authority + // merely by using a provider account. + if ( + membership?.status !== "active" || + membership.membershipRole === "viewer" + ) { + return { principal, userId: null, linkedDenied: true }; + } + return { principal, userId: link.userId, linkedDenied: false }; + } + return { principal, userId: null, linkedDenied: false }; + } + + async function sponsorAllowsGuest(endpoint: EndpointRow): Promise { + // A null sponsor is the local board/operator context used by self-hosted + // instances. Cloud endpoints retain the creating user and must revalidate + // that user's current company authority on every sponsored guest turn. + if (!endpoint.sponsorUserId) return true; + const membership = await db + .select({ + status: companyMemberships.status, + membershipRole: companyMemberships.membershipRole, + }) + .from(companyMemberships) + .where( + and( + eq(companyMemberships.companyId, endpoint.companyId), + eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, endpoint.sponsorUserId), + ), + ) + .then((rows) => rows[0] ?? null); + return ( + membership?.status === "active" && membership.membershipRole !== "viewer" + ); + } + + async function lockCurrentPrincipalAuthorization( + tx: DbOrTransaction, + endpoint: EndpointRow, + principalId: string, + ): Promise<{ + allowed: boolean; + linkedDenied: boolean; + userId: string | null; + sponsorUserId?: string | null; + }> { + if (endpoint.provider === "speko") { + const principal = await tx.select({ externalId: chatExternalPrincipals.externalId }).from(chatExternalPrincipals) + .where(and(eq(chatExternalPrincipals.id, principalId), eq(chatExternalPrincipals.companyId, endpoint.companyId), eq(chatExternalPrincipals.provider, "speko"))).then((rows) => rows[0]); + if (!principal) return { allowed: false, linkedDenied: true, userId: null }; + try { return await voiceStore.authorizePrincipal(tx as import("../../voice/voice-session-store.js").VoiceTransaction, endpoint.companyId, endpoint.id, principal.externalId); } + catch (error) { + if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error; + return { allowed: false, linkedDenied: true, userId: null }; + } + } + const githubAccess = await githubChatPrincipalAccess(tx, endpoint, principalId); + if (githubAccess) return githubAccess; + // Link confirmation already uses this transaction-scoped identity key. + // Taking it at the final task mutation boundary prevents a newly confirmed + // identity from racing the authorization snapshot. Row locks below also + // serialize revocation and membership changes that do not use this key. + await tx.execute( + sql`select pg_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0))`, + ); + const link = await tx + .select({ + status: chatIdentityLinks.status, + userId: chatIdentityLinks.paperclipUserId, + }) + .from(chatIdentityLinks) + .where( + and( + eq(chatIdentityLinks.companyId, endpoint.companyId), + eq(chatIdentityLinks.endpointId, endpoint.id), + eq(chatIdentityLinks.principalId, principalId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + if (link?.status === "linked") { + if (!link.userId) { + return { allowed: false, linkedDenied: true, userId: null }; + } + const membership = await tx + .select({ + status: companyMemberships.status, + membershipRole: companyMemberships.membershipRole, + }) + .from(companyMemberships) + .where( + and( + eq(companyMemberships.companyId, endpoint.companyId), + eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, link.userId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + const allowed = + membership?.status === "active" && + membership.membershipRole !== "viewer"; + return { + allowed, + linkedDenied: !allowed, + userId: allowed ? link.userId : null, + }; + } + if (!endpoint.allowUnlinkedPeople) { + return { allowed: false, linkedDenied: false, userId: null }; + } + if (!endpoint.sponsorUserId) { + return { allowed: true, linkedDenied: false, userId: null }; + } + const sponsorMembership = await tx + .select({ + status: companyMemberships.status, + membershipRole: companyMemberships.membershipRole, + }) + .from(companyMemberships) + .where( + and( + eq(companyMemberships.companyId, endpoint.companyId), + eq(companyMemberships.principalType, "user"), + eq(companyMemberships.principalId, endpoint.sponsorUserId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + return { + allowed: + sponsorMembership?.status === "active" && + sponsorMembership.membershipRole !== "viewer", + linkedDenied: false, + userId: null, + }; + } + + async function requireCurrentExternalActionAuthorization( + tx: DbTransaction, + input: { + conversationId: string; + endpointId: string; + expectedUserId: string; + principalId: string; + runtimeContext: LifecycleRuntimeFence; + }, + ): Promise { + const endpoint = await runtimeCallbackEndpoint( + tx, + input.endpointId, + input.runtimeContext, + ["active", "verifying"], + ); + // Photon setup already admits linked senders and publishes agent prompts. + // Let those prompts resolve so a clarifying question cannot deadlock the + // actual-reply qualification. Other providers retain their active-only gate. + if (!endpoint || (endpoint.status !== "active" && + (endpoint.provider !== "imessage-photon" || endpoint.setup.step !== "test"))) { + throw forbidden("This chat action is no longer authorized", { + code: "chat_action_authorization_changed", + }); + } + const conversation = await tx + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, endpoint.companyId), + eq(chatConversations.endpointId, endpoint.id), + eq(chatConversations.id, input.conversationId), + inArray(chatConversations.state, ["active", "waiting"]), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + if (!conversation) { + throw forbidden("This chat action is no longer authorized", { + code: "chat_action_authorization_changed", + }); + } + const resource = conversation.resourceId + ? await tx + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.companyId, endpoint.companyId), + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.id, conversation.resourceId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const destinationAllowed = conversation.isDirectMessage + ? endpoint.allowDirectMessages + : nonDirectDestinationAllowed(endpoint, resource); + const authorization = await lockCurrentPrincipalAuthorization( + tx, + endpoint, + input.principalId, + ); + // Governed actions never fall back to sponsored-guest authority. If the + // provider identity was relinked while this callback waited, reject this + // stale actor snapshot and let a fresh callback use the new identity. + if ( + !destinationAllowed || + !authorization.allowed || + authorization.userId !== input.expectedUserId + ) { + throw forbidden("This chat action is no longer authorized", { + code: "chat_action_authorization_changed", + }); + } + } + + function isExternalActionAuthorizationChange(error: unknown): boolean { + if (!error || typeof error !== "object") return false; + const value = error as { details?: unknown }; + return ( + Boolean(value.details) && + typeof value.details === "object" && + (value.details as { code?: unknown }).code === + "chat_action_authorization_changed" + ); + } + + async function ensureResource( + endpoint: EndpointRow, + thread: Thread, + enabledBySetupActivation: boolean, + database: DbOrTransaction = db, + ) { + const type = providerResourceType( + endpoint.provider, + chatSurfaceKind(endpoint.provider, thread), + ); + const providerResourceId = canonicalProviderResourceId( + endpoint.provider, + thread, + ); + const resourceLabel = providerResourceLabelFromThread( + endpoint.provider, + thread, + ); + const [resource] = await database + .insert(chatEndpointResources) + .values({ + companyId: endpoint.companyId, + endpointId: endpoint.id, + type, + providerResourceId, + label: resourceLabel.label, + availability: "available", + // Slack can deliver the invitation's app_mention before the membership + // callback. Give either arrival order the same new-channel default. + enabled: + thread.isDM || + enabledBySetupActivation || + (endpoint.provider === "slack" && type === "channel"), + }) + .onConflictDoUpdate({ + target: [ + chatEndpointResources.endpointId, + chatEndpointResources.type, + chatEndpointResources.providerResourceId, + ], + set: { + // Provider inventory and membership lifecycle own availability. + // A delayed message must not resurrect a destination after the bot + // was removed or an installation lost access. + // Slack callbacks are not guaranteed to carry a channel name. Keep + // an inventory- or conversations.info-derived label instead of + // downgrading it to C… / slack:C… when a delayed message arrives. + ...(resourceLabel.fallback + ? {} + : { + label: sql`case + when ${chatEndpointResources.availability} = 'available' then ${resourceLabel.label} + else ${chatEndpointResources.label} + end`, + }), + updatedAt: new Date(), + }, + }) + .returning(); + return resource; + } + return { findAuthorizedIdentityLink, ensurePrincipal, sponsorAllowsGuest, lockCurrentPrincipalAuthorization, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, ensureResource }; +} diff --git a/server/src/services/chat-channels/inbound/conversation-queue.ts b/server/src/services/chat-channels/inbound/conversation-queue.ts new file mode 100644 index 00000000000..6b0c4af9cfa --- /dev/null +++ b/server/src/services/chat-channels/inbound/conversation-queue.ts @@ -0,0 +1,767 @@ +import { createGitHubMessageProvenance } from "../github/message-provenance.js"; +import { createDiscordRootThreads } from "../discord/root-threads.js"; + +import type { EndpointRow, RuntimeContext, InboundRuntimeContext } from "../runtime-types.js"; + +import { + telegramDeliveryHasZeroMessageId, +} from "../telegram/messages.js"; + +import { notifyChatDeliveryWork } from "../../chat-work-notifications.js"; + +import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js"; + +import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared"; + +import { + createHash, + randomUUID, +} from "node:crypto"; +import { + and, + eq, + inArray, + lte, + notInArray, + or, + sql, +} from "drizzle-orm"; + +import { + externalThreadIdentityCondition, +} from "../../chat-control-chronology.js"; + +import type { Db } from "@paperclipai/db"; + +import { + chatDeliveries, + chatEndpointLeases, + chatEndpoints, +} from "@paperclipai/db"; + +import { + normalizeContentType, +} from "../../../attachment-types.js"; + +import { logger } from "../../../middleware/logger.js"; + +import { + type ChatSdkMessageCallbackEvent, +} from "../../chat-sdk-runtime.js"; + +import { isExternalChatWaitAuthorizationContention } from "../../native-runtime/chat-attachment-reuse.js"; + +import { + restoreGitHubAttachmentLimitOmissions, +} from "../../chat-github-attachments.js"; + +import type { + Attachment, + Message, + Thread, +} from "chat"; +import { DELIVERY_PROCESSING_STALE_MS, deliveryContentWasRedacted, setupDestinationActivationSuppressed, normalizedDeliveryThreadId, normalizedLifecycleTargetEventId, normalizedLifecycleEffect, deliveryReady, conversationDeliveryOrder, pendingInboundWakeupCondition, lifecycleRuntimeFence } from "./delivery-ledger.js"; +import type { createChatMessageLifecycle } from "./lifecycle.js"; +import type { LiveInboundMessage } from "../runtime-types.js"; +type DeliveryRow = typeof chatDeliveries.$inferSelect; +type ConversationDeliveryQueueOptions = { + DELIVERY_DRAIN_LIMIT: number; + DELIVERY_LEASE_TTL_MS: number; + discordRootThreadPending: ReturnType["discordRootThreadPending"]; + reconcileDiscordRootThread: ReturnType["reconcileDiscordRootThread"]; + durableExternalThreadIdentity: (threadId: string) => string; + liveInboundMessages: Map; + processInboundWakeup: (deliveryId: string) => Promise; + processLifecycleDelivery: ReturnType["processLifecycleDelivery"]; + processMessage: (endpoint: EndpointRow, thread: Thread, message: Message, trigger: ChatSdkMessageCallbackEvent["trigger"], ingressOnly?: boolean, recoveredProviderUrl?: string | null, runtimeContext?: InboundRuntimeContext, providerUpdateId?: number, admittedDeliveryId?: string | null, receiptReactionSupported?: boolean, preauthorizedUserId?: string | null, deferDrainUntilFollowup?: boolean, suppressSetupDestinationActivation?: boolean, slackSlashControl?: boolean) => Promise; + redactError: (error: unknown) => string; + runtimeContexts: WeakMap; + runtimeFor: (endpoint: EndpointRow) => Promise; + settleRejectedInboundWakeups: (onlyDeliveryId?: string) => Promise; + sanitizeFilename: (value: string | undefined) => string | null; + githubManualMessages: WeakMap; + githubAutomaticMessages: WeakMap; + githubIssueMessages: WeakMap; + conversationDrainKey: (endpointId: string, threadId: string) => string; + conversationLeaseRenewalIntervalMs?: number; + renewConversationDeliveryLease?: (input: { endpointId: string; leaseKey: string; token: string; expiresAt: Date }) => Promise; +}; +export function createChatConversationDeliveryQueue(db: Db, options: ConversationDeliveryQueueOptions) { + const { DELIVERY_DRAIN_LIMIT, DELIVERY_LEASE_TTL_MS, discordRootThreadPending, reconcileDiscordRootThread, durableExternalThreadIdentity, liveInboundMessages, processInboundWakeup, processLifecycleDelivery, processMessage, redactError, runtimeContexts, runtimeFor, settleRejectedInboundWakeups, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages, conversationDrainKey } = options; + const { restoreGitHubMessageProvenance } = createGitHubMessageProvenance({ githubManualMessages, githubAutomaticMessages, githubIssueMessages }); + + + async function earliestOpenConversationDelivery( + endpointId: string, + threadId: string, + ): Promise { + const externalThreadId = sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId'`; + const candidate = await db + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.endpointId, endpointId), + or( + inArray(chatDeliveries.state, ["received", "retry", "processing"]), + and( + eq(chatDeliveries.state, "processed"), + pendingInboundWakeupCondition(), + ), + ), + notInArray(chatDeliveries.eventKind, [ + "reaction_added", + "reaction_removed", + ]), + externalThreadIdentityCondition(externalThreadId, threadId), + ), + ) + .orderBy(...conversationDeliveryOrder()) + .limit(1) + .then((rows) => rows[0] ?? null); + if (!candidate) return null; + + // A lifecycle callback can reach Paperclip before its create callback. If + // the root becomes durable while the lifecycle row is waiting, always + // return that exact dependency first even when a coarse or malformed + // provider timestamp would otherwise put the edit/delete at the head. + const targetProviderEventId = normalizedLifecycleTargetEventId(candidate); + if (!targetProviderEventId) return candidate; + return db + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.endpointId, endpointId), + eq(chatDeliveries.providerEventId, targetProviderEventId), + inArray(chatDeliveries.state, ["received", "retry", "processing"]), + ), + ) + .limit(1) + .then((rows) => rows[0] ?? candidate); + } + + async function acquireConversationDeliveryLease(input: { + companyId: string; + endpointId: string; + threadId: string; + }): Promise<{ leaseKey: string; token: string } | null> { + const now = new Date(); + const token = randomUUID(); + const leaseKey = `inbound:${createHash("sha256").update(durableExternalThreadIdentity(input.threadId)).digest("hex")}`; + const expiresAt = new Date(now.getTime() + DELIVERY_LEASE_TTL_MS); + const inserted = await db + .insert(chatEndpointLeases) + .values({ + companyId: input.companyId, + endpointId: input.endpointId, + leaseKey, + token, + expiresAt, + }) + .onConflictDoNothing() + .returning({ id: chatEndpointLeases.id }); + if (inserted.length > 0) return { leaseKey, token }; + const reclaimed = await db + .update(chatEndpointLeases) + .set({ token, expiresAt, updatedAt: now }) + .where( + and( + eq(chatEndpointLeases.companyId, input.companyId), + eq(chatEndpointLeases.endpointId, input.endpointId), + eq(chatEndpointLeases.leaseKey, leaseKey), + lte(chatEndpointLeases.expiresAt, now), + ), + ) + .returning({ id: chatEndpointLeases.id }); + return reclaimed.length > 0 ? { leaseKey, token } : null; + } + + async function releaseConversationDeliveryLease(input: { + endpointId: string; + leaseKey: string; + token: string; + }): Promise { + await db + .delete(chatEndpointLeases) + .where( + and( + eq(chatEndpointLeases.endpointId, input.endpointId), + eq(chatEndpointLeases.leaseKey, input.leaseKey), + eq(chatEndpointLeases.token, input.token), + ), + ); + } + + function messageFromDelivery( + delivery: DeliveryRow, + thread: Thread, + endpointRuntime: ChatEndpointRuntime, + ): { + message: Message; + receiptReactionSupported: boolean; + suppressSetupDestinationActivation: boolean; + trigger: ChatSdkMessageCallbackEvent["trigger"]; + providerUrl: string | null; + } | null { + if (telegramDeliveryHasZeroMessageId(delivery, endpointRuntime.provider)) + return null; + const normalized = delivery.normalizedEvent as { + acknowledgement?: { receiptReactionSupported?: unknown }; + kind?: unknown; + trigger?: unknown; + principal?: { + externalId?: unknown; + displayName?: unknown; + handle?: unknown; + }; + message?: { + providerMessageId?: unknown; + sourceProviderMessageId?: unknown; + text?: unknown; + mentionedBot?: unknown; + attachments?: Array<{ + name?: unknown; + mimeType?: unknown; + size?: unknown; + recovery?: unknown; + }>; + attachmentLimitOmissions?: unknown; + photonReply?: { guid?: unknown; part?: unknown }; + }; + conversation?: { providerUrl?: unknown }; + }; + const providerMessageId = normalized.message?.providerMessageId; + const externalId = normalized.principal?.externalId; + if (typeof providerMessageId !== "string" || typeof externalId !== "string") + return null; + const isTeams = endpointRuntime.provider === "microsoft-teams"; + const referenceOnlyTeamsAttachments = isTeams && !thread.isDM; + const originFence = lifecycleRuntimeFence(delivery); + const storedAttachments = normalized.message?.attachments ?? []; + const attachments = ( + isTeams ? storedAttachments.slice(0, 20) : storedAttachments + ) + .map((attachment) => { + if (!attachment || typeof attachment !== "object") return null; + const rehydrated = + attachment.recovery && + (!referenceOnlyTeamsAttachments || + (typeof attachment.recovery === "object" && + attachment.recovery !== null && + "locator" in attachment.recovery && + typeof attachment.recovery.locator === "object" && + attachment.recovery.locator !== null && + "kind" in attachment.recovery.locator && + attachment.recovery.locator.kind === "teams_inline_image")) + ? endpointRuntime.rehydrateAttachment(attachment.recovery, { + threadId: thread.id, + messageId: typeof normalized.message?.sourceProviderMessageId === "string" ? normalized.message.sourceProviderMessageId : providerMessageId, + runtimeGeneration: originFence?.generation, + credentialFingerprint: originFence?.credentialFingerprint, + principalExternalId: externalId, + isDirectMessage: thread.isDM, + }) + : null; + if (rehydrated) return rehydrated; + if ( + isTeams || + endpointRuntime.provider === "imessage-photon" || + (endpointRuntime.provider === "telegram" && + typeof attachment.recovery === "object" && + attachment.recovery !== null && + "locator" in attachment.recovery && + typeof attachment.recovery.locator === "object" && + attachment.recovery.locator !== null && + "kind" in attachment.recovery.locator && + attachment.recovery.locator.kind === "telegram_media") + ) { + // Non-personal references and personal files whose safe download + // capability did not survive restart still belong to this input. + // Keep bounded metadata so ingestion records an explicit omission; + // never persist bearer URLs or fabricate a download capability. + // Even a legacy URL locator cannot enable non-personal downloads. + // A Telegram media locator which fails its exact source proof also + // remains an unavailable input, never an empty attachment-free turn. + return { + type: "file", + name: + sanitizeFilename( + typeof attachment.name === "string" + ? attachment.name + : undefined, + ) ?? undefined, + mimeType: normalizeContentType( + typeof attachment.mimeType === "string" + ? attachment.mimeType + : undefined, + ).slice(0, 255), + size: + typeof attachment.size === "number" && + Number.isSafeInteger(attachment.size) && + attachment.size >= 0 + ? attachment.size + : undefined, + } as Attachment; + } + return null; + }) + .filter((attachment): attachment is Attachment => Boolean(attachment)); + const message = { + id: providerMessageId, + threadId: thread.id, + text: + typeof normalized.message?.text === "string" + ? normalized.message.text + : "", + formatted: { type: "root", children: [] }, + raw: + endpointRuntime.provider === "imessage-photon" && + typeof normalized.message?.photonReply?.guid === "string" + ? { + replyTargetGuid: normalized.message.photonReply.guid, + threadOriginatorPart: normalized.message.photonReply.part, + } + : endpointRuntime.provider === "github" && typeof normalized.message?.sourceProviderMessageId === "string" + ? { githubSourceMessageId: normalized.message.sourceProviderMessageId } : {}, + author: { + userId: externalId, + userName: + typeof normalized.principal?.handle === "string" + ? normalized.principal.handle + : externalId, + fullName: + typeof normalized.principal?.displayName === "string" + ? normalized.principal.displayName + : externalId, + isBot: false, + isMe: false, + isSystem: false, + }, + metadata: { dateSent: delivery.receivedAt, edited: false }, + attachments, + links: [], + isMention: normalized.message?.mentionedBot === true, + } as unknown as Message; + restoreGitHubMessageProvenance(delivery, endpointRuntime, message); + if ( + endpointRuntime.provider === "github" || + endpointRuntime.provider === "imessage-photon" + ) { + restoreGitHubAttachmentLimitOmissions( + message, + normalized.message?.attachmentLimitOmissions, + ); + } + const normalizedTrigger = normalized.trigger; + const trigger: ChatSdkMessageCallbackEvent["trigger"] = + normalizedTrigger === "direct_message" || + normalizedTrigger === "mention" || + normalizedTrigger === "subscribed_message" || + normalizedTrigger === "unaddressed_message" + ? normalizedTrigger + : normalized.kind === "direct_message" + ? "direct_message" + : normalized.kind === "mention" + ? "mention" + : "subscribed_message"; + return { + message, + // Legacy deliveries all originated from native provider messages and + // therefore retain the historical default. + receiptReactionSupported: + normalized.acknowledgement?.receiptReactionSupported !== false, + suppressSetupDestinationActivation: setupDestinationActivationSuppressed( + delivery.normalizedEvent, + ), + trigger, + providerUrl: + typeof normalized.conversation?.providerUrl === "string" + ? normalized.conversation.providerUrl + : null, + }; + } + + /** + * Drain one external conversation in durable receipt order. The database + * lease is endpoint + thread scoped, so separate server processes can work + * on different conversations concurrently but can never mutate the same + * Paperclip task from two inbound turns at once. + */ + async function drainConversationDeliveries( + endpointId: string, + threadId: string, + ): Promise { + const endpoint = await db + .select() + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpointId)) + .then((rows) => rows[0] ?? null); + if (!endpoint) return false; + if (endpoint.status === "paused" || endpoint.status === "attention") + return false; + + const lease = await acquireConversationDeliveryLease({ + companyId: endpoint.companyId, + endpointId, + threadId, + }); + if (!lease) return false; + + let leaseOwned = true; + let renewal: Promise | null = null; + const renewTimer = setInterval( + () => { + if (!leaseOwned || renewal) return; + const expiresAt = new Date(Date.now() + DELIVERY_LEASE_TTL_MS); + renewal = Promise.resolve() + .then(async () => { + const renewed = options.renewConversationDeliveryLease + ? await options.renewConversationDeliveryLease({ + endpointId, + leaseKey: lease.leaseKey, + token: lease.token, + expiresAt, + }) + : await db + .update(chatEndpointLeases) + .set({ expiresAt, updatedAt: new Date() }) + .where( + and( + eq(chatEndpointLeases.endpointId, endpointId), + eq(chatEndpointLeases.leaseKey, lease.leaseKey), + eq(chatEndpointLeases.token, lease.token), + ), + ) + .returning({ id: chatEndpointLeases.id }) + .then((rows) => rows.length > 0); + if (!renewed) leaseOwned = false; + }) + .catch((error) => { + // A failed renewal has unknown durability. Stop this local drain at + // the next delivery boundary so another worker can never overlap + // later turns after the original lease expires or is reclaimed. + leaseOwned = false; + logger.warn( + { endpointId, error: redactError(error) }, + "could not renew external chat conversation lease", + ); + }) + .finally(() => { + renewal = null; + }); + }, + options.conversationLeaseRenewalIntervalMs ?? DELIVERY_LEASE_TTL_MS / 3, + ); + renewTimer.unref?.(); + + try { + for ( + let processed = 0; + processed < DELIVERY_DRAIN_LIMIT; + processed += 1 + ) { + if (!leaseOwned) break; + const delivery = await earliestOpenConversationDelivery( + endpointId, + threadId, + ); + if (!delivery || !deliveryReady(delivery, new Date())) break; + if (delivery.state === "processed") { + if (!(await processInboundWakeup(delivery.id))) break; + continue; + } + if (telegramDeliveryHasZeroMessageId(delivery, endpoint.provider)) { + const filtered = await db + .transaction(async (tx) => { + // Do not round-trip a PostgreSQL microsecond timestamp through + // JS Date for a CAS. Lock the exact receipt, then revalidate the + // current source and claim before changing it. A competing live + // claim is a retry boundary, not permission to settle its work. + const [current] = await tx + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.id, delivery.id), + eq(chatDeliveries.companyId, endpoint.companyId), + eq(chatDeliveries.endpointId, endpoint.id), + ), + ) + .for("update", { noWait: true }); + const filteredAt = new Date(); + if ( + !current || + current.state !== delivery.state || + current.eventKind !== delivery.eventKind || + !["received", "retry", "processing"].includes(current.state) || + normalizedDeliveryThreadId(current) !== threadId || + !deliveryReady(current, filteredAt) || + !telegramDeliveryHasZeroMessageId(current, endpoint.provider) + ) + return false; + await notifyChatDeliveryWork(tx); + const changed = await tx + .update(chatDeliveries) + .set({ + state: "filtered", + nextAttemptAt: null, + processedAt: filteredAt, + updatedAt: filteredAt, + redactedError: + "Telegram message ID zero cannot authorize ordinary work", + }) + .where(eq(chatDeliveries.id, current.id)) + .returning({ id: chatDeliveries.id }); + return changed.length === 1; + }) + .catch((error: unknown) => { + if (isExternalChatWaitAuthorizationContention(error)) + return false; + throw error; + }); + if (!filtered) break; + // Retain original evidence/comments; filtering adds no user content + // and cannot hydrate attachments, invoke commands, or send feedback. + await settleRejectedInboundWakeups(delivery.id); + liveInboundMessages.delete(delivery.id); + continue; + } + if (endpoint.status === "archived" || endpoint.status === "revoked") { + await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: "filtered", + redactedError: "Connection was removed before processing", + processedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(chatDeliveries.id, delivery.id)); + }); + liveInboundMessages.delete(delivery.id); + continue; + } + + if ( + endpoint.provider === "discord" && + discordRootThreadPending(delivery) + ) { + // This is the only recovery path for the narrow crash window after + // pre-admission but before Chat's ordinary callback. A successful + // read proves the root thread exists; an explicit 404/410 proves it + // does not. Ambiguous transport and auth failures remain retryable. + if (await reconcileDiscordRootThread(endpoint, delivery, threadId)) { + continue; + } + break; + } + + const live = liveInboundMessages.get(delivery.id); + try { + if ( + delivery.eventKind === "message_updated" || + delivery.eventKind === "message_deleted" || + delivery.eventKind === "message_restored" + ) { + await processLifecycleDelivery(endpoint, delivery); + } else if (live) { + await processMessage( + endpoint, + live.thread, + live.message, + live.trigger, + false, + live.providerUrl, + live.runtimeContext, + undefined, + delivery.id, + live.receiptReactionSupported, + undefined, + false, + setupDestinationActivationSuppressed(delivery.normalizedEvent), + ); + } else { + if (deliveryContentWasRedacted(delivery.normalizedEvent)) { + const filteredAt = new Date(); + await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: "filtered", + principalId: null, + nextAttemptAt: null, + processedAt: filteredAt, + redactedError: + "Provisional Teams setup reply could not be hydrated from a current provider event", + updatedAt: filteredAt, + }) + .where( + and( + eq(chatDeliveries.id, delivery.id), + inArray(chatDeliveries.state, [ + "received", + "retry", + "processing", + ]), + ), + ); + }); + continue; + } + const endpointRuntime = await runtimeFor(endpoint); + const deliveryRuntimeFence = lifecycleRuntimeFence(delivery); + const currentRuntimeContext = runtimeContexts.get( + endpointRuntime as object, + ); + const thread = endpointRuntime.thread(threadId); + const reconstructed = messageFromDelivery( + delivery, + thread, + endpointRuntime, + ); + if (!reconstructed) { + await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: "failed", + redactedError: "Normalized delivery is incomplete", + updatedAt: new Date(), + }) + .where(eq(chatDeliveries.id, delivery.id)); + }); + continue; + } + await processMessage( + endpoint, + thread, + reconstructed.message, + reconstructed.trigger, + false, + reconstructed.providerUrl, + deliveryRuntimeFence + ? { + ...deliveryRuntimeFence, + endpointRuntime, + } + : currentRuntimeContext, + undefined, + delivery.id, + reconstructed.receiptReactionSupported, + undefined, + false, + reconstructed.suppressSetupDestinationActivation, + ); + } + } catch (error) { + logger.warn( + { + endpointId, + deliveryId: delivery.id, + error: redactError(error), + }, + "external chat delivery will retry in receipt order", + ); + } + + const state = await db + .select({ state: chatDeliveries.state }) + .from(chatDeliveries) + .where(eq(chatDeliveries.id, delivery.id)) + .then((rows) => rows[0]?.state ?? null); + if ( + state === "processed" || + state === "filtered" || + state === "failed" + ) { + if (state !== "processed") + await settleRejectedInboundWakeups(delivery.id); + liveInboundMessages.delete(delivery.id); + continue; + } + // A retry delay or another still-live processing claim is a strict + // head-of-line barrier; later replies may not overtake it. + break; + } + } finally { + clearInterval(renewTimer); + await renewal; + await releaseConversationDeliveryLease({ endpointId, ...lease }); + } + + const next = await earliestOpenConversationDelivery(endpointId, threadId); + if (next?.state === "processed") { + const [ready] = await db + .select({ id: chatDeliveries.id }) + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.id, next.id), + pendingInboundWakeupCondition(true), + ), + ) + .limit(1); + return Boolean(ready); + } + return Boolean(next && deliveryReady(next, new Date())); + } + + async function nextInboundDeliveryAt(): Promise { + const rows = await db.select({ delivery: { + id: chatDeliveries.id, endpointId: chatDeliveries.endpointId, providerEventId: chatDeliveries.providerEventId, + state: chatDeliveries.state, eventKind: chatDeliveries.eventKind, + // Retain scheduling metadata only, not message bodies or attachment data. + normalizedEvent: sql`jsonb_build_object( + 'conversation', ${chatDeliveries.normalizedEvent}->'conversation', + 'lifecycle', ${chatDeliveries.normalizedEvent}->'lifecycle', + 'message', jsonb_build_object( + 'targetProviderEventId', ${chatDeliveries.normalizedEvent}->'message'->'targetProviderEventId', + 'admissionDependencyEventId', ${chatDeliveries.normalizedEvent}->'message'->'admissionDependencyEventId'))`, + }, at: sql`case + when ${chatDeliveries.state} = 'processing' then ${chatDeliveries.updatedAt} + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond' + when ${chatDeliveries.state} = 'processed' then ( + select min(case when a.status = 'processing' then a.updated_at + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond' + else coalesce((a.result->>'retryAt')::timestamptz, now()) end) + from chat_actions a where a.delivery_id = chat_deliveries.id + and a.company_id = chat_deliveries.company_id and a.endpoint_id = chat_deliveries.endpoint_id + and a.kind = 'inbound_wakeup' and a.status in ('issued', 'processing')) + else coalesce(${chatDeliveries.nextAttemptAt}, now()) end` }) + .from(chatDeliveries).where(and( + sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.provider = 'agentmail')`, + // Provider lifecycle receipts can restore an unavailable endpoint. + // The ordinary conversation drain still waits for operator resume. + or(sql`${chatDeliveries.normalizedEvent}->'lifecycle' is not null`, + sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.status in ('paused', 'attention'))`), + or(inArray(chatDeliveries.state, ["received", "retry", "processing"]), + and(eq(chatDeliveries.state, "processed"), pendingInboundWakeupCondition())), + )).orderBy(...conversationDeliveryOrder()); + // Only an ordered conversation head can run. Reaction and provider + // lifecycle receipts are independent; lifecycle edits may select their + // exact admission dependency before the chronologically earliest row. + const byEvent = new Map(rows.map(row => [`${row.delivery.endpointId}:${row.delivery.providerEventId}`, row])); + const conversations = new Set(); + let next: number | null = null; + for (let row of rows) { + const delivery = row.delivery; + const threadId = normalizedDeliveryThreadId(delivery); + if (threadId && !normalizedLifecycleEffect(delivery) && !["reaction_added", "reaction_removed"].includes(delivery.eventKind)) { + const key = conversationDrainKey(delivery.endpointId, threadId); + if (conversations.has(key)) continue; + conversations.add(key); + const dependency = normalizedLifecycleTargetEventId(delivery); + const target = dependency ? byEvent.get(`${delivery.endpointId}:${dependency}`) : undefined; + if (target && ["received", "retry", "processing"].includes(target.delivery.state)) row = target; + } + if (row.at !== null) { + const at = new Date(row.at).getTime(); + next = next === null ? at : Math.min(next, at); + } + } + return next; + } + return { drainConversationDeliveries, nextInboundDeliveryAt }; +} diff --git a/server/src/services/chat-channels/inbound/delivery-ledger.ts b/server/src/services/chat-channels/inbound/delivery-ledger.ts new file mode 100644 index 00000000000..841378675ff --- /dev/null +++ b/server/src/services/chat-channels/inbound/delivery-ledger.ts @@ -0,0 +1,241 @@ +import type { LifecycleRuntimeFence } from "../runtime-types.js"; + +import { + asc, + sql, + type SQL, +} from "drizzle-orm"; + +import { + chatActions, + chatDeliveries, +} from "@paperclipai/db"; + +import { + type ChatProviderLifecycleEffect, +} from "../../chat-provider-lifecycle.js"; + +type DeliveryRow = typeof chatDeliveries.$inferSelect; + +const DELIVERY_PROCESSING_STALE_MS = 60_000; + + function deliveryContentWasRedacted(normalizedEvent: unknown): boolean { + if ( + !normalizedEvent || + typeof normalizedEvent !== "object" || + Array.isArray(normalizedEvent) + ) + return false; + const filtering = (normalizedEvent as { filtering?: unknown }).filtering; + return ( + Boolean(filtering) && + typeof filtering === "object" && + !Array.isArray(filtering) && + (filtering as { contentRetained?: unknown }).contentRetained === false + ); + } + + function setupDestinationActivationSuppressed( + normalizedEvent: unknown, + ): boolean { + if ( + !normalizedEvent || + typeof normalizedEvent !== "object" || + Array.isArray(normalizedEvent) + ) + return false; + const normalized = normalizedEvent as { + admission?: { origin?: unknown }; + admissionAuthorization?: { version?: unknown }; + }; + return ( + normalized.admission?.origin === "provider_confirmed_action" || + normalized.admissionAuthorization?.version === 1 + ); + } + + function normalizedDeliveryThreadId(delivery: Pick): string | null { + const normalized = delivery.normalizedEvent as { + conversation?: { externalThreadId?: unknown }; + }; + return typeof normalized.conversation?.externalThreadId === "string" + ? normalized.conversation.externalThreadId + : null; + } + + function normalizedLifecycleTargetEventId( + delivery: Pick, + ): string | null { + if ( + delivery.eventKind !== "message_updated" && + delivery.eventKind !== "message_deleted" && + delivery.eventKind !== "message_restored" + ) + return null; + const normalized = delivery.normalizedEvent as { + message?: { targetProviderEventId?: unknown; admissionDependencyEventId?: unknown }; + }; + // An edit can also introduce a new addressed request. Process that durable + // admission before waiting for the old, potentially unadmitted source. + if (typeof normalized.message?.admissionDependencyEventId === "string") + return normalized.message.admissionDependencyEventId; + return typeof normalized.message?.targetProviderEventId === "string" + ? normalized.message.targetProviderEventId + : null; + } + + function normalizedLifecycleEffect( + delivery: Pick, + ): ChatProviderLifecycleEffect | null { + const normalized = delivery.normalizedEvent as { lifecycle?: unknown }; + const value = normalized.lifecycle; + if (!value || typeof value !== "object" || Array.isArray(value)) + return null; + const effect = value as Record; + if ( + (effect.provider !== "slack" && + effect.provider !== "github" && + effect.provider !== "discord" && + effect.provider !== "microsoft-teams" && + effect.provider !== "telegram") || + typeof effect.providerEventId !== "string" + ) + return null; + if ( + effect.kind === "endpoint" && + (effect.availability === "available" || + effect.availability === "attention" || + effect.availability === "revoked") && + typeof effect.reason === "string" + ) + return effect as ChatProviderLifecycleEffect; + if ( + effect.kind === "resource" && + (effect.availability === "available" || + effect.availability === "unavailable" || + effect.availability === "removed") && + typeof effect.providerResourceId === "string" && + (effect.previousProviderResourceId === undefined || + typeof effect.previousProviderResourceId === "string") && + typeof effect.resourceType === "string" && + typeof effect.label === "string" + ) + return effect as ChatProviderLifecycleEffect; + return null; + } + + function deliveryReady(delivery: DeliveryRow, now: Date): boolean { + // Accepted input may still own the conversation head through its wake + // outbox. The action dispatcher checks its own retry/claim deadline. + if (delivery.state === "processed") return true; + if (delivery.state === "received") + return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now; + if (delivery.state === "retry") { + return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now; + } + return ( + delivery.state === "processing" && + delivery.updatedAt <= + new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS) + ); + } + + function conversationDeliveryOrder() { + return [ + asc( + sql`coalesce(nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')::timestamptz, ${chatDeliveries.receivedAt})`, + ), + // GitHub timestamps and Telegram message dates have one-second + // resolution. GitHub's numeric comment id and Telegram's message_id + // are monotonic within one conversation, so use them before receipt + // order. The Telegram update_id is a final provider-native tie-breaker + // for unusual payloads that lack a usable message_id. + asc(sql`coalesce( + case + when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence' ~ '^[0-9]+$' + then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence')::numeric + else null + end, + case + when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId' ~ '^[0-9]+$' + then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId')::numeric + else null + end + )`), + asc(sql`case + when ${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId' ~ '^[0-9]+$' + then (${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId')::numeric + else null + end`), + // GitHub exposes no sortable webhook sequence. When an edit and delete + // share its whole-second updated_at value, preserve the only valid + // lifecycle state transition: update before delete. Provider-native + // update ids (Telegram) remain the stronger preceding key. + asc(sql`case ${chatDeliveries.eventKind} + when 'message_updated' then 1 + when 'message_deleted' then 2 + when 'message_restored' then 3 + else 0 + end`), + asc(chatDeliveries.receivedAt), + asc(chatDeliveries.id), + ]; + } + + function pendingInboundWakeupCondition(readyOnly = false): SQL { + return sql`exists ( + select 1 from ${chatActions} + where ${chatActions.deliveryId} = ${chatDeliveries.id} + and ${chatActions.endpointId} = ${chatDeliveries.endpointId} + and ${chatActions.companyId} = ${chatDeliveries.companyId} + and ${chatActions.kind} = 'inbound_wakeup' + and ${chatActions.status} in ('issued', 'processing') + ${ + readyOnly + ? sql`and ( + (${chatActions.status} = 'issued' and (${chatActions.result}->>'retryAt' is null or (${chatActions.result}->>'retryAt')::timestamptz <= now())) + or (${chatActions.status} = 'processing' and ${chatActions.updatedAt} <= ${new Date(Date.now() - DELIVERY_PROCESSING_STALE_MS).toISOString()}::timestamptz) + )` + : sql`` + } + )`; + } + + function lifecycleRuntimeFence( + delivery: DeliveryRow, + ): LifecycleRuntimeFence | null { + const normalized = delivery.normalizedEvent; + if ( + !normalized || + typeof normalized !== "object" || + Array.isArray(normalized) + ) + return null; + const value = (normalized as Record).runtimeContext; + if (!value || typeof value !== "object" || Array.isArray(value)) + return null; + const context = value as Record; + return typeof context.generation === "number" && + Number.isSafeInteger(context.generation) && + typeof context.credentialFingerprint === "string" + ? { + generation: context.generation as number, + credentialFingerprint: context.credentialFingerprint, + } + : null; + } + + function normalizedLifecycleWithRuntimeFence( + delivery: DeliveryRow, + runtimeContext: LifecycleRuntimeFence, + ): Record { + const normalized = + delivery.normalizedEvent && + typeof delivery.normalizedEvent === "object" && + !Array.isArray(delivery.normalizedEvent) + ? (delivery.normalizedEvent as Record) + : {}; + return { ...normalized, runtimeContext }; + } + +export { DELIVERY_PROCESSING_STALE_MS, deliveryContentWasRedacted, setupDestinationActivationSuppressed, normalizedDeliveryThreadId, normalizedLifecycleTargetEventId, normalizedLifecycleEffect, deliveryReady, conversationDeliveryOrder, pendingInboundWakeupCondition, lifecycleRuntimeFence, normalizedLifecycleWithRuntimeFence }; diff --git a/server/src/services/chat-channels/inbound/envelope.ts b/server/src/services/chat-channels/inbound/envelope.ts new file mode 100644 index 00000000000..8bc459e5c73 --- /dev/null +++ b/server/src/services/chat-channels/inbound/envelope.ts @@ -0,0 +1,360 @@ +import { providerResourceType, durableExternalThreadIdentity, canonicalProviderResourceId, chatSurfaceKind } from "./surfaces.js"; + +import type { EndpointRow, InboundRuntimeContext } from "../runtime-types.js"; + +import { + MAX_INBOUND_TEXT, + stableExternalPrincipalId, +} from "../message-lifecycle.js"; +import { + telegramMessageSequence, + telegramMessageSentAt, +} from "../telegram/messages.js"; +import { + slackMessageSentAt, +} from "../slack/messages.js"; +import { + teamsActivitySentAt, +} from "../microsoft-teams/messages.js"; + +import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js"; + +import { chatGitHubConfigurations } from "@paperclipai/db"; +import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared"; + +import { photonReplyReference } from "../../photon/adapter.js"; + +import { + and, + eq, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatEndpointResources, +} from "@paperclipai/db"; +import type { + ChatEventKind, +} from "@paperclipai/shared"; + +import { + normalizeContentType, +} from "../../../attachment-types.js"; +import { + deriveTeamsPersonalRecipient, +} from "../../chat-teams-personal-recipient.js"; + +import { + type ChatSdkMessageCallbackEvent, +} from "../../chat-sdk-runtime.js"; + +import { + githubAttachmentLimitOmissions, +} from "../../chat-github-attachments.js"; + +import { chatProviderConversationUrl } from "../../chat-provider-links.js"; + +import type { + Attachment, + Message, + Thread, +} from "chat"; + +export type InboundEnvelopeInput = { + endpoint: EndpointRow; + thread: Thread; + message: Message; + trigger: ChatSdkMessageCallbackEvent["trigger"]; + receiptReactionSupported: boolean; + recoveredProviderUrl: string | null; + runtimeContext?: InboundRuntimeContext; + providerUpdateId?: number; + admittedDeliveryId: string | null; + suppressSetupDestinationActivation: boolean; + slackSlashControl: boolean; +}; +type InboundEnvelopeOptions = { + runtimeFor: (endpoint: EndpointRow) => Promise; + sanitizeFilename: (value: string | undefined) => string | null; + githubManualMessages: WeakMap; + githubAutomaticMessages: WeakMap; + githubIssueMessages: WeakMap; +}; +export function createInboundEnvelope(db: Db, options: InboundEnvelopeOptions) { + const { runtimeFor, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages } = options; + async function prepareInboundEnvelope(input: InboundEnvelopeInput) { + const { endpoint, thread, message, trigger, receiptReactionSupported, recoveredProviderUrl, runtimeContext, providerUpdateId, admittedDeliveryId, suppressSetupDestinationActivation, slackSlashControl } = input; + const githubAutomatic = endpoint.provider === "github" ? githubAutomaticMessages.get(message) : undefined; + const githubIssue = endpoint.provider === "github" ? githubIssueMessages.get(message) : undefined; + let githubManual = githubManualMessages.get(message); + if (endpoint.provider === "github" && !githubAutomatic && !githubIssue && !githubManual) { + const [config] = await db.select().from(chatGitHubConfigurations).where(eq(chatGitHubConfigurations.endpointId, endpoint.id)); + if (config) { + const repository = thread.channelId.replace(/^github:/, "").toLowerCase(); + const [repo] = await db.select().from(chatEndpointResources).where(and(eq(chatEndpointResources.endpointId, endpoint.id), eq(chatEndpointResources.providerResourceId, repository))); + const repositoryId = String(repo?.metadata?.providerRepositoryId ?? ""); + githubManual = { policy: { ...config.configuration.defaults, ...config.configuration.repositories[repositoryId] }, revision: config.revision, event: trigger === "mention" || message.isMention ? "mention" : "comment" }; + } + } + const providerEventId = `${durableExternalThreadIdentity(thread.id)}:${message.id}`; + const surfaceKind = chatSurfaceKind(endpoint.provider, thread); + const addressed = + endpoint.provider === "imessage-photon" || + trigger === "mention" || + trigger === "direct_message" || + message.isMention === true; + const eventKind: ChatEventKind = + trigger === "direct_message" + ? "direct_message" + : trigger === "mention" + ? "mention" + : "message"; + // A callback remains tied to the runtime that authenticated and parsed it. + // In particular, do not resolve the registry again here: pause or rotation + // may already have removed that instance, and recreating a runtime before + // the durable admission fence would either throw or bind old payload data + // to new credentials. + const endpointRuntime = + runtimeContext?.endpointRuntime ?? (await runtimeFor(endpoint)); + // Non-personal generic files still require a separate Graph grant. Only + // exact inline image resources from the authenticated original activity + // receive a scope-bound, URL-free recovery locator. Reconstructed input + // can retain that proof, but cannot mint one from legacy fetch metadata. + const teamsNonPersonal = + endpoint.provider === "microsoft-teams" && !thread.isDM; + const attachmentSource = { + threadId: thread.id, + messageId: message.id, + runtimeGeneration: runtimeContext?.generation, + credentialFingerprint: runtimeContext?.credentialFingerprint, + principalExternalId: stableExternalPrincipalId( + endpoint.provider, + message.author, + message.raw, + ), + isDirectMessage: thread.isDM, + ...(runtimeContext?.endpointRuntime && !admittedDeliveryId + ? { message } + : {}), + }; + const inlineRecovery = new Map< + Attachment, + ReturnType + >(); + const nativeInboundAttachments = teamsNonPersonal + ? message.attachments.slice(0, 20).flatMap((attachment) => { + const descriptor = endpointRuntime.attachmentRecoveryDescriptor( + attachment, + attachmentSource, + ); + if (descriptor?.locator.kind !== "teams_inline_image") return []; + const restored = endpointRuntime.rehydrateAttachment( + descriptor, + attachmentSource, + ); + if (!restored) return []; + inlineRecovery.set(attachment, descriptor); + return [restored]; + }) + : message.attachments; + const unavailableTeamsReferences = teamsNonPersonal + ? Math.min(message.attachments.length, 20) - + nativeInboundAttachments.length + : 0; + const providerSentAt = + endpoint.provider === "microsoft-teams" + ? teamsActivitySentAt(message.raw) + : endpoint.provider === "telegram" + ? telegramMessageSentAt(message.raw) + : endpoint.provider === "slack" + ? slackMessageSentAt(message.raw, message.id) + : message.metadata.dateSent instanceof Date && + Number.isFinite(message.metadata.dateSent.getTime()) + ? message.metadata.dateSent + : null; + const providerSentAtSource = providerSentAt + ? endpoint.provider === "microsoft-teams" + ? "teams_activity_timestamp" + : endpoint.provider === "telegram" + ? "telegram_message_date" + : endpoint.provider === "slack" + ? "slack_message_ts" + : null + : null; + const providerUrl = + (githubAutomatic + ? `https://github.com/${githubAutomatic.context.repository}/pull/${githubAutomatic.context.pullNumber}` + : githubIssue ? `https://github.com/${githubIssue.context.repository}/issues/${githubIssue.context.issueNumber}` : null) ?? + // Description/review requests and edits have synthetic ledger IDs, not + // native comment IDs. Their verified source URL must not acquire a fake + // issuecomment anchor from that ledger identity. + (endpoint.provider === "github" && githubManual && !receiptReactionSupported ? recoveredProviderUrl : null) ?? + chatProviderConversationUrl({ + provider: endpoint.provider, + providerAccountId: endpoint.providerAccountId, + botUsername: endpoint.botUsername, + threadId: thread.id, + providerMessageId: message.id, + raw: message.raw, + }) ?? recoveredProviderUrl; + // Preserve only the closed personal-recipient proof from the runtime that + // authenticated this original activity. A reconstructed delivery's AAD-only + // author and SDK route caches cannot recreate its Bot Framework recipient. + const teamsPersonalRecipient = + endpoint.provider === "microsoft-teams" && + runtimeContext?.endpointRuntime && + !admittedDeliveryId && + endpoint.providerAccountId && + endpoint.botExternalId + ? deriveTeamsPersonalRecipient(message, { + companyId: endpoint.companyId, + endpointId: endpoint.id, + runtimeGeneration: runtimeContext.generation, + credentialFingerprint: runtimeContext.credentialFingerprint, + tenantId: endpoint.providerAccountId, + botAppId: endpoint.botExternalId, + providerEventId, + threadId: thread.id, + isDirectMessage: thread.isDM, + }) + : null; + const normalized = { + providerEventId, + kind: eventKind, + trigger, + ...(teamsPersonalRecipient ? { teamsPersonalRecipient } : {}), + ...(slackSlashControl + ? { admission: { origin: "slack_slash_control" } } + : suppressSetupDestinationActivation + ? // A slash-command root is provider-confirmed only after an enabled + // destination authorized its transport. Persist that closed origin so + // crash recovery can never reinterpret it as first-time setup traffic + // and undo a later operator reach revocation. + { admission: { origin: "provider_confirmed_action" } } + : {}), + ...(runtimeContext + ? { + runtimeContext: { + credentialFingerprint: runtimeContext.credentialFingerprint, + generation: runtimeContext.generation, + }, + } + : {}), + acknowledgement: { + // Some provider callbacks represent an auditable user command without + // a provider message that can receive a reaction. Persist the + // distinction so a crash/retry cannot mistake the synthetic ledger id + // for a native message id. + receiptReactionSupported, + }, + ...(githubAutomatic ? { githubAutomatic } : {}), + ...(githubIssue ? { githubIssue } : {}), + ...(githubManual ? { githubManual } : {}), + principal: { + // Bot/system authors have already been rejected at this intake boundary. + isBot: false, + externalId: stableExternalPrincipalId( + endpoint.provider, + message.author, + message.raw, + ), + displayName: message.author.fullName, + handle: message.author.userName, + }, + resource: { + type: providerResourceType(endpoint.provider, surfaceKind), + providerResourceId: canonicalProviderResourceId( + endpoint.provider, + thread, + ), + label: thread.channel.name ?? thread.channelId, + }, + conversation: { + externalConversationId: thread.channelId, + externalThreadId: thread.id, + label: thread.channel.name ?? thread.channelId, + isDirectMessage: thread.isDM, + providerUrl, + }, + message: { + providerMessageId: message.id, + ...(endpoint.provider === "github" && typeof (message.raw as { githubSourceMessageId?: unknown })?.githubSourceMessageId === "string" && /^[1-9][0-9]*$/.test((message.raw as { githubSourceMessageId: string }).githubSourceMessageId) + ? { sourceProviderMessageId: (message.raw as { githubSourceMessageId: string }).githubSourceMessageId } : {}), + ...(endpoint.provider === "imessage-photon" + ? { photonReply: photonReplyReference(message.raw) } + : {}), + providerMessageSequence: + endpoint.provider === "telegram" + ? telegramMessageSequence(message.raw) + : null, + providerUpdateId: + endpoint.provider === "telegram" ? (providerUpdateId ?? null) : null, + text: message.text.slice(0, MAX_INBOUND_TEXT), + mentionedBot: message.isMention === true, + providerSentAt: providerSentAt?.toISOString() ?? null, + ...(providerSentAtSource ? { providerSentAtSource } : {}), + ...((endpoint.provider === "github" || + endpoint.provider === "imessage-photon") && + (githubAttachmentLimitOmissions(message) || + Math.max(0, message.attachments.length - 20)) + ? { + attachmentLimitOmissions: + githubAttachmentLimitOmissions(message) || + Math.max(0, message.attachments.length - 20), + } + : {}), + attachments: message.attachments.slice(0, 20).map((attachment) => ({ + name: sanitizeFilename(attachment.name), + mimeType: normalizeContentType(attachment.mimeType), + size: attachment.size ?? null, + recovery: teamsNonPersonal + ? (inlineRecovery.get(attachment) ?? null) + : nativeInboundAttachments.includes(attachment) + ? endpointRuntime.attachmentRecoveryDescriptor( + attachment, + ["telegram", "imessage-photon"].includes(endpoint.provider) + ? attachmentSource + : undefined, + ) + : null, + })), + }, + }; + // Teams RSC can deliver messages from every installed channel, and a + // Telegram bot can receive addressed traffic from a provider-available + // group that the operator has not enabled in Paperclip. Until the reach + // gate admits that destination, retain only identifiers required for + // deduplication and operator-visible filtering—not user text, attachment + // metadata, or an external principal profile. + const redactedDestinationNormalized = { + providerEventId, + kind: eventKind, + trigger, + ...(runtimeContext + ? { + runtimeContext: { + credentialFingerprint: runtimeContext.credentialFingerprint, + generation: runtimeContext.generation, + }, + } + : {}), + resource: { + type: providerResourceType(endpoint.provider, surfaceKind), + providerResourceId: canonicalProviderResourceId( + endpoint.provider, + thread, + ), + }, + conversation: { externalThreadId: thread.id }, + message: { + providerMessageId: message.id, + providerSentAt: providerSentAt?.toISOString() ?? null, + }, + filtering: { contentRetained: false }, + }; + return { githubAutomatic, githubIssue, githubManual, providerEventId, surfaceKind, addressed, eventKind, endpointRuntime, nativeInboundAttachments, unavailableTeamsReferences, providerSentAt, providerUrl, normalized, redactedDestinationNormalized }; + } + return { prepareInboundEnvelope }; +} +export type InboundEnvelope = Awaited["prepareInboundEnvelope"]>>; diff --git a/server/src/services/chat-channels/inbound/lifecycle.ts b/server/src/services/chat-channels/inbound/lifecycle.ts new file mode 100644 index 00000000000..30faad3ab63 --- /dev/null +++ b/server/src/services/chat-channels/inbound/lifecycle.ts @@ -0,0 +1,1086 @@ +import type { EndpointRow, DbTransaction, DbOrTransaction, RuntimeContext, LifecycleRuntimeFence, InboundRuntimeContext } from "../runtime-types.js"; + +import { + MAX_INBOUND_TEXT, + type LifecycleActor, + lifecycleActorFromAuthor, +} from "../message-lifecycle.js"; +import { + telegramMessageSequence, +} from "../telegram/messages.js"; +import { + slackLifecycleFilesDigest, +} from "../slack/messages.js"; +import { + discordLifecycleFilesDigest, +} from "../discord/messages.js"; + +import { notifyChatDeliveryWork } from "../../chat-work-notifications.js"; + +import { + createHash, +} from "node:crypto"; +import { + and, + desc, + eq, + inArray, + lte, + ne, + sql, +} from "drizzle-orm"; + +import { + externalThreadIdentityCondition, + teamsConversationId, +} from "../../chat-control-chronology.js"; + +import type { Db } from "@paperclipai/db"; + +import { + chatConversations, + chatDeliveries, + chatEndpointResources, + chatEndpoints, + chatExternalPrincipals, + chatMessageLinks, +} from "@paperclipai/db"; +import type { + ChatProvider, +} from "@paperclipai/shared"; + +import { normalizeTelegramRichMessage } from "../../chat-telegram-rich-intake.js"; + +import { + notFound, +} from "../../../errors.js"; + +import { + type ChatSdkCallbackEvent, + type ChatSdkMessageUpdatedCallbackEvent, +} from "../../chat-sdk-runtime.js"; + +import { issueService } from "../../issues.js"; + +import type { + MessageDeletedEvent, +} from "chat"; +import { normalizedDeliveryThreadId, lifecycleRuntimeFence } from "./delivery-ledger.js"; +type DeliveryRow = typeof chatDeliveries.$inferSelect; +type MessageLifecycleOptions = { + DELIVERY_PROCESSING_STALE_MS: number; + ORPHAN_FOLLOW_UP_GRACE_MS: number; + ORPHAN_FOLLOW_UP_MAX_ATTEMPTS: number; + INGRESS_REORDER_WINDOW_MS: Partial>; + issuesSvc: ReturnType; + lockCurrentPrincipalAuthorization: (tx: DbOrTransaction, endpoint: EndpointRow, principalId: string) => Promise<{ allowed: boolean; linkedDenied: boolean; userId?: string | null }>; + nonDirectDestinationAllowed: (endpoint: EndpointRow, resource: typeof chatEndpointResources.$inferSelect | null | undefined) => boolean; + redactError: (error: unknown) => string; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; + sameTeamsThreadIdentity: (left: string, right: string) => boolean; + durableExternalThreadIdentity: (threadId: string) => string; + recordCurrentMicrosoftTeamsRoute: (endpoint: EndpointRow, context: InboundRuntimeContext | undefined, threadId: string, raw: unknown, recordAcceptedActivity?: boolean) => Promise; + conversationDrainKey: (endpointId: string, threadId: string) => string; + scheduleConversationDrain: (endpointId: string, threadId: string, scheduledAt?: number) => void; + scheduledConversationDrains: Map; + drainConversationDeliveries: (endpointId: string, threadId: string) => Promise; + ensureDiscordGatewayRuntimeIsCurrent: (endpointId: string, context: RuntimeContext) => Promise; + deferWebhookProcessing?: boolean; +}; +export function createChatMessageLifecycle(db: Db, options: MessageLifecycleOptions) { + const { DELIVERY_PROCESSING_STALE_MS, ORPHAN_FOLLOW_UP_GRACE_MS, ORPHAN_FOLLOW_UP_MAX_ATTEMPTS, INGRESS_REORDER_WINDOW_MS, issuesSvc, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed, redactError, runtimeCallbackEndpoint, sameTeamsThreadIdentity, durableExternalThreadIdentity, recordCurrentMicrosoftTeamsRoute, conversationDrainKey, scheduleConversationDrain, scheduledConversationDrains, drainConversationDeliveries, ensureDiscordGatewayRuntimeIsCurrent } = options; + + + async function conversationForThread(endpointId: string, threadId: string) { + const exact = await db + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.endpointId, endpointId), + eq(chatConversations.externalThreadId, threadId), + ), + ) + .orderBy(desc(chatConversations.sessionGeneration)) + .then((rows) => rows[0] ?? null); + if (exact || !threadId.startsWith("teams:")) return exact; + + // Before Paperclip made Teams route state independent, the adapter + // embedded the mutable Bot Connector serviceUrl in every thread id. Read + // those rows by the immutable conversation id so a regional route change + // continues the same task. New rows use the canonical route-free id. + if (!teamsConversationId(threadId)) return null; + return db + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.endpointId, endpointId), + externalThreadIdentityCondition( + sql`${chatConversations.externalThreadId}`, + threadId, + ), + ), + ) + .orderBy(desc(chatConversations.sessionGeneration)) + .then( + (rows) => + rows.find((row) => + sameTeamsThreadIdentity(row.externalThreadId, threadId), + ) ?? null, + ); + } + + async function recordLifecycleDelivery( + input: { + actor?: LifecycleActor; + isBotMessage?: true; + endpointId: string; + threadId: string; + messageId: string; + eventKind: "message_updated" | "message_deleted" | "message_restored"; + text: string; + providerEventId?: string; + providerMessageSequence?: number | null; + providerSentAt?: string | null; + providerUpdateId?: number | null; + raw?: unknown; + revision?: string | null; + admissionDependencyEventId?: string; + }, + runtimeContext?: RuntimeContext, + githubSourceMapped = false, + ) { + if (!githubSourceMapped && input.threadId.startsWith("github:") && /^[1-9][0-9]*$/.test(input.messageId)) { + const requests = await db.select({ id: chatDeliveries.id, messageId: sql`${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId'` }) + .from(chatDeliveries).innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatDeliveries.endpointId), eq(chatEndpoints.companyId, chatDeliveries.companyId))) + .where(and(eq(chatDeliveries.endpointId, input.endpointId), eq(chatEndpoints.provider, "github"), + inArray(chatDeliveries.eventKind, ["mention", "message", "direct_message"]), + input.admissionDependencyEventId ? ne(chatDeliveries.providerEventId, input.admissionDependencyEventId) : undefined, + sql`${chatDeliveries.normalizedEvent}->'message'->>'sourceProviderMessageId' = ${input.messageId}`, + sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId' = ${input.threadId}`)); + if (requests.length) { + for (const request of requests) await recordLifecycleDelivery({ ...input, messageId: request.messageId, + providerEventId: `${input.providerEventId ?? `${input.eventKind}:${input.threadId}:${input.messageId}:${input.revision ?? "once"}`}:${request.id}` }, runtimeContext, true); + return; + } + } + const durableThreadId = durableExternalThreadIdentity(input.threadId); + const providerEventId = + input.providerEventId ?? + `${input.eventKind}:${durableThreadId}:${input.messageId}:${input.revision ?? "once"}`; + const targetProviderEventId = `${durableThreadId}:${input.messageId}`; + const admission = await db.transaction(async (tx) => { + const endpoint = await tx + .select() + .from(chatEndpoints) + .where(eq(chatEndpoints.id, input.endpointId)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!endpoint) throw notFound("Chat endpoint not found"); + if ( + runtimeContext && + !(await runtimeCallbackEndpoint(tx, endpoint.id, runtimeContext, [ + "verifying", + "active", + ])) + ) + return null; + // A provider has already authenticated this callback. During setup the + // message can belong to the test conversation, so retain its correction. + // Paused or unhealthy connections acknowledge late callbacks without + // mutating the bound task or causing provider retry storms. + if (endpoint.status !== "verifying" && endpoint.status !== "active") + return null; + const reorderWindow = INGRESS_REORDER_WINDOW_MS[endpoint.provider]; + const scheduledAt = + options.deferWebhookProcessing === true && reorderWindow + ? (scheduledConversationDrains.get( + conversationDrainKey(input.endpointId, input.threadId), + ) ?? Date.now() + reorderWindow) + : null; + await notifyChatDeliveryWork(tx); + const [delivery] = await tx + .insert(chatDeliveries) + .values({ + companyId: endpoint.companyId, + endpointId: input.endpointId, + providerEventId, + deduplicationKey: createHash("sha256") + .update(providerEventId) + .digest("hex"), + eventKind: input.eventKind, + normalizedEvent: { + providerEventId, + kind: input.eventKind, + ...(runtimeContext + ? { + runtimeContext: { + credentialFingerprint: runtimeContext.credentialFingerprint, + generation: runtimeContext.generation, + }, + } + : {}), + ...(input.actor ? { principal: input.actor } : {}), + conversation: { externalThreadId: input.threadId }, + message: { + providerMessageId: input.messageId, + targetProviderEventId, + ...(input.admissionDependencyEventId ? { admissionDependencyEventId: input.admissionDependencyEventId } : {}), + ...(endpoint.provider === "github" && input.isBotMessage === true + ? { isBotMessage: true } + : {}), + providerMessageSequence: input.providerMessageSequence ?? null, + providerSentAt: input.providerSentAt ?? null, + providerUpdateId: input.providerUpdateId ?? null, + text: input.text, + }, + }, + state: "received", + nextAttemptAt: scheduledAt ? new Date(scheduledAt) : null, + }) + .onConflictDoNothing() + .returning(); + if (delivery) return { delivery, endpoint, scheduledAt }; + const existing = await tx + .select() + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.endpointId, input.endpointId), + eq(chatDeliveries.providerEventId, providerEventId), + ), + ) + .then((rows) => rows[0] ?? null); + return existing ? { delivery: existing, endpoint, scheduledAt } : null; + }); + if (admission) { + // Lifecycle callbacks can carry a new regional Bot Connector route. Keep + // that provider-authenticated route current even when processing is + // deferred, but deliberately omit the raw activity here. Lifecycle + // callbacks never populate the adapter's user cache; the admitted root + // message already established the principal metadata they must match. + await recordCurrentMicrosoftTeamsRoute( + admission.endpoint, + runtimeContext, + input.threadId, + input.raw, + false, + ); + } + if ( + !admission || + admission.delivery.state === "processed" || + admission.delivery.state === "filtered" || + admission.delivery.state === "failed" + ) + return; + if (options.deferWebhookProcessing === true) { + scheduleConversationDrain( + input.endpointId, + input.threadId, + admission.scheduledAt ?? Date.now(), + ); + return; + } + await drainConversationDeliveries(input.endpointId, input.threadId); + } + + function lifecycleMessageFromDelivery(delivery: DeliveryRow): { + actor: LifecycleActor | null; + isBotMessage: boolean; + messageId: string; + providerSentAt: string | null; + targetProviderEventId: string; + text: string; + threadId: string; + } | null { + if ( + delivery.eventKind !== "message_updated" && + delivery.eventKind !== "message_deleted" && + delivery.eventKind !== "message_restored" + ) + return null; + const normalized = delivery.normalizedEvent as { + principal?: { + displayName?: unknown; + externalId?: unknown; + handle?: unknown; + }; + message?: { + isBotMessage?: unknown; + providerMessageId?: unknown; + providerSentAt?: unknown; + targetProviderEventId?: unknown; + text?: unknown; + }; + }; + const messageId = normalized.message?.providerMessageId; + const targetProviderEventId = normalized.message?.targetProviderEventId; + const providerSentAt = normalized.message?.providerSentAt; + const text = normalized.message?.text; + const threadId = normalizedDeliveryThreadId(delivery); + const externalId = normalized.principal?.externalId; + const actor = + typeof externalId === "string" && externalId + ? { + externalId, + displayName: + typeof normalized.principal?.displayName === "string" + ? normalized.principal.displayName + : externalId, + handle: + typeof normalized.principal?.handle === "string" + ? normalized.principal.handle + : externalId, + } + : null; + return typeof messageId === "string" && + typeof targetProviderEventId === "string" && + typeof text === "string" && + threadId !== null + ? { + actor, + isBotMessage: normalized.message?.isBotMessage === true, + messageId, + providerSentAt: + typeof providerSentAt === "string" ? providerSentAt : null, + targetProviderEventId, + text, + threadId, + } + : null; + } + + async function processLifecycleDelivery( + endpoint: EndpointRow, + candidate: DeliveryRow, + ): Promise { + const lifecycle = lifecycleMessageFromDelivery(candidate); + if (!lifecycle) { + await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: "failed", + redactedError: "Normalized lifecycle delivery is incomplete", + updatedAt: new Date(), + }) + .where(eq(chatDeliveries.id, candidate.id)); + }); + return; + } + + const now = new Date(); + const staleBefore = new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS); + const claimConditions = [ + eq(chatDeliveries.id, candidate.id), + eq(chatDeliveries.state, candidate.state), + ]; + if (candidate.state === "processing") + claimConditions.push(lte(chatDeliveries.updatedAt, staleBefore)); + const [activeDelivery] = await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: "processing", + attempts: candidate.attempts + 1, + nextAttemptAt: null, + redactedError: null, + updatedAt: now, + }) + .where(and(...claimConditions)) + .returning(); + }); + if (!activeDelivery) return; + + try { + await db.transaction(async (tx) => { + const admittedRuntimeContext = lifecycleRuntimeFence(activeDelivery); + const currentEndpoint = admittedRuntimeContext + ? await runtimeCallbackEndpoint( + tx, + endpoint.id, + admittedRuntimeContext, + ["verifying", "active"], + ) + : null; + if (!admittedRuntimeContext || !currentEndpoint) { + const filteredAt = new Date(); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + nextAttemptAt: null, + processedAt: filteredAt, + redactedError: + "Message lifecycle callback belonged to a superseded runtime", + updatedAt: filteredAt, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + let githubBotSourceEdit = false; + if ( + currentEndpoint.provider === "github" && + activeDelivery.eventKind === "message_updated" + ) { + // Bot callbacks can arrive before the outbound publication commits + // its message link. The Boolean comes only from GitHub's verified + // sender/comment-author type, never from comment text. Legacy rows + // lack it, so also recognize exact already-published outbound links. + const linkedMessages = await tx + .select({ direction: chatMessageLinks.direction }) + .from(chatMessageLinks) + .innerJoin( + chatConversations, + and( + eq(chatConversations.id, chatMessageLinks.conversationId), + eq(chatConversations.companyId, chatMessageLinks.companyId), + eq(chatConversations.endpointId, chatMessageLinks.endpointId), + ), + ) + .where( + and( + eq(chatMessageLinks.companyId, currentEndpoint.companyId), + eq(chatMessageLinks.endpointId, currentEndpoint.id), + eq(chatMessageLinks.providerMessageId, lifecycle.messageId), + eq(chatConversations.externalThreadId, lifecycle.threadId), + ), + ); + const outbound = linkedMessages.some( + (row) => row.direction === "outbound", + ); + // A bot editor can change an already-admitted human source. That + // authenticated change revokes the old source, but never admits the + // bot's new content. Unknown/self/outbound updates remain suppressed. + githubBotSourceEdit = + lifecycle.isBotMessage && + !outbound && + linkedMessages.some((row) => row.direction === "inbound"); + if (outbound || (lifecycle.isBotMessage && !githubBotSourceEdit)) { + const filteredAt = new Date(); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + principalId: null, + nextAttemptAt: null, + processedAt: filteredAt, + updatedAt: filteredAt, + redactedError: + "Provider bot or outbound message updates do not create inbound work", + normalizedEvent: { + providerEventId: + activeDelivery.normalizedEvent.providerEventId ?? + activeDelivery.providerEventId, + kind: activeDelivery.eventKind, + conversation: { externalThreadId: lifecycle.threadId }, + message: { + providerMessageId: lifecycle.messageId, + targetProviderEventId: lifecycle.targetProviderEventId, + }, + filtering: { contentRetained: false }, + }, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + } + if ( + activeDelivery.eventKind === "message_updated" || + activeDelivery.eventKind === "message_deleted" || + activeDelivery.eventKind === "message_restored" + ) { + const latestDeleteTransition = await tx + .select({ + eventKind: chatDeliveries.eventKind, + providerSentAt: sql< + string | null + >`nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')`, + }) + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.companyId, activeDelivery.companyId), + eq(chatDeliveries.endpointId, endpoint.id), + inArray(chatDeliveries.eventKind, [ + "message_deleted", + "message_restored", + ]), + eq(chatDeliveries.state, "processed"), + externalThreadIdentityCondition( + sql`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId'`, + lifecycle.threadId, + ), + sql`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId' = ${lifecycle.targetProviderEventId}`, + ), + ) + .orderBy( + desc( + sql`coalesce(nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')::timestamptz, ${chatDeliveries.processedAt})`, + ), + desc(sql`case ${chatDeliveries.eventKind} + when 'message_restored' then 2 + when 'message_deleted' then 1 + else 0 + end`), + desc(chatDeliveries.processedAt), + desc(chatDeliveries.id), + ) + .limit(1) + .then((rows) => rows[0] ?? null); + const latestTransitionAt = latestDeleteTransition?.providerSentAt + ? Date.parse(latestDeleteTransition.providerSentAt) + : Number.NaN; + const incomingTransitionAt = lifecycle.providerSentAt + ? Date.parse(lifecycle.providerSentAt) + : Number.NaN; + const incomingTransitionIsStale = + latestDeleteTransition !== null && + Number.isFinite(latestTransitionAt) && + Number.isFinite(incomingTransitionAt) && + (incomingTransitionAt < latestTransitionAt || + (incomingTransitionAt === latestTransitionAt && + activeDelivery.eventKind === "message_deleted" && + latestDeleteTransition.eventKind === "message_restored")); + const editTargetsDeletedMessage = + activeDelivery.eventKind === "message_updated" && + latestDeleteTransition?.eventKind === "message_deleted"; + const editPredatesRestore = + activeDelivery.eventKind === "message_updated" && + latestDeleteTransition?.eventKind === "message_restored" && + Number.isFinite(latestTransitionAt) && + Number.isFinite(incomingTransitionAt) && + incomingTransitionAt < latestTransitionAt; + if ( + incomingTransitionIsStale || + editTargetsDeletedMessage || + editPredatesRestore + ) { + const filteredAt = new Date(); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "filtered", + normalizedEvent: { + providerEventId: + activeDelivery.normalizedEvent.providerEventId ?? + activeDelivery.providerEventId, + kind: activeDelivery.eventKind, + conversation: { externalThreadId: lifecycle.threadId }, + message: { + providerMessageId: lifecycle.messageId, + targetProviderEventId: lifecycle.targetProviderEventId, + }, + filtering: { contentRetained: false }, + }, + principalId: null, + nextAttemptAt: null, + processedAt: filteredAt, + redactedError: editTargetsDeletedMessage + ? "Message edit arrived after the provider message was deleted" + : "Message lifecycle callback was older than the current provider state", + updatedAt: filteredAt, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + } + const originalDelivery = await tx + .select({ state: chatDeliveries.state }) + .from(chatDeliveries) + .where( + and( + eq(chatDeliveries.endpointId, endpoint.id), + eq( + chatDeliveries.providerEventId, + lifecycle.targetProviderEventId, + ), + ), + ) + .then((rows) => rows[0] ?? null); + const linkedTargets = await tx + .select({ + conversationId: chatConversations.id, + externalThreadId: chatConversations.externalThreadId, + issueId: chatConversations.issueId, + }) + .from(chatMessageLinks) + .innerJoin( + chatConversations, + and( + eq(chatConversations.companyId, chatMessageLinks.companyId), + eq(chatConversations.id, chatMessageLinks.conversationId), + ), + ) + .where( + and( + eq(chatMessageLinks.endpointId, endpoint.id), + eq(chatMessageLinks.providerMessageId, lifecycle.messageId), + eq(chatMessageLinks.direction, "inbound"), + ), + ) + .orderBy(desc(chatConversations.sessionGeneration)) + .then((rows) => rows); + const linkedTarget = + linkedTargets.find((candidate) => + currentEndpoint.provider === "microsoft-teams" + ? sameTeamsThreadIdentity( + candidate.externalThreadId, + lifecycle.threadId, + ) + : candidate.externalThreadId === lifecycle.threadId, + ) ?? null; + + if (!linkedTarget) { + const originalIsOpen = + originalDelivery !== null && + ["received", "processing", "retry"].includes( + originalDelivery.state, + ); + const waitForRoot = + originalIsOpen || + activeDelivery.attempts <= ORPHAN_FOLLOW_UP_MAX_ATTEMPTS; + const resolutionAt = new Date(); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set( + waitForRoot + ? { + state: "retry", + nextAttemptAt: new Date( + resolutionAt.getTime() + ORPHAN_FOLLOW_UP_GRACE_MS, + ), + redactedError: originalIsOpen + ? "Waiting for the original message to finish processing" + : "Waiting briefly for the original message", + updatedAt: resolutionAt, + } + : { + state: "filtered", + nextAttemptAt: null, + redactedError: + "Original message was not admitted to this conversation", + processedAt: resolutionAt, + updatedAt: resolutionAt, + }, + ) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + + const currentConversation = await tx + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, currentEndpoint.companyId), + eq(chatConversations.endpointId, currentEndpoint.id), + eq(chatConversations.id, linkedTarget.conversationId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + const currentResource = + currentConversation?.resourceId && + !currentConversation.isDirectMessage + ? await tx + .select() + .from(chatEndpointResources) + .where( + and( + eq( + chatEndpointResources.companyId, + currentEndpoint.companyId, + ), + eq(chatEndpointResources.endpointId, currentEndpoint.id), + eq( + chatEndpointResources.id, + currentConversation.resourceId, + ), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const retainSourceInvalidation = async ( + principalId: string | null, + reason: string, + ): Promise => { + if ( + !currentConversation || + activeDelivery.eventKind !== "message_updated" + ) + return false; + const invalidatedAt = new Date(); + // Provider authentication proves the exact source changed. The + // editor's current Paperclip rights govern admitting new content, + // not whether a later regrant can resurrect the stale old source. + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: "processed", + conversationId: currentConversation.id, + principalId, + normalizedEvent: { + providerEventId: + activeDelivery.normalizedEvent.providerEventId ?? + activeDelivery.providerEventId, + kind: activeDelivery.eventKind, + runtimeContext: { + generation: admittedRuntimeContext.generation, + credentialFingerprint: + admittedRuntimeContext.credentialFingerprint, + }, + conversation: { externalThreadId: lifecycle.threadId }, + message: { + providerMessageId: lifecycle.messageId, + targetProviderEventId: lifecycle.targetProviderEventId, + providerSentAt: lifecycle.providerSentAt, + }, + filtering: { contentRetained: false }, + }, + nextAttemptAt: null, + processedAt: invalidatedAt, + redactedError: reason, + updatedAt: invalidatedAt, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return true; + }; + if ( + githubBotSourceEdit && + (await retainSourceInvalidation( + null, + "Provider bot edit invalidated an existing source without admitting bot content", + )) + ) + return; + + let lifecyclePrincipalId: string | null = null; + const requiresLifecycleActorAuthorization = + (activeDelivery.eventKind === "message_updated" || + activeDelivery.eventKind === "message_restored") && + (lifecycle.actor !== null || + currentEndpoint.provider === "github" || + currentEndpoint.provider === "telegram" || + currentEndpoint.provider === "microsoft-teams"); + if (requiresLifecycleActorAuthorization) { + const lifecyclePrincipal = lifecycle.actor + ? await tx + .select({ + id: chatExternalPrincipals.id, + isBot: chatExternalPrincipals.isBot, + kind: chatExternalPrincipals.kind, + }) + .from(chatExternalPrincipals) + .where( + and( + eq( + chatExternalPrincipals.companyId, + currentEndpoint.companyId, + ), + eq( + chatExternalPrincipals.provider, + currentEndpoint.provider, + ), + eq( + chatExternalPrincipals.providerAccountId, + currentEndpoint.providerAccountId ?? "unknown", + ), + eq( + chatExternalPrincipals.externalId, + lifecycle.actor.externalId, + ), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const authorization = + lifecyclePrincipal && + lifecyclePrincipal.kind === "user" && + !lifecyclePrincipal.isBot + ? await lockCurrentPrincipalAuthorization( + tx, + currentEndpoint, + lifecyclePrincipal.id, + ) + : null; + const authorizedPrincipalId = authorization?.allowed + ? (lifecyclePrincipal?.id ?? null) + : null; + if (!authorizedPrincipalId) { + if ( + await retainSourceInvalidation( + lifecyclePrincipal?.id ?? null, + "Provider edit invalidation retained without admitting content from an unauthorized actor", + ) + ) + return; + const filteredAt = new Date(); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + principalId: lifecyclePrincipal?.id ?? null, + state: "filtered", + normalizedEvent: { + providerEventId: + activeDelivery.normalizedEvent.providerEventId ?? + activeDelivery.providerEventId, + kind: activeDelivery.eventKind, + conversation: { externalThreadId: lifecycle.threadId }, + message: { + providerMessageId: lifecycle.messageId, + targetProviderEventId: lifecycle.targetProviderEventId, + }, + filtering: { contentRetained: false }, + }, + nextAttemptAt: null, + processedAt: filteredAt, + redactedError: + "External message actor is no longer authorized for lifecycle events", + updatedAt: filteredAt, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + lifecyclePrincipalId = authorizedPrincipalId; + } + + const destinationAllowed = + currentConversation !== null && + ["active", "waiting"].includes(currentConversation.state) && + (currentConversation.isDirectMessage + ? currentEndpoint.allowDirectMessages + : nonDirectDestinationAllowed(currentEndpoint, currentResource)); + if (!destinationAllowed) { + const filteredAt = new Date(); + // A verified deletion or authorized edit still revokes the exactly + // linked source while reach is disabled. Re-enabling access must not + // revive stale files. Keep only the invalidation, never the new + // content: no task comment, attachment import or wake is allowed. + const retainInvalidation = + currentConversation !== null && + (activeDelivery.eventKind === "message_deleted" || + (activeDelivery.eventKind === "message_updated" && + lifecyclePrincipalId !== null)); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + state: retainInvalidation ? "processed" : "filtered", + ...(retainInvalidation + ? { + conversationId: currentConversation.id, + principalId: lifecyclePrincipalId, + } + : {}), + normalizedEvent: { + providerEventId: + activeDelivery.normalizedEvent.providerEventId ?? + activeDelivery.providerEventId, + kind: activeDelivery.eventKind, + ...(retainInvalidation + ? { + runtimeContext: { + generation: admittedRuntimeContext.generation, + credentialFingerprint: + admittedRuntimeContext.credentialFingerprint, + }, + } + : {}), + conversation: { externalThreadId: lifecycle.threadId }, + message: { + providerMessageId: lifecycle.messageId, + targetProviderEventId: lifecycle.targetProviderEventId, + ...(retainInvalidation + ? { providerSentAt: lifecycle.providerSentAt } + : {}), + }, + filtering: { contentRetained: false }, + }, + nextAttemptAt: null, + processedAt: filteredAt, + redactedError: retainInvalidation + ? activeDelivery.eventKind === "message_deleted" + ? "Provider deletion retained without task content while destination access is disabled" + : "Provider edit invalidation retained without task content while destination access is disabled" + : "Destination is no longer enabled for message lifecycle events", + updatedAt: filteredAt, + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + return; + } + + await issuesSvc.addComment( + linkedTarget.issueId, + lifecycle.text, + {}, + { authorType: "system" }, + tx, + ); + await notifyChatDeliveryWork(tx); + await tx + .update(chatDeliveries) + .set({ + conversationId: linkedTarget.conversationId, + principalId: lifecyclePrincipalId, + state: "processed", + processedAt: new Date(), + redactedError: null, + updatedAt: new Date(), + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + }); + } catch (error) { + const terminal = activeDelivery.attempts >= 5; + await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + return tx + .update(chatDeliveries) + .set({ + state: terminal ? "failed" : "retry", + nextAttemptAt: terminal + ? null + : new Date( + Date.now() + + Math.min(60_000, 1000 * 2 ** activeDelivery.attempts), + ), + redactedError: redactError(error), + updatedAt: new Date(), + }) + .where( + and( + eq(chatDeliveries.id, activeDelivery.id), + eq(chatDeliveries.state, "processing"), + ), + ); + }); + throw error; + } + } + + async function handleMessageUpdated( + event: ChatSdkMessageUpdatedCallbackEvent, + runtimeContext: RuntimeContext, + ) { + if ( + event.provider === "discord" && + !(await ensureDiscordGatewayRuntimeIsCurrent( + event.endpointId, + runtimeContext, + )) + ) { + return; + } + const text = `An external message was edited:\n\n${event.message.text.slice(0, MAX_INBOUND_TEXT)}`; + const telegramRich = + event.provider === "telegram" + ? normalizeTelegramRichMessage(event.message.raw) + : null; + await recordLifecycleDelivery( + { + actor: lifecycleActorFromAuthor( + event.provider, + event.message.author, + event.message.raw, + ), + endpointId: event.endpointId, + threadId: event.thread.id, + messageId: event.message.id, + eventKind: "message_updated", + text, + providerMessageSequence: + event.provider === "telegram" + ? telegramMessageSequence(event.message.raw) + : null, + providerSentAt: + ( + event.message.metadata.editedAt ?? event.message.metadata.dateSent + )?.toISOString() ?? null, + raw: event.message.raw, + revision: `${event.message.metadata.editedAt?.toISOString() ?? "unknown"}:${createHash("sha256").update(event.message.text).digest("hex")}${event.provider === "slack" ? `:${slackLifecycleFilesDigest(event.message.raw)}` : event.provider === "discord" ? `:${discordLifecycleFilesDigest(event.message.raw)}` : telegramRich ? `:${telegramRich.mediaDigest}` : ""}`, + }, + runtimeContext, + ); + } + + async function handleMessageDeleted( + event: ChatSdkCallbackEvent, + runtimeContext: RuntimeContext, + ) { + if ( + event.provider === "discord" && + !(await ensureDiscordGatewayRuntimeIsCurrent( + event.endpointId, + runtimeContext, + )) + ) { + return; + } + await recordLifecycleDelivery( + { + endpointId: event.endpointId, + threadId: event.event.threadId, + messageId: event.event.messageId, + eventKind: "message_deleted", + text: "An external message in this conversation was deleted.", + providerSentAt: event.event.deletedAt?.toISOString() ?? null, + raw: event.event.raw, + revision: event.event.deletedAt?.toISOString() ?? null, + }, + runtimeContext, + ); + } + return { conversationForThread, recordLifecycleDelivery, processLifecycleDelivery, handleMessageUpdated, handleMessageDeleted }; +} diff --git a/server/src/services/chat-channels/inbound/resource-admission.ts b/server/src/services/chat-channels/inbound/resource-admission.ts new file mode 100644 index 00000000000..e83ba1c9749 --- /dev/null +++ b/server/src/services/chat-channels/inbound/resource-admission.ts @@ -0,0 +1,113 @@ +import { createChatInboundAuthority } from "./authorization.js"; + +import type { EndpointRow } from "../runtime-types.js"; + +import { + and, + eq, + ne, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatEndpointResources, + chatEndpoints, +} from "@paperclipai/db"; + +import { + notFound, +} from "../../../errors.js"; + +import type { + Thread, +} from "chat"; + +import type { ChatSurfaceKind } from "./surfaces.js"; + +type MessageResourceAdmissionInput = { + endpoint: EndpointRow; + thread: Thread; + addressed: boolean; + suppressSetupDestinationActivation: boolean; + surfaceKind: ChatSurfaceKind; +}; +export function createMessageResourceAdmission(db: Db, options: { ensureResource: ReturnType["ensureResource"] }) { + const { ensureResource } = options; + async function admitMessageResource(input: MessageResourceAdmissionInput) { + const { endpoint, thread, addressed, suppressSetupDestinationActivation, surfaceKind } = input; + const mayEnableSetupDestination = + // Slack channels start enabled. Never use first-test activation to + // override an existing channel the operator explicitly disabled. + endpoint.provider !== "slack" && + endpoint.provider !== "imessage-photon" && + !thread.isDM && + endpoint.status === "verifying" && + addressed && + !suppressSetupDestinationActivation && + !( + endpoint.provider === "microsoft-teams" && + surfaceKind === "linear_group" + ); + const resourceAdmission = mayEnableSetupDestination + ? await db.transaction(async (tx) => { + const currentEndpoint = await tx + .select({ status: chatEndpoints.status }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, endpoint.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!currentEndpoint) throw notFound("Chat endpoint not found"); + const enabledResourceCount = await tx + .select({ count: sql`count(*)::int` }) + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.enabled, true), + eq(chatEndpointResources.availability, "available"), + ne(chatEndpointResources.type, "direct_message"), + ne(chatEndpointResources.type, "group_chat"), + ), + ) + .then((rows) => rows[0]?.count ?? 0); + const enableSetupDestination = + currentEndpoint.status === "verifying" && + enabledResourceCount === 0; + let resource = await ensureResource( + endpoint, + thread, + enableSetupDestination, + tx, + ); + if (enableSetupDestination && !resource.enabled) { + resource = await tx + .update(chatEndpointResources) + .set({ enabled: true, updatedAt: new Date() }) + .where(eq(chatEndpointResources.id, resource.id)) + .returning() + .then((rows) => rows[0] ?? resource); + } + return { enabledResourceCount, resource }; + }) + : { + enabledResourceCount: await db + .select({ count: sql`count(*)::int` }) + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.enabled, true), + eq(chatEndpointResources.availability, "available"), + ne(chatEndpointResources.type, "direct_message"), + ne(chatEndpointResources.type, "group_chat"), + ), + ) + .then((rows) => rows[0]?.count ?? 0), + resource: await ensureResource(endpoint, thread, false), + }; + return resourceAdmission; + } + return { admitMessageResource }; +} diff --git a/server/src/services/chat-channels/inbound/surfaces.ts b/server/src/services/chat-channels/inbound/surfaces.ts new file mode 100644 index 00000000000..7f89d794c6b --- /dev/null +++ b/server/src/services/chat-channels/inbound/surfaces.ts @@ -0,0 +1,180 @@ +import { slackThreadChannelId } from "../slack/messages.js"; + +import type { EndpointRow } from "../runtime-types.js"; + +import { baseTeamsConversationId } from "../microsoft-teams/resources.js"; +import { slackResourceLabelIsFallback } from "../slack/resources.js"; +import { telegramResourceLabelIsFallback } from "../telegram/resources.js"; + +import { + canonicalTeamsThreadId, + teamsConversationId, +} from "../../chat-control-chronology.js"; + +import { + chatEndpointResources, +} from "@paperclipai/db"; +import type { + ChatProvider, +} from "@paperclipai/shared"; + +import type { + Thread, +} from "chat"; + +type ResourceRow = typeof chatEndpointResources.$inferSelect; + + +type ChatSurfaceKind = "direct_message" | "linear_group" | "native_thread"; + +function providerResourceType( + provider: ChatProvider, + surfaceKind: ChatSurfaceKind, +): string { + if (surfaceKind === "direct_message") return "direct_message"; + if (provider === "github") return "repository"; + if (provider === "imessage-photon") return "group_chat"; + if (provider === "discord") return "channel"; + if (provider === "microsoft-teams") + return surfaceKind === "linear_group" ? "group_chat" : "channel"; + // Telegram topics are task/thread boundaries inside one group resource. The + // operator grants access to the chat once rather than having to rediscover + // and enable every topic independently. + if (provider === "telegram") return "chat"; + return "channel"; +} + + +function sameTeamsThreadIdentity(left: string, right: string): boolean { + const leftConversationId = teamsConversationId(left); + const rightConversationId = teamsConversationId(right); + return ( + leftConversationId !== null && + rightConversationId !== null && + leftConversationId === rightConversationId + ); +} + + +function durableExternalThreadIdentity(threadId: string): string { + return canonicalTeamsThreadId(threadId) ?? threadId; +} + +function teamsThreadRootMessageId(threadId: string): string | null { + const conversationId = teamsConversationId(threadId); + if (!conversationId) return null; + return /;messageid=([^;]+)/i.exec(conversationId)?.[1] ?? null; +} + +function canonicalProviderResourceId( + provider: ChatProvider, + thread: Pick, +): string { + if (provider === "slack") return thread.channelId.replace(/^slack:/, ""); + if (provider === "github") + return thread.channelId.replace(/^github:/, "").toLowerCase(); + if (provider === "discord") { + const parts = thread.channelId.split(":"); + return parts[2] ?? thread.channelId; + } + if (provider === "microsoft-teams") { + const conversationId = + teamsConversationId(thread.channelId) ?? teamsConversationId(thread.id); + return conversationId + ? baseTeamsConversationId(conversationId) + : baseTeamsConversationId(thread.channelId); + } + if (provider === "telegram") + return thread.channelId.replace(/^telegram:/, ""); + return thread.channelId; +} + +function providerResourceLabelFromThread( + provider: ChatProvider, + thread: Pick, +): { label: string; fallback: boolean } { + const providerResourceId = canonicalProviderResourceId(provider, thread); + const candidate = thread.channel.name?.trim(); + if (!candidate) return { label: providerResourceId, fallback: true }; + if ( + (provider === "slack" && + slackResourceLabelIsFallback(providerResourceId, candidate)) || + (provider === "telegram" && + telegramResourceLabelIsFallback(providerResourceId, candidate)) + ) { + return { label: providerResourceId, fallback: true }; + } + return { label: candidate, fallback: false }; +} + +/** + * Native channel threads keep one issue forever. Linear surfaces instead + * advance through Paperclip task generations because their provider id is + * reused after a task completes or the user explicitly starts a new task. + */ +function chatSurfaceKind( + provider: ChatProvider, + thread: Thread, +): ChatSurfaceKind { + if (thread.isDM) return "direct_message"; + if (provider === "imessage-photon") return "linear_group"; + if (provider === "telegram") { + return /^telegram:[^:]+:[^:]+$/.test(thread.id) + ? "native_thread" + : "linear_group"; + } + if (provider === "microsoft-teams") { + const conversationId = teamsConversationId(thread.id); + return conversationId?.includes(";messageid=") + ? "native_thread" + : "linear_group"; + } + return "native_thread"; +} + +/** + * Teams group chats are admitted as a class: installation at the provider and + * the endpoint's group-chat toggle are the two gates the operator can + * actually control. Teams channels, and every other non-DM destination, + * retain the explicit per-resource enablement gate shown in Settings. + */ +function nonDirectDestinationAllowed( + endpoint: EndpointRow, + resource: ResourceRow | null | undefined, +): boolean { + if (!resource || resource.availability !== "available") return false; + if ( + endpoint.provider === "microsoft-teams" && + resource.type === "group_chat" + ) { + return endpoint.allowGroupChats; + } + return resource.enabled; +} + +function linearControlCommand(text: string): "new" | "close" | "status" | null { + const match = /^\/(new|close|status)(?:@[\w.-]+)?\s*$/i.exec(text.trim()); + return ( + (match?.[1]?.toLowerCase() as "new" | "close" | "status" | undefined) ?? + null + ); +} + +function actionThreadMatchesConversation( + provider: ChatProvider, + actionThreadId: string, + conversationThreadId: string, +): boolean { + if (provider === "microsoft-teams") { + return sameTeamsThreadIdentity(actionThreadId, conversationThreadId); + } + if (provider !== "slack") return actionThreadId === conversationThreadId; + const actionChannelId = slackThreadChannelId(actionThreadId); + return ( + actionChannelId !== null && + actionChannelId === slackThreadChannelId(conversationThreadId) + ); +} + +export { providerResourceType, sameTeamsThreadIdentity, durableExternalThreadIdentity, teamsThreadRootMessageId, canonicalProviderResourceId, providerResourceLabelFromThread, chatSurfaceKind, nonDirectDestinationAllowed, linearControlCommand, actionThreadMatchesConversation }; +export type { ChatSurfaceKind }; diff --git a/server/src/services/chat-channels/interactions/callbacks.ts b/server/src/services/chat-channels/interactions/callbacks.ts new file mode 100644 index 00000000000..7db0a302b3b --- /dev/null +++ b/server/src/services/chat-channels/interactions/callbacks.ts @@ -0,0 +1,2013 @@ +import { createDiscordQuestionForms } from "../discord/question-forms.js"; + +import { createProviderEffects, type ProviderEffectPayload } from "../provider-effects.js"; +import { createTelegramCallbackNoticeAuthority } from "../telegram/callback-notices.js"; + +import type { EndpointRow, DbTransaction, DbOrTransaction, RuntimeContext, LifecycleRuntimeFence } from "../runtime-types.js"; + +import { notifyChatDeliveryWork, notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { + createHash, +} from "node:crypto"; +import { + and, + eq, + inArray, + isNull, + ne, + or, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatConversations, + chatDeliveries, + chatEndpointResources, + chatEndpoints, + chatExternalPrincipals, + chatMessageLinks, + chatPublications, + issues, + toolConnections, +} from "@paperclipai/db"; +import type { + ChatAdapterCapabilities, + ChatProvider, + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { + readTelegramCallbackProvenance, + telegramCallbackThreadId, + TELEGRAM_PRIVATE_ACTION_UNAVAILABLE, +} from "../../chat-telegram-ephemeral.js"; + +import { + forbidden, +} from "../../../errors.js"; +import { logger } from "../../../middleware/logger.js"; + +import { + type ChatSdkCallbackEvent, +} from "../../chat-sdk-runtime.js"; +import type { + ChatSdkStatePersistence, +} from "../../chat-sdk-state.js"; +import { + logActivity, +} from "../../activity-log.js"; + +import { projectSafeChatPublication } from "../../chat-publication-projection.js"; + +import { + discordQuestionFormCorrectionModal, + discordQuestionFormDenialResponse, + discordQuestionFormThreadId, + isDiscordQuestionFormCorrectionId, + loadDiscordQuestionFormCorrection, +} from "../../chat-discord-question-forms.js"; + +import { + enqueueTerminalIssueInteractionChatPublications, + nativeChatQuestion, + TELEGRAM_CALLBACK_DATA_LIMIT_BYTES, + telegramChatSdkCallbackData, +} from "../../chat-interaction-publications.js"; + +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; + +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; +import { + questionResponseDeliveryService, +} from "../../question-response-delivery.js"; +import { + chatQuestionFormDenialResponse, + chatQuestionFormValidationResponse, + claimChatQuestionFormSubmission, + completeChatQuestionFormSubmission, + isChatQuestionFormOpenActionId, + loadChatQuestionFormSubmissionToken, + resolveChatQuestionFormOpen, + validateChatQuestionFormSubmission, +} from "../../chat-question-forms.js"; + +import type { + ActionEvent, + Author, + ModalResponse, + ModalSubmitEvent, + Thread, +} from "chat"; + +import type { EndpointRecord } from "../runtime-types.js"; +import { createTelegramConfirmationActions } from "../telegram/confirmation-actions.js"; +type ResourceRow = typeof chatEndpointResources.$inferSelect; +type PrincipalResolution = { principal: typeof chatExternalPrincipals.$inferSelect; userId: string | null; linkedDenied: boolean }; +type InteractionCallbacksOptions = { + CAPABILITIES: Record; + MAX_ERROR_TEXT: number; + actionThreadMatchesConversation: (provider: ChatProvider, actionThreadId: string, conversationThreadId: string) => boolean; + nonDirectDestinationAllowed: (endpoint: EndpointRow, resource: ResourceRow | null | undefined) => boolean; + sameTeamsThreadIdentity: (left: string, right: string) => boolean; + ensurePrincipal: (endpoint: EndpointRow, author: Author, raw?: unknown) => Promise; + endpointRecord: (endpointId: string) => Promise; + runtimeCallbackRecord: (endpointId: string, context: RuntimeContext, statuses: EndpointRow["status"][]) => Promise; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; + requireCurrentExternalActionAuthorization: (tx: DbTransaction, input: { conversationId: string; endpointId: string; expectedUserId: string; principalId: string; runtimeContext: LifecycleRuntimeFence }) => Promise; + isExternalActionAuthorizationChange: (error: unknown) => boolean; + recordCurrentMicrosoftTeamsRoute: (endpoint: EndpointRow, context: RuntimeContext, threadId: string, raw: unknown) => Promise; + lockTelegramCallbackNoticeAuthority: ReturnType["lockTelegramCallbackNoticeAuthority"]; + stageProviderEffect: ReturnType["stageProviderEffect"]; + scheduleProviderEffect: (actionId: string, thread?: Thread) => void; + invalidateRuntime: (endpointId: string) => Promise; + redactError: (error: unknown) => string; + persistence: ChatSdkStatePersistence; + questionResponses: Pick, "deliver">; + scheduleMessageProcessing: (task: () => Promise) => void; + processPendingPublications: () => Promise; + confirmationResolutionPersistBarrier?: () => Promise; + questionFormOpenAuthorizationBarrier?: () => Promise; + questionResolutionPersistBarrier?: () => Promise; +}; + +export function createChatInteractionCallbacks(db: Db, options: InteractionCallbacksOptions) { + const { CAPABILITIES, MAX_ERROR_TEXT, actionThreadMatchesConversation, nonDirectDestinationAllowed, sameTeamsThreadIdentity, ensurePrincipal, endpointRecord, runtimeCallbackRecord, runtimeCallbackEndpoint, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, recordCurrentMicrosoftTeamsRoute, lockTelegramCallbackNoticeAuthority, stageProviderEffect, scheduleProviderEffect, invalidateRuntime, redactError, persistence, questionResponses, scheduleMessageProcessing, processPendingPublications } = options; + const { handleDiscordFormCorrection, acceptDiscordQuestionForm } = createDiscordQuestionForms(db, { requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, persistence }); + const { reconcileTerminalConfirmationActions, handleTelegramConfirmation } = createTelegramConfirmationActions(db, { + requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, scheduleMessageProcessing, processPendingPublications, + get confirmationResolutionPersistBarrier() { return options.confirmationResolutionPersistBarrier; }, + }); + + + async function denyExternalAction( + endpoint: EndpointRow, + event: ChatSdkCallbackEvent, + runtimeContext: LifecycleRuntimeFence, + safelyKnown: { + conversationId?: string | null; + principalId?: string | null; + } = {}, + ): Promise { + const telegramCallback = + endpoint.provider === "telegram" && endpoint.botExternalId + ? readTelegramCallbackProvenance(event.telegramCallback, { + companyId: endpoint.companyId, + endpointId: endpoint.id, + botUserId: endpoint.botExternalId, + threadId: event.event.threadId, + messageId: event.event.messageId, + userId: event.event.user.userId, + }) + : null; + const fingerprint = createHash("sha256") + .update( + JSON.stringify( + telegramCallback + ? [ + endpoint.id, + "telegram_callback_denial", + telegramCallback.botUserId, + telegramCallback.callbackId, + ] + : [ + endpoint.id, + event.provider, + event.event.threadId, + event.event.messageId, + event.event.user.userId, + event.event.actionId, + event.event.triggerId ?? null, + ], + ), + ) + .digest("hex"); + const providerEventId = `action-denied:${fingerprint}`; + const processedAt = new Date(); + const actionThread = event.event.thread; + const effectPayload = + actionThread && + typeof actionThread.postEphemeral === "function" && + CAPABILITIES[endpoint.provider].ephemeralMessages + ? ({ + version: 1, + effect: "ephemeral_message", + authorizationMode: "safe_notice", + threadId: event.event.threadId, + userId: event.event.user.userId, + text: "This action is no longer available. Open the linked Paperclip task or ask an operator to link this account.", + fallbackText: "This Paperclip action is no longer available.", + settleDelivery: false, + } as const) + : null; + let effect: typeof chatActions.$inferSelect | null = null; + try { + effect = await db.transaction(async (tx) => { + const currentEndpoint = telegramCallback + ? await runtimeCallbackEndpoint(tx, endpoint.id, runtimeContext, [ + "active", + ]) + : null; + const telegramAuthorizationSha256 = + currentEndpoint && telegramCallback + ? await lockTelegramCallbackNoticeAuthority( + tx, + currentEndpoint, + telegramCallback, + safelyKnown.principalId ?? null, + ) + : null; + await notifyChatDeliveryWork(tx); + const [inserted] = await tx + .insert(chatDeliveries) + .values({ + companyId: endpoint.companyId, + endpointId: endpoint.id, + conversationId: safelyKnown.conversationId ?? null, + principalId: safelyKnown.principalId ?? null, + providerEventId, + deduplicationKey: fingerprint, + eventKind: "action", + normalizedEvent: { + providerEventId, + kind: "action", + authorization: { outcome: "denied" }, + ...(telegramCallback + ? { telegramCallback, telegramAuthorizationSha256 } + : {}), + }, + state: "filtered", + attempts: 1, + redactedError: "External action denied by Paperclip authorization", + processedAt, + updatedAt: processedAt, + }) + .onConflictDoNothing() + .returning(); + const notice: Omit< + ProviderEffectPayload, + "runtimeGeneration" | "credentialFingerprint" + > | null = + telegramCallback && telegramAuthorizationSha256 + ? { + version: 1, + effect: "telegram_callback_notice", + authorizationMode: "safe_notice", + telegramCallback, + telegramAuthorizationSha256, + threadId: telegramCallbackThreadId(telegramCallback), + userId: telegramCallback.receiverUserId, + text: TELEGRAM_PRIVATE_ACTION_UNAVAILABLE, + settleDelivery: false, + } + : effectPayload; + if (!inserted || !notice) return null; + return stageProviderEffect(tx, { + endpoint, + deliveryId: inserted.id, + conversationId: safelyKnown.conversationId ?? null, + principalId: safelyKnown.principalId ?? null, + providerActionId: `provider_effect:${providerEventId}`, + payload: notice, + runtimeContext, + }); + }); + } catch (error) { + logger.warn( + { + endpointId: endpoint.id, + provider: endpoint.provider, + error: redactError(error), + }, + "could not record denied external chat action", + ); + // Provider retries are useful only while the authoritative denial could + // not be recorded. Once the filtered delivery exists, a policy denial + // is a successfully handled webhook and must not be surfaced as a + // callback failure (which the Chat SDK correctly converts to a 503). + throw error; + } + if (effect && actionThread) { + // The callback is acknowledged after the authoritative denial and its + // provider effect are durable. Slow provider I/O runs out of band; a + // redelivery finds the same action and cannot enqueue another notice. + scheduleProviderEffect(effect.id, actionThread); + } + } + + async function recordModalOpenFailure( + endpoint: EndpointRow, + event: ChatSdkCallbackEvent, + runtimeContext: LifecycleRuntimeFence, + safelyKnown: { + attemptActionId?: string | null; + conversationId?: string | null; + principalId?: string | null; + resource?: ResourceRow | null; + }, + error: unknown, + ): Promise { + const disposition = classifyChatPublicationError(error, 1); + const fingerprint = createHash("sha256") + .update( + JSON.stringify([ + endpoint.id, + event.provider, + "modal_open", + event.event.triggerId ?? null, + event.event.threadId, + event.event.messageId, + event.event.user.userId, + event.event.actionId, + ]), + ) + .digest("hex"); + const providerEventId = `modal-open-failed:${fingerprint}`; + const failure = redactError(error); + const processedAt = new Date(); + const actionThread = event.event.thread; + const effectPayload = + actionThread && + typeof actionThread.postEphemeral === "function" && + CAPABILITIES[endpoint.provider].ephemeralMessages + ? ({ + version: 1, + effect: "ephemeral_message", + authorizationMode: "safe_notice", + threadId: event.event.threadId, + userId: event.event.user.userId, + text: "Paperclip could not open this form. Try the action again or open the linked Paperclip task.", + fallbackText: "Paperclip could not open this form.", + settleDelivery: false, + } as const) + : null; + let effect: typeof chatActions.$inferSelect | null = null; + let invalidateEndpointRuntime = false; + try { + effect = await db.transaction(async (tx) => { + await notifyChatDeliveryWork(tx); + const [inserted] = await tx + .insert(chatDeliveries) + .values({ + companyId: endpoint.companyId, + endpointId: endpoint.id, + conversationId: safelyKnown.conversationId ?? null, + principalId: safelyKnown.principalId ?? null, + providerEventId, + deduplicationKey: fingerprint, + eventKind: "action", + normalizedEvent: { + providerEventId, + kind: "action", + modal: { + phase: "open", + outcome: "failed", + disposition: disposition.kind, + }, + }, + // Modal callbacks are acknowledged after this audit row commits. + // They must never enter the replayable inbound-delivery queue. + state: "filtered", + attempts: 1, + redactedError: `Provider modal open failed: ${failure}`.slice( + 0, + MAX_ERROR_TEXT, + ), + processedAt, + updatedAt: processedAt, + }) + .onConflictDoNothing() + .returning(); + if (!inserted) return null; + + if (safelyKnown.attemptActionId) { + await tx + .update(chatActions) + .set({ + status: "failed", + result: { + code: `question_form_open_${disposition.kind}`, + retryable: false, + }, + updatedAt: processedAt, + }) + .where( + and( + eq(chatActions.id, safelyKnown.attemptActionId), + eq(chatActions.status, "processing"), + ), + ); + } + + if (disposition.kind === "endpoint_attention") { + const currentEndpoint = await runtimeCallbackEndpoint( + tx, + endpoint.id, + runtimeContext, + ["active"], + ); + if (currentEndpoint) { + await tx + .update(chatEndpoints) + .set({ + status: "attention", + healthMessage: + "Provider credentials or permissions need attention", + lastError: failure, + updatedAt: processedAt, + }) + .where(eq(chatEndpoints.id, endpoint.id)); + await tx + .update(toolConnections) + .set({ + status: "disabled", + enabled: false, + healthStatus: "degraded", + healthMessage: + "Provider credentials or permissions need attention", + lastError: failure, + healthCheckedAt: processedAt, + updatedAt: processedAt, + }) + .where(eq(toolConnections.id, endpoint.connectionId)); + invalidateEndpointRuntime = true; + } + } else if ( + disposition.kind === "resource_unavailable" && + safelyKnown.resource + ) { + await tx + .update(chatEndpointResources) + .set({ availability: "unavailable", updatedAt: processedAt }) + .where( + and( + eq(chatEndpointResources.companyId, endpoint.companyId), + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.id, safelyKnown.resource.id), + ), + ); + } + + if (!effectPayload) return null; + return stageProviderEffect(tx, { + endpoint, + deliveryId: inserted.id, + conversationId: safelyKnown.conversationId ?? null, + principalId: safelyKnown.principalId ?? null, + providerActionId: `provider_effect:${providerEventId}`, + payload: effectPayload, + runtimeContext, + }); + }); + } catch (auditError) { + logger.warn( + { + endpointId: endpoint.id, + provider: endpoint.provider, + error: redactError(auditError), + }, + "could not record external chat modal-open failure", + ); + throw auditError; + } + if (effect && actionThread) scheduleProviderEffect(effect.id, actionThread); + if (invalidateEndpointRuntime) { + await invalidateRuntime(endpoint.id).catch(() => undefined); + } + } + + async function recordModalSubmitDenial( + endpoint: EndpointRow, + event: ChatSdkCallbackEvent, + input: { + code: string; + conversationId?: string | null; + principalId?: string | null; + }, + database: DbOrTransaction = db, + ): Promise { + const fingerprint = createHash("sha256") + .update( + JSON.stringify([ + endpoint.id, + event.provider, + "modal_submit", + event.event.callbackId, + event.event.viewId, + event.event.user.userId, + input.code, + ]), + ) + .digest("hex"); + const providerEventId = `modal-submit-denied:${fingerprint}`; + const processedAt = new Date(); + try { + await database + .insert(chatDeliveries) + .values({ + companyId: endpoint.companyId, + endpointId: endpoint.id, + conversationId: input.conversationId ?? null, + principalId: input.principalId ?? null, + providerEventId, + deduplicationKey: fingerprint, + eventKind: "action", + normalizedEvent: { + providerEventId, + kind: "action", + authorization: { outcome: "denied" }, + modal: { + phase: "submit", + outcome: "denied", + code: input.code, + }, + }, + state: "filtered", + attempts: 1, + redactedError: "External chat modal submission denied by Paperclip", + processedAt, + updatedAt: processedAt, + }) + .onConflictDoNothing(); + } catch (error) { + logger.warn( + { + endpointId: endpoint.id, + provider: endpoint.provider, + error: redactError(error), + }, + "could not record denied external chat modal submission", + ); + throw error; + } + } + + async function handleAction( + event: ChatSdkCallbackEvent, + runtimeContext: RuntimeContext, + ) { + const record = await runtimeCallbackRecord( + event.endpointId, + runtimeContext, + ["active"], + ); + if (!record) { + throw forbidden("This chat action is not a current Paperclip question"); + } + const deny = async (safelyKnown?: { + conversationId?: string | null; + principalId?: string | null; + }) => { + await denyExternalAction( + record.endpoint, + event, + runtimeContext, + safelyKnown, + ); + if ( + event.provider === "discord" && + event.transport === "discord_gateway" + ) { + // Discord renders deferUpdate as a successful click. After the denial + // audit is durable, surface a transport-only sentinel so the Gateway + // adapter can withhold that misleading acknowledgement. The runtime, + // not raw JSON or discord.js methods, supplies the ingress context. + throw Object.assign( + new Error("Discord Gateway action was not admitted by Paperclip"), + { code: "chat_discord_gateway_action_rejected" }, + ); + } + }; + if ( + record.endpoint.status !== "active" || + record.endpoint.provider !== event.provider || + record.endpoint.capabilities.actions !== true + ) { + return deny(); + } + const principal = await ensurePrincipal( + record.endpoint, + event.event.user, + event.event.raw, + ); + // Executable external actions are deliberately stricter than ordinary + // sponsored-guest messages. They require a current endpoint-scoped link to + // an active non-viewer Paperclip member, and never run as a guest sponsor. + if ( + principal.linkedDenied || + !principal.userId || + principal.principal.kind !== "user" || + principal.principal.isBot + ) { + return deny({ principalId: principal.principal.id }); + } + const isCorrection = isDiscordQuestionFormCorrectionId( + event.event.actionId, + ); + const correction = + isCorrection && + event.provider === "discord" && + event.transport === "discord_gateway" + ? await loadDiscordQuestionFormCorrection( + persistence, + { + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + }, + event.event.actionId, + { + principalId: principal.principal.id, + userId: principal.userId, + externalUserId: event.event.user.userId, + }, + event.event.threadId, + ) + : null; + if ( + isCorrection && + (!correction || + discordQuestionFormThreadId(event.event.raw) !== event.event.threadId) + ) + return deny({ principalId: principal.principal.id }); + // The private correction handle selects the original publication, never + // the new ephemeral error message. It does not bypass any current source, + // destination, actor, open-token, or pending-interaction check below. + const sourceActionId = correction?.openActionId ?? event.event.actionId; + const sourceMessageId = + correction?.providerMessageId ?? event.event.messageId; + const actionBinding = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, record.endpoint.companyId), + eq(chatActions.endpointId, record.endpoint.id), + inArray(chatActions.kind, [ + "question_answer", + "question_form_open", + "confirmation_response", + ]), + eq(chatActions.providerActionId, sourceActionId), + ), + ) + .then((rows) => (rows.length === 1 ? rows[0]! : null)); + const originalPublicationId = actionBinding?.payload.publicationId; + const actionInteractionId = actionBinding?.payload.interactionId; + if ( + correction && + (actionBinding?.kind !== "question_form_open" || + actionBinding.conversationId !== correction.conversationId || + originalPublicationId !== correction.publicationId || + actionInteractionId !== correction.interactionId || + actionBinding.payload.formActionId !== correction.submitActionId) + ) + return deny({ principalId: principal.principal.id }); + if ( + !actionBinding?.conversationId || + typeof originalPublicationId !== "string" || + typeof actionInteractionId !== "string" + ) { + return deny({ principalId: principal.principal.id }); + } + // The opaque provider action is the immutable link to the original + // publication. Resolve the conversation from that token, then verify the + // provider's thread independently; never trust a callback message id to + // select a Paperclip task. + const conversation = await db + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, record.endpoint.companyId), + eq(chatConversations.endpointId, record.endpoint.id), + eq(chatConversations.id, actionBinding.conversationId), + inArray(chatConversations.state, ["active", "waiting"]), + ), + ) + .then((rows) => rows[0] ?? null); + if ( + !conversation || + !actionThreadMatchesConversation( + event.provider, + event.event.threadId, + conversation.externalThreadId, + ) + ) { + return deny({ principalId: principal.principal.id }); + } + const safelyKnown = { + conversationId: conversation.id, + principalId: principal.principal.id, + }; + const resource = conversation.resourceId + ? await db + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.companyId, record.endpoint.companyId), + eq(chatEndpointResources.endpointId, record.endpoint.id), + eq(chatEndpointResources.id, conversation.resourceId), + ), + ) + .then((rows) => rows[0] ?? null) + : null; + const resourceAllowed = conversation.isDirectMessage + ? record.endpoint.allowDirectMessages + : nonDirectDestinationAllowed(record.endpoint, resource); + if (!resourceAllowed) return deny(safelyKnown); + + let originalPublication = await db + .select() + .from(chatPublications) + .where( + and( + eq(chatPublications.companyId, record.endpoint.companyId), + eq(chatPublications.endpointId, record.endpoint.id), + eq(chatPublications.conversationId, conversation.id), + eq(chatPublications.issueId, conversation.issueId), + eq(chatPublications.id, originalPublicationId), + inArray(chatPublications.state, ["published", "delivery_unknown"]), + or( + isNull(chatPublications.providerMessageId), + eq(chatPublications.providerMessageId, sourceMessageId), + ), + ), + ) + .then((rows) => rows[0] ?? null); + if (!originalPublication) return deny(safelyKnown); + if ( + correction && + (originalPublication.state !== "published" || + originalPublication.providerMessageId !== correction.providerMessageId) + ) + return deny(safelyKnown); + if (!originalPublication.providerMessageId) { + const reconciledPublication = await db.transaction(async (tx) => { + const current = await tx + .select() + .from(chatPublications) + .where(eq(chatPublications.id, originalPublicationId)) + .for("update") + .then((rows) => rows[0] ?? null); + if ( + !current || + current.companyId !== record.endpoint.companyId || + current.endpointId !== record.endpoint.id || + current.conversationId !== conversation.id || + current.issueId !== conversation.issueId || + !["published", "delivery_unknown"].includes(current.state) || + (current.providerMessageId !== null && + current.providerMessageId !== event.event.messageId) + ) { + return null; + } + const occupied = await tx + .select({ publicationId: chatMessageLinks.publicationId }) + .from(chatMessageLinks) + .where( + and( + eq(chatMessageLinks.endpointId, record.endpoint.id), + eq(chatMessageLinks.conversationId, conversation.id), + eq(chatMessageLinks.providerMessageId, event.event.messageId), + eq(chatMessageLinks.direction, "outbound"), + ), + ) + .then((rows) => rows[0] ?? null); + if (occupied && occupied.publicationId !== current.id) return null; + const reconciledAt = new Date(); + await notifyChatPublicationWork(tx); + const [reconciled] = await tx + .update(chatPublications) + .set({ + state: "published", + providerMessageId: event.event.messageId, + publishedAt: current.publishedAt ?? reconciledAt, + nextAttemptAt: null, + redactedError: null, + updatedAt: reconciledAt, + }) + .where( + and( + eq(chatPublications.id, current.id), + inArray(chatPublications.state, [ + "published", + "delivery_unknown", + ]), + isNull(chatPublications.providerMessageId), + ), + ) + .returning(); + if (!reconciled) return null; + await tx + .insert(chatMessageLinks) + .values({ + companyId: current.companyId, + endpointId: current.endpointId, + conversationId: current.conversationId, + publicationId: current.id, + commentId: current.commentId, + providerMessageId: event.event.messageId, + direction: "outbound", + }) + .onConflictDoNothing(); + const linked = await tx + .select({ publicationId: chatMessageLinks.publicationId }) + .from(chatMessageLinks) + .where( + and( + eq(chatMessageLinks.endpointId, record.endpoint.id), + eq(chatMessageLinks.conversationId, conversation.id), + eq(chatMessageLinks.providerMessageId, event.event.messageId), + eq(chatMessageLinks.direction, "outbound"), + ), + ) + .then((rows) => rows[0] ?? null); + return linked?.publicationId === reconciled.id ? reconciled : null; + }); + if (!reconciledPublication) return deny(safelyKnown); + originalPublication = reconciledPublication; + } + const messageBinding = await db + .select({ + link: chatMessageLinks, + linkedPublication: chatPublications, + }) + .from(chatMessageLinks) + .innerJoin( + chatPublications, + and( + eq(chatPublications.companyId, chatMessageLinks.companyId), + eq(chatPublications.endpointId, chatMessageLinks.endpointId), + eq(chatPublications.conversationId, chatMessageLinks.conversationId), + eq(chatPublications.id, chatMessageLinks.publicationId), + ), + ) + .where( + and( + eq(chatMessageLinks.companyId, record.endpoint.companyId), + eq(chatMessageLinks.endpointId, record.endpoint.id), + eq(chatMessageLinks.conversationId, conversation.id), + eq(chatMessageLinks.providerMessageId, sourceMessageId), + eq(chatMessageLinks.direction, "outbound"), + eq(chatPublications.issueId, conversation.issueId), + eq(chatPublications.state, "published"), + eq(chatPublications.providerMessageId, sourceMessageId), + ), + ) + .then((rows) => (rows.length === 1 ? rows[0]! : null)); + if (!messageBinding) return deny(safelyKnown); + const linkedPayload = messageBinding.linkedPublication + .payload as SafeChatPublicationPayload; + const linkStillAuthoritative = + messageBinding.link.publicationId === originalPublicationId || + (messageBinding.linkedPublication.idempotencyKey === + `interaction-resolution:${actionInteractionId}:${record.endpoint.id}` && + linkedPayload.interactionId === actionInteractionId); + if (!originalPublication || !linkStillAuthoritative) { + return deny(safelyKnown); + } + const issued = { + publication: originalPublication, + link: messageBinding.link, + conversation, + }; + + const payload = issued.publication.payload as SafeChatPublicationPayload; + if (event.provider === "telegram") { + const raw = event.event.raw; + const callbackData = + raw && typeof raw === "object" && "data" in raw + ? (raw as { data?: unknown }).data + : null; + if ( + typeof callbackData !== "string" || + Buffer.byteLength(callbackData, "utf8") > + TELEGRAM_CALLBACK_DATA_LIMIT_BYTES || + callbackData !== telegramChatSdkCallbackData(event.event.actionId) || + event.event.value !== undefined + ) { + return deny(safelyKnown); + } + } + if ( + !payload.interactionId || + (!correction && + event.provider !== "telegram" && + event.event.value !== payload.interactionId) || + !payload.card?.actions?.some( + (action) => + action.type === "callback" && action.actionId === sourceActionId, + ) + ) { + return deny(safelyKnown); + } + + const interaction = ( + await issueThreadInteractionService(db).listForIssue(conversation.issueId) + ).find((candidate) => candidate.id === payload.interactionId); + if ( + !interaction || + interaction.companyId !== record.endpoint.companyId || + interaction.issueId !== conversation.issueId + ) { + return deny(safelyKnown); + } + if (interaction.status !== "pending") { + if (correction) return deny(safelyKnown); + const completedAction = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, record.endpoint.companyId), + eq(chatActions.endpointId, record.endpoint.id), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.providerActionId, event.event.actionId), + eq(chatActions.principalId, principal.principal.id), + eq(chatActions.status, "processed"), + ), + ) + .then((rows) => rows[0] ?? null); + const completedPayload = completedAction?.payload; + const completedResult = completedAction?.result; + const completedResultMatches = + completedResult?.interactionId === interaction.id && + completedResult?.interactionStatus === interaction.status && + (interaction.kind !== "request_confirmation" || + completedResult.code === + "confirmation_resolution_committed_by_provider"); + if ( + completedAction && + completedResultMatches && + completedPayload?.publicationId === issued.publication.id && + completedPayload?.interactionId === interaction.id && + completedPayload?.messageId === event.event.messageId && + completedPayload?.actionId === event.event.actionId + ) { + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.threadId, + event.event.raw, + ); + return; + } + return deny(safelyKnown); + } + if (interaction.kind === "request_confirmation") { + return handleTelegramConfirmation({ record, event, runtimeContext, conversation, principal: { ...principal, userId: principal.userId }, issued, interaction, deny, safelyKnown }); + } + if (interaction.kind !== "ask_user_questions") { + return deny(safelyKnown); + } + if (isChatQuestionFormOpenActionId(sourceActionId)) { + if ( + event.provider !== "slack" && + event.provider !== "microsoft-teams" && + !( + event.provider === "discord" && + event.transport === "discord_gateway" && + record.endpoint.capabilities.modals === true + ) + ) { + return deny(safelyKnown); + } + await options.questionFormOpenAuthorizationBarrier?.(); + let providerAttempted = false; + let attemptActionId: string | null = null; + try { + const attemptProviderActionId = event.event.triggerId + ? `question_form_open_attempt:${createHash("sha256") + .update( + JSON.stringify([ + record.endpoint.id, + event.provider, + event.event.triggerId, + ]), + ) + .digest("hex")}` + : null; + const preflight = await db.transaction(async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + const resolved = await resolveChatQuestionFormOpen(tx, { + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + conversationId: conversation.id, + interaction, + openActionId: sourceActionId, + }); + if (!resolved || resolved.publicationId !== issued.publication.id) { + return { kind: "denied" as const }; + } + if (!attemptProviderActionId) { + return { + kind: "ready" as const, + attemptActionId: null, + resolved, + }; + } + const [attempt] = await tx + .insert(chatActions) + .values({ + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + conversationId: conversation.id, + principalId: principal.principal.id, + kind: "question_form_open_attempt", + providerActionId: attemptProviderActionId, + payload: { version: 1 }, + status: "processing", + }) + .onConflictDoNothing() + .returning({ id: chatActions.id }); + return attempt + ? { + kind: "ready" as const, + attemptActionId: attempt.id, + resolved, + } + : { kind: "duplicate" as const }; + }); + if (preflight.kind === "denied") return deny(safelyKnown); + if (preflight.kind === "duplicate") { + if (event.provider === "discord") { + // A prior attempt may have consumed Discord's single response. + // A fresh handler has no showModal closure state for that attempt; + // do not turn its processing/failed receipt into a success ACK. + throw Object.assign( + new Error("Discord modal response was already attempted"), + { code: "chat_discord_gateway_modal_response_indeterminate" }, + ); + } + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.threadId, + event.event.raw, + ); + return; + } + attemptActionId = preflight.attemptActionId; + + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.threadId, + event.event.raw, + ); + + // A provider modal open can take seconds and can outlive Slack's + // trigger. The final Paperclip authorization snapshot above commits + // before transport starts so provider latency never holds endpoint, + // destination, identity-link, or membership row locks. + const modal = correction + ? discordQuestionFormCorrectionModal( + preflight.resolved.modal, + correction, + ) + : preflight.resolved.modal; + if (!modal) return deny(safelyKnown); + providerAttempted = true; + const opened = await event.event.openModal(modal); + if (!opened) { + await recordModalOpenFailure( + record.endpoint, + event, + runtimeContext, + { ...safelyKnown, attemptActionId, resource }, + new Error("Provider did not confirm the modal open"), + ); + if (event.provider === "discord") return deny(safelyKnown); + return; + } + if (attemptActionId) { + const [settledAttempt] = await db + .update(chatActions) + .set({ + status: "processed", + result: { code: "question_form_opened" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, attemptActionId), + eq(chatActions.status, "processing"), + ), + ) + .returning({ id: chatActions.id }); + if (!settledAttempt) { + throw new Error( + "Question-form modal ownership changed after provider acceptance", + ); + } + } + } catch (error) { + if (isExternalActionAuthorizationChange(error)) { + return deny(safelyKnown); + } + if (!providerAttempted) throw error; + await recordModalOpenFailure( + record.endpoint, + event, + runtimeContext, + { ...safelyKnown, attemptActionId, resource }, + error, + ); + if (event.provider === "discord") return deny(safelyKnown); + // The provider attempt is now a durable, non-replayable Activity row. + // Acknowledge the callback so Chat SDK does not turn a definite denial + // or an ambiguous modal open into a duplicate retry loop. + return; + } + return; + } + + const action = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, record.endpoint.companyId), + eq(chatActions.endpointId, record.endpoint.id), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.kind, "question_answer"), + eq(chatActions.providerActionId, event.event.actionId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!action || action.status !== "issued") return deny(safelyKnown); + const tokenPayload = action.payload; + const tokenExpiresAt = + typeof tokenPayload.expiresAt === "string" + ? Date.parse(tokenPayload.expiresAt) + : Number.NaN; + if (!Number.isFinite(tokenExpiresAt) || tokenExpiresAt <= Date.now()) { + await db + .update(chatActions) + .set({ + status: "expired", + result: { code: "question_action_token_expired" }, + updatedAt: new Date(), + }) + .where( + and(eq(chatActions.id, action.id), eq(chatActions.status, "issued")), + ); + return deny(safelyKnown); + } + if ( + tokenPayload.version !== 1 || + tokenPayload.publicationId !== issued.publication.id || + tokenPayload.interactionId !== payload.interactionId || + typeof tokenPayload.questionId !== "string" || + typeof tokenPayload.optionId !== "string" + ) { + return deny(safelyKnown); + } + + const question = nativeChatQuestion(interaction); + const option = question?.options.find( + (candidate) => candidate.id === tokenPayload.optionId, + ); + if (!question || question.id !== tokenPayload.questionId || !option) + return deny(safelyKnown); + + await options.questionResolutionPersistBarrier?.(); + + const issue = await db + .select({ + id: issues.id, + companyId: issues.companyId, + status: issues.status, + }) + .from(issues) + .where( + and( + eq(issues.companyId, record.endpoint.companyId), + eq(issues.id, conversation.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!issue) return deny(safelyKnown); + try { + const answered = await issueThreadInteractionService(db).answerQuestions( + issue, + interaction.id, + { + answers: [{ questionId: question.id, optionIds: [option.id] }], + }, + { userId: principal.userId }, + { + beforeResolveInTransaction: async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + }, + afterResolveInTransaction: async (tx, resolved) => { + const [completed] = await tx + .update(chatActions) + .set({ + principalId: principal.principal.id, + status: "processed", + payload: { + ...tokenPayload, + messageId: event.event.messageId, + actionId: event.event.actionId, + value: event.event.value ?? null, + }, + result: { + interactionId: resolved.id, + interactionStatus: resolved.status, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "issued"), + ), + ) + .returning({ id: chatActions.id }); + if (!completed) { + throw new Error( + "External question action ownership changed before commit", + ); + } + await tx + .update(chatActions) + .set({ + status: "expired", + result: { code: "interaction_resolved_by_sibling_action" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.companyId, issue.companyId), + eq(chatActions.endpointId, record.endpoint.id), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.kind, "question_answer"), + eq(chatActions.status, "issued"), + ne(chatActions.id, action.id), + eq( + sql`${chatActions.payload}->>'interactionId'`, + resolved.id, + ), + ), + ); + await logActivity(tx as unknown as Db, { + companyId: issue.companyId, + actorType: "user", + actorId: principal.userId!, + action: "issue.thread_interaction_answered", + entityType: "issue", + entityId: issue.id, + details: { + source: "external_chat", + endpointId: record.endpoint.id, + provider: record.endpoint.provider, + conversationId: conversation.id, + publicationId: issued.publication.id, + providerMessageId: event.event.messageId, + interactionId: resolved.id, + interactionKind: resolved.kind, + interactionStatus: resolved.status, + resolutionActorKind: "user", + answeredQuestionCount: 1, + }, + }); + await notifyChatPublicationWork(tx); + await tx + .insert(chatPublications) + .values({ + companyId: issue.companyId, + endpointId: record.endpoint.id, + conversationId: conversation.id, + issueId: issue.id, + idempotencyKey: `interaction-resolution:${resolved.id}:${record.endpoint.id}`, + payload: projectSafeChatPublication({ + classification: "external", + source: "issue_interaction", + text: `Answered: ${option.label}.`, + interaction: { + id: resolved.id, + card: { + kind: "question", + title: question.prompt, + body: `Answered: ${option.label}.`, + actions: [], + }, + }, + }), + state: "pending", + }) + .onConflictDoNothing(); + }, + }, + ); + scheduleMessageProcessing(async () => { + await processPendingPublications(); + }); + // The interaction, provider token, and response-delivery receipt are + // already durable. A provider callback must not wait for agent startup; + // drain the outbox after acknowledging the provider callback. + scheduleMessageProcessing(async () => { + await questionResponses.deliver(answered.id); + }); + } catch (error) { + if (isExternalActionAuthorizationChange(error)) { + return deny(safelyKnown); + } + // `answerQuestions` commits the interaction and this action in one + // transaction, then performs a best-effort issue timestamp touch. If + // that post-commit touch fails, the provider callback is nevertheless + // durably complete. Do not corrupt the processed token to `failed` or + // return a retryable webhook response that turns Telegram's redelivery + // into a misleading stale-action notice. + try { + const raceResult = await db.transaction(async (tx) => { + const currentAction = await tx + .select({ status: chatActions.status }) + .from(chatActions) + .where(eq(chatActions.id, action.id)) + .then((rows) => rows[0] ?? null); + const currentInteraction = ( + await issueThreadInteractionService( + tx as unknown as Db, + ).listForIssue(issue.id) + ).find((candidate) => candidate.id === interaction.id); + if (currentInteraction && currentInteraction.status !== "pending") { + await tx + .update(chatActions) + .set({ + status: "expired", + result: { code: "interaction_resolved_elsewhere" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + inArray(chatActions.status, ["issued", "processing"]), + ), + ); + await enqueueTerminalIssueInteractionChatPublications( + tx, + currentInteraction, + ); + return { actionStatus: currentAction?.status, settledRace: true }; + } + return { actionStatus: currentAction?.status, settledRace: false }; + }); + if (raceResult.actionStatus === "processed") { + logger.warn( + { + endpointId: record.endpoint.id, + interactionId: interaction.id, + error: redactError(error), + }, + "external chat question resolved but post-commit follow-up failed", + ); + return; + } + if (raceResult.settledRace) { + scheduleMessageProcessing(async () => { + await processPendingPublications(); + }); + return; + } + } catch (recoveryError) { + logger.warn( + { + endpointId: record.endpoint.id, + interactionId: interaction.id, + error: redactError(recoveryError), + }, + "could not reconcile an external chat question after resolution failed", + ); + } + throw error; + } + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.threadId, + event.event.raw, + ); + } + + async function handleModalSubmit( + event: ChatSdkCallbackEvent, + runtimeContext: RuntimeContext, + ): Promise { + const denialError = () => + forbidden("This chat form is not a current Paperclip question"); + const record = await runtimeCallbackRecord( + event.endpointId, + runtimeContext, + ["active"], + ); + if ( + !record || + record.endpoint.status !== "active" || + record.endpoint.provider !== event.provider || + record.endpoint.capabilities.modals !== true || + (event.provider !== "slack" && + event.provider !== "microsoft-teams" && + !( + event.provider === "discord" && event.transport === "discord_gateway" + )) + ) { + const staleRecord = + record ?? + (await endpointRecord(event.endpointId).then((candidate) => + candidate?.endpoint.provider === event.provider ? candidate : null, + )); + if (staleRecord) { + await recordModalSubmitDenial(staleRecord.endpoint, event, { + code: "chat_modal_runtime_not_current", + }); + } + return event.provider === "discord" + ? discordQuestionFormDenialResponse() + : chatQuestionFormDenialResponse(); + } + const deny = async ( + code: string, + safelyKnown: { + conversationId?: string | null; + principalId?: string | null; + } = {}, + payload?: Parameters[0], + ): Promise => { + await recordModalSubmitDenial(record.endpoint, event, { + code, + ...safelyKnown, + }); + return event.provider === "discord" + ? discordQuestionFormDenialResponse() + : chatQuestionFormDenialResponse(payload); + }; + const loaded = await loadChatQuestionFormSubmissionToken(db, { + callbackId: event.event.callbackId, + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + includeProcessed: true, + }); + if (!loaded) return deny("chat_modal_token_not_current"); + const conversation = await db + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, record.endpoint.companyId), + eq(chatConversations.endpointId, record.endpoint.id), + eq(chatConversations.id, loaded.conversationId), + inArray(chatConversations.state, ["active", "waiting"]), + ), + ) + .then((rows) => rows[0] ?? null); + if ( + !conversation || + (event.provider === "discord" && + discordQuestionFormThreadId(event.event.raw) !== + conversation.externalThreadId) || + (event.event.relatedThread && + (event.provider === "microsoft-teams" + ? !sameTeamsThreadIdentity( + event.event.relatedThread.id, + conversation.externalThreadId, + ) + : event.event.relatedThread.id !== conversation.externalThreadId)) + ) { + return deny( + "chat_modal_conversation_not_current", + { + conversationId: conversation?.id ?? null, + }, + loaded.payload, + ); + } + const principal = await ensurePrincipal( + record.endpoint, + event.event.user, + event.event.raw, + ); + if ( + principal.linkedDenied || + !principal.userId || + principal.principal.kind !== "user" || + principal.principal.isBot + ) { + return deny( + "chat_modal_principal_not_authorized", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const resource = conversation.resourceId + ? await db + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.companyId, record.endpoint.companyId), + eq(chatEndpointResources.endpointId, record.endpoint.id), + eq(chatEndpointResources.id, conversation.resourceId), + ), + ) + .then((rows) => rows[0] ?? null) + : null; + const resourceAllowed = conversation.isDirectMessage + ? record.endpoint.allowDirectMessages + : nonDirectDestinationAllowed(record.endpoint, resource); + if (!resourceAllowed) { + return deny( + "chat_modal_destination_not_authorized", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + + const originalPublication = await db + .select() + .from(chatPublications) + .where( + and( + eq(chatPublications.companyId, record.endpoint.companyId), + eq(chatPublications.endpointId, record.endpoint.id), + eq(chatPublications.conversationId, conversation.id), + eq(chatPublications.issueId, conversation.issueId), + eq(chatPublications.id, loaded.publicationId), + eq(chatPublications.state, "published"), + event.provider !== "discord" && event.event.relatedMessage + ? eq( + chatPublications.providerMessageId, + event.event.relatedMessage.id, + ) + : undefined, + ), + ) + .then((rows) => rows[0] ?? null); + const providerMessageId = + (event.provider === "discord" + ? undefined + : event.event.relatedMessage?.id) ?? + originalPublication?.providerMessageId ?? + null; + if (!originalPublication || !providerMessageId) { + return deny( + "chat_modal_publication_not_current", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const currentMessageBinding = await db + .select({ + link: chatMessageLinks, + publication: chatPublications, + }) + .from(chatMessageLinks) + .innerJoin( + chatPublications, + and( + eq(chatPublications.companyId, chatMessageLinks.companyId), + eq(chatPublications.endpointId, chatMessageLinks.endpointId), + eq(chatPublications.conversationId, chatMessageLinks.conversationId), + eq(chatPublications.id, chatMessageLinks.publicationId), + ), + ) + .where( + and( + eq(chatMessageLinks.companyId, record.endpoint.companyId), + eq(chatMessageLinks.endpointId, record.endpoint.id), + eq(chatMessageLinks.conversationId, conversation.id), + eq(chatMessageLinks.providerMessageId, providerMessageId), + eq(chatMessageLinks.direction, "outbound"), + eq(chatPublications.issueId, conversation.issueId), + eq(chatPublications.state, "published"), + eq(chatPublications.providerMessageId, providerMessageId), + ), + ) + .then((rows) => (rows.length === 1 ? rows[0]! : null)); + const linkedPayload = currentMessageBinding?.publication.payload as + | SafeChatPublicationPayload + | undefined; + const linkStillAuthoritative = + currentMessageBinding?.link.publicationId === originalPublication.id || + (currentMessageBinding?.publication.idempotencyKey === + `interaction-resolution:${loaded.interactionId}:${record.endpoint.id}` && + linkedPayload?.interactionId === loaded.interactionId); + if (!currentMessageBinding || !linkStillAuthoritative) { + return deny( + "chat_modal_message_binding_not_current", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const issued = { + publication: originalPublication, + link: currentMessageBinding.link, + }; + const payload = issued.publication.payload as SafeChatPublicationPayload; + if (payload.interactionId !== loaded.interactionId) { + return deny( + "chat_modal_interaction_binding_not_current", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const interaction = ( + await issueThreadInteractionService(db).listForIssue(conversation.issueId) + ).find((candidate) => candidate.id === loaded.interactionId); + const discordReplayAnswersMatch = ( + resolved: typeof interaction, + ): boolean => { + if (event.provider !== "discord") return true; + if ( + resolved?.kind !== "ask_user_questions" || + resolved.status !== "answered" + ) + return false; + // Parse only for comparison after current token/source/actor checks. + // This local pending projection cannot reopen or resolve an interaction. + const submitted = validateChatQuestionFormSubmission({ + callbackId: event.event.callbackId, + privateMetadata: event.event.privateMetadata, + interaction: { ...resolved, status: "pending" }, + payload: loaded.payload, + values: event.event.values, + }); + return ( + submitted.ok && + JSON.stringify(submitted.answers) === + JSON.stringify(resolved.result?.answers) + ); + }; + const isExactProcessedReplay = ( + current: Awaited>, + ) => + interaction?.companyId === record.endpoint.companyId && + interaction.issueId === conversation.issueId && + interaction.kind === "ask_user_questions" && + interaction.status === "answered" && + interaction.resolvedByUserId === principal.userId && + current?.status === "processed" && + current.actionRowId === loaded.actionRowId && + current.conversationId === conversation.id && + current.publicationId === originalPublication.id && + current.interactionId === loaded.interactionId && + current.principalId === principal.principal.id && + current.result?.code === "question_form_answered" && + current.result?.interactionId === loaded.interactionId && + discordReplayAnswersMatch(interaction); + const acceptedResponse = async (): Promise => { + if (event.provider === "discord") { + return acceptDiscordQuestionForm({ record, event, principal: { ...principal, userId: principal.userId! }, loaded }); + } + return { action: "clear" }; + }; + if (loaded.status === "processed") { + if (!isExactProcessedReplay(loaded)) { + return deny( + "chat_modal_replay_not_authorized", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + if (event.event.relatedThread) { + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.relatedThread.id, + event.event.raw, + ); + } + return acceptedResponse(); + } + if ( + !interaction || + interaction.companyId !== record.endpoint.companyId || + interaction.issueId !== conversation.issueId || + interaction.kind !== "ask_user_questions" || + interaction.status !== "pending" + ) { + // Another callback can commit the same token between the initial token + // read and this interaction read. Only its exact processed receipt may + // acknowledge the stale snapshot; an answer by another token or actor + // does not authorize this form. Recheck current policy under its locks. + let processedReplay = false; + if ( + interaction?.companyId === record.endpoint.companyId && + interaction.issueId === conversation.issueId && + interaction.kind === "ask_user_questions" && + interaction.status === "answered" + ) { + try { + processedReplay = await db.transaction(async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + return isExactProcessedReplay( + await loadChatQuestionFormSubmissionToken(tx, { + callbackId: event.event.callbackId, + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + includeProcessed: true, + }), + ); + }); + } catch (error) { + if (!isExternalActionAuthorizationChange(error)) throw error; + } + } + if (processedReplay) { + if (event.event.relatedThread) { + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.relatedThread.id, + event.event.raw, + ); + } + return acceptedResponse(); + } + return deny( + "chat_modal_interaction_not_pending", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const validation = validateChatQuestionFormSubmission({ + callbackId: event.event.callbackId, + privateMetadata: event.event.privateMetadata, + interaction, + payload: loaded.payload, + values: event.event.values, + }); + if (!validation.ok) { + if (validation.code === "invalid_form") { + if (event.provider === "discord") { + return handleDiscordFormCorrection({ record, event, runtimeContext, conversation, principal: { ...principal, userId: principal.userId }, loaded, originalPublication, providerMessageId, interaction, payload, validation, deny }); + } + return chatQuestionFormValidationResponse({ + provider: event.provider, + callbackId: event.event.callbackId, + privateMetadata: event.event.privateMetadata, + interaction, + payload: loaded.payload, + values: event.event.values, + }); + } + return deny( + `chat_modal_${validation.code}`, + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + const issue = await db + .select({ + id: issues.id, + companyId: issues.companyId, + status: issues.status, + }) + .from(issues) + .where( + and( + eq(issues.companyId, record.endpoint.companyId), + eq(issues.id, conversation.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!issue) { + return deny( + "chat_modal_issue_not_current", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + await options.questionResolutionPersistBarrier?.(); + let answered: Awaited< + ReturnType< + ReturnType["answerQuestions"] + > + >; + try { + answered = await issueThreadInteractionService(db).answerQuestions( + issue, + interaction.id, + { answers: validation.answers }, + { userId: principal.userId }, + { + beforeResolveInTransaction: async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: event.endpointId, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + }, + afterResolveInTransaction: async (tx, resolved) => { + const claimed = await claimChatQuestionFormSubmission(tx, { + actionRowId: loaded.actionRowId, + principalId: principal.principal.id, + }); + if (!claimed) throw denialError(); + const completed = await completeChatQuestionFormSubmission(tx, { + actionRowId: loaded.actionRowId, + interactionId: resolved.id, + }); + if (!completed) throw denialError(); + await tx + .update(chatActions) + .set({ + status: "expired", + result: { code: "question_form_submitted" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.companyId, issue.companyId), + eq(chatActions.endpointId, record.endpoint.id), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.kind, "question_form_open"), + eq(chatActions.status, "issued"), + eq( + sql`${chatActions.payload}->>'interactionId'`, + resolved.id, + ), + ), + ); + await logActivity(tx as unknown as Db, { + companyId: issue.companyId, + actorType: "user", + actorId: principal.userId!, + action: "issue.thread_interaction_answered", + entityType: "issue", + entityId: issue.id, + details: { + source: "external_chat_modal", + endpointId: record.endpoint.id, + provider: record.endpoint.provider, + conversationId: conversation.id, + publicationId: issued.publication.id, + interactionId: resolved.id, + interactionKind: resolved.kind, + interactionStatus: resolved.status, + resolutionActorKind: "user", + answeredQuestionCount: validation.answers.length, + }, + }); + }, + }, + ); + } catch (error) { + if (isExternalActionAuthorizationChange(error)) { + await db.transaction(async (tx) => { + await tx + .update(chatActions) + .set({ + status: "expired", + result: { code: "chat_action_authorization_changed" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, loaded.actionRowId), + eq(chatActions.status, "issued"), + ), + ); + await recordModalSubmitDenial( + record.endpoint, + event, + { + code: "chat_action_authorization_changed", + conversationId: conversation.id, + principalId: principal.principal.id, + }, + tx, + ); + }); + return event.provider === "discord" + ? discordQuestionFormDenialResponse() + : chatQuestionFormDenialResponse(loaded.payload); + } + const settledRace = await db.transaction(async (tx) => { + const currentInteraction = ( + await issueThreadInteractionService(tx as unknown as Db).listForIssue( + issue.id, + ) + ).find((candidate) => candidate.id === interaction.id); + if (!currentInteraction || currentInteraction.status === "pending") { + return false; + } + await enqueueTerminalIssueInteractionChatPublications( + tx, + currentInteraction, + ); + return true; + }); + if (!settledRace) { + const status = + error && typeof error === "object" + ? (error as { status?: unknown }).status + : null; + if (status === 403) { + return deny( + "chat_modal_resolution_not_authorized", + { + conversationId: conversation.id, + principalId: principal.principal.id, + }, + loaded.payload, + ); + } + throw error; + } + scheduleMessageProcessing(async () => { + await processPendingPublications(); + }); + if (event.provider === "discord") { + const current = await loadChatQuestionFormSubmissionToken(db, { + callbackId: event.event.callbackId, + companyId: record.endpoint.companyId, + endpointId: record.endpoint.id, + includeProcessed: true, + }); + const resolved = ( + await issueThreadInteractionService(db).listForIssue( + conversation.issueId, + ) + ).find((candidate) => candidate.id === interaction.id); + if ( + current?.status !== "processed" || + current.actionRowId !== loaded.actionRowId || + current.principalId !== principal.principal.id || + current.result?.code !== "question_form_answered" || + current.result.interactionId !== interaction.id || + resolved?.status !== "answered" || + resolved.resolvedByUserId !== principal.userId || + !discordReplayAnswersMatch(resolved) + ) + return deny("chat_modal_resolution_committed_elsewhere"); + } + return acceptedResponse(); + } + if (event.event.relatedThread) { + await recordCurrentMicrosoftTeamsRoute( + record.endpoint, + runtimeContext, + event.event.relatedThread.id, + event.event.raw, + ); + } + scheduleMessageProcessing(async () => { + await questionResponses.deliver(answered.id); + }); + return acceptedResponse(); + } + return { handleAction, handleModalSubmit, reconcileTerminalConfirmationActions }; +} diff --git a/server/src/services/chat-channels/interactions/continuations.ts b/server/src/services/chat-channels/interactions/continuations.ts new file mode 100644 index 00000000000..8b6697e91c3 --- /dev/null +++ b/server/src/services/chat-channels/interactions/continuations.ts @@ -0,0 +1,432 @@ +import { + and, + asc, + desc, + eq, + isNotNull, + isNull, + lte, + notInArray, + or, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + agentWakeupRequests, + chatActions, + chatEndpoints, + chatMessageLinks, + issues, +} from "@paperclipai/db"; + +import { isUniqueViolation } from "../../../db-errors.js"; + +import { logger } from "../../../middleware/logger.js"; + +import { + type IssueAssignmentWakeupDeps, +} from "../../issue-assignment-wakeup.js"; + +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; + +type InteractionContinuationsOptions = { + heartbeat: IssueAssignmentWakeupDeps; + redactError: (error: unknown) => string; + CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS: number; +}; + +export function createChatInteractionContinuations(db: Db, options: InteractionContinuationsOptions) { + const { redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS } = options; + + + async function processPendingInteractionWakeups( + limit = 100, + ): Promise { + const staleBefore = new Date(Date.now() - 30_000); + const retryBefore = new Date( + Date.now() - CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, + ); + const candidates = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.kind, "interaction_wakeup"), + or( + and( + eq(chatActions.status, "issued"), + or( + isNull(chatActions.result), + lte(chatActions.updatedAt, retryBefore), + ), + ), + and( + eq(chatActions.status, "processing"), + lte(chatActions.updatedAt, staleBefore), + ), + ), + ), + ) + .orderBy(asc(chatActions.updatedAt), asc(chatActions.id)) + .limit(limit); + let processed = 0; + for (const candidate of candidates) { + const [claimed] = await db + .update(chatActions) + .set({ status: "processing", updatedAt: new Date() }) + .where( + and( + eq(chatActions.id, candidate.id), + or( + and( + eq(chatActions.status, "issued"), + or( + isNull(chatActions.result), + lte(chatActions.updatedAt, retryBefore), + ), + ), + and( + eq(chatActions.status, "processing"), + lte(chatActions.updatedAt, staleBefore), + ), + ), + ), + ) + .returning(); + if (!claimed) continue; + const payload = claimed.payload; + const requestedByActorType = + payload.requestedByActorType === "user" || + payload.requestedByActorType === "agent" || + payload.requestedByActorType === "system" + ? payload.requestedByActorType + : typeof payload.requestedByUserId === "string" + ? "user" + : null; + const requestedByActorId = + typeof payload.requestedByActorId === "string" + ? payload.requestedByActorId + : typeof payload.requestedByUserId === "string" + ? payload.requestedByUserId + : null; + if ( + payload.version !== 1 || + typeof payload.interactionId !== "string" || + typeof payload.interactionKind !== "string" || + !["accepted", "rejected", "answered", "cancelled", "failed"].includes( + String(payload.interactionStatus), + ) || + typeof payload.issueId !== "string" || + typeof payload.agentId !== "string" || + !requestedByActorType || + !requestedByActorId + ) { + await db + .update(chatActions) + .set({ + status: "failed", + result: { code: "interaction_wakeup_payload_invalid" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, claimed.id), + eq(chatActions.status, "processing"), + ), + ); + continue; + } + const currentIssue = await db + .select({ + assigneeAgentId: issues.assigneeAgentId, + status: issues.status, + }) + .from(issues) + .where( + and( + eq(issues.companyId, claimed.companyId), + eq(issues.id, payload.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + const currentInteraction = currentIssue + ? ( + await issueThreadInteractionService(db).listForIssue( + payload.issueId, + ) + ).find((interaction) => interaction.id === payload.interactionId) + : null; + if ( + !currentIssue || + !currentInteraction || + currentInteraction.companyId !== claimed.companyId || + currentInteraction.kind !== payload.interactionKind || + currentInteraction.status !== payload.interactionStatus || + currentIssue.assigneeAgentId !== payload.agentId || + currentIssue.status === "done" || + currentIssue.status === "cancelled" + ) { + await db + .update(chatActions) + .set({ + status: "processed", + result: { code: "interaction_wakeup_no_longer_applicable" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, claimed.id), + eq(chatActions.status, "processing"), + ), + ); + processed += 1; + continue; + } + const idempotencyKey = `interaction:${payload.interactionId}:${payload.interactionStatus}`; + try { + const sourceCommentId = currentInteraction.sourceCommentId ?? null; + const sourceRunId = currentInteraction.sourceRunId ?? null; + // A resolved interaction is still part of the originating external + // turn. Preserve one verified inbound comment edge on the continuation + // run so queued/working/failure milestones return to this exact + // endpoint and conversation instead of disappearing as an internal + // automation run. + const chatOrigin = claimed.conversationId + ? await db + .select({ + commentId: chatMessageLinks.commentId, + provider: chatEndpoints.provider, + }) + .from(chatEndpoints) + .innerJoin( + chatMessageLinks, + and( + eq(chatMessageLinks.companyId, chatEndpoints.companyId), + eq(chatMessageLinks.endpointId, chatEndpoints.id), + ), + ) + .where( + and( + eq(chatEndpoints.companyId, claimed.companyId), + eq(chatEndpoints.id, claimed.endpointId), + eq(chatMessageLinks.conversationId, claimed.conversationId), + eq(chatMessageLinks.direction, "inbound"), + isNotNull(chatMessageLinks.commentId), + ), + ) + .orderBy( + desc(chatMessageLinks.createdAt), + desc(chatMessageLinks.id), + ) + .limit(1) + .then((rows) => rows[0] ?? null) + : null; + const wakeCommentId = sourceCommentId ?? chatOrigin?.commentId ?? null; + const existing = await db + .select({ + agentId: agentWakeupRequests.agentId, + id: agentWakeupRequests.id, + }) + .from(agentWakeupRequests) + .where( + and( + eq(agentWakeupRequests.companyId, claimed.companyId), + eq(agentWakeupRequests.idempotencyKey, idempotencyKey), + notInArray(agentWakeupRequests.status, [ + "skipped", + "failed", + "cancelled", + ]), + ), + ) + .limit(1) + .then((rows) => rows[0] ?? null); + const queuedRun = !existing + ? await options.heartbeat.wakeup(payload.agentId, { + source: "automation", + triggerDetail: "system", + reason: "issue_commented", + payload: { + issueId: payload.issueId, + interactionId: payload.interactionId, + interactionKind: payload.interactionKind, + interactionStatus: payload.interactionStatus, + sourceCommentId, + sourceRunId, + ...(wakeCommentId + ? { + wakeCommentId, + wakeCommentIds: [wakeCommentId], + } + : {}), + ...(payload.planReviewInteraction && + typeof payload.planReviewInteraction === "object" + ? { planReviewInteraction: payload.planReviewInteraction } + : {}), + mutation: "interaction", + }, + idempotencyKey, + allowRunCoalescing: false, + requestedByActorType, + requestedByActorId, + contextSnapshot: { + issueId: payload.issueId, + taskId: payload.issueId, + interactionId: payload.interactionId, + interactionKind: payload.interactionKind, + interactionStatus: payload.interactionStatus, + sourceCommentId, + sourceRunId, + ...(payload.planReviewInteraction && + typeof payload.planReviewInteraction === "object" + ? { planReviewInteraction: payload.planReviewInteraction } + : {}), + wakeReason: "issue_commented", + source: chatOrigin + ? `chat:${chatOrigin.provider}` + : "external_chat.interaction.resolve", + ...(wakeCommentId + ? { + wakeCommentId, + wakeCommentIds: [wakeCommentId], + } + : {}), + ...(payload.forceFreshSession === true + ? { forceFreshSession: true } + : {}), + ...(typeof payload.workspaceRefreshReason === "string" + ? { + workspaceRefreshReason: payload.workspaceRefreshReason, + } + : {}), + }, + }) + : null; + const durable = + existing ?? + (await db + .select({ + agentId: agentWakeupRequests.agentId, + id: agentWakeupRequests.id, + }) + .from(agentWakeupRequests) + .where( + and( + eq(agentWakeupRequests.companyId, claimed.companyId), + eq(agentWakeupRequests.idempotencyKey, idempotencyKey), + notInArray(agentWakeupRequests.status, [ + "skipped", + "failed", + "cancelled", + ]), + ), + ) + .limit(1) + .then((rows) => rows[0] ?? null)); + if (!queuedRun && !durable) { + await db + .update(chatActions) + .set({ + status: "issued", + result: { code: "interaction_wakeup_deferred" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, claimed.id), + eq(chatActions.status, "processing"), + ), + ); + continue; + } + await db + .update(chatActions) + .set({ + status: "processed", + result: { + code: durable + ? durable.agentId === payload.agentId + ? "interaction_wakeup_already_durable" + : "interaction_wakeup_coalesced_after_reassignment" + : "interaction_wakeup_queued", + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, claimed.id), + eq(chatActions.status, "processing"), + ), + ); + processed += 1; + } catch (error) { + const durable = isUniqueViolation(error) + ? await db + .select({ + agentId: agentWakeupRequests.agentId, + id: agentWakeupRequests.id, + }) + .from(agentWakeupRequests) + .where( + and( + eq(agentWakeupRequests.companyId, claimed.companyId), + eq(agentWakeupRequests.idempotencyKey, idempotencyKey), + notInArray(agentWakeupRequests.status, [ + "skipped", + "failed", + "cancelled", + ]), + ), + ) + .limit(1) + .then((rows) => rows[0] ?? null) + : null; + const previousResult = + claimed.result && typeof claimed.result === "object" + ? (claimed.result as Record) + : {}; + const previousAttempts = + typeof previousResult.attemptCount === "number" && + Number.isInteger(previousResult.attemptCount) && + previousResult.attemptCount >= 0 + ? previousResult.attemptCount + : 0; + const attemptCount = previousAttempts + 1; + await db + .update(chatActions) + .set({ + status: durable ? "processed" : "issued", + result: { + code: durable + ? durable.agentId === payload.agentId + ? "interaction_wakeup_already_durable" + : "interaction_wakeup_coalesced_after_reassignment" + : "interaction_wakeup_failed", + ...(!durable ? { attemptCount } : {}), + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, claimed.id), + eq(chatActions.status, "processing"), + ), + ); + if (!durable) { + logger.warn( + { + endpointId: claimed.endpointId, + interactionId: payload.interactionId, + attemptCount, + error: redactError(error), + }, + "failed to queue external interaction continuation wake", + ); + } + } + } + return processed; + } + return { processPendingInteractionWakeups }; +} diff --git a/server/src/services/chat-channels/microsoft-teams/destination-admission.ts b/server/src/services/chat-channels/microsoft-teams/destination-admission.ts new file mode 100644 index 00000000000..917aec4b9dc --- /dev/null +++ b/server/src/services/chat-channels/microsoft-teams/destination-admission.ts @@ -0,0 +1,68 @@ +import type { Message, Thread } from "chat"; +import { and, eq, sql } from "drizzle-orm"; +import { chatEndpointResources } from "@paperclipai/db"; +import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js"; +import { nonDirectDestinationAllowed, teamsThreadRootMessageId } from "../inbound/surfaces.js"; + +type DestinationAdmissionInput = { + endpoint: EndpointRow; + currentEndpoint: Pick; + thread: Thread; + message: Message; + addressed: boolean; +}; +type DestinationAdmissionOptions = { + ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise; +}; +export function createTeamsDestinationAdmission(options: DestinationAdmissionOptions) { + const { ensureResource } = options; + async function admitTeamsDestination(input: DestinationAdmissionInput, tx: DbTransaction) { + const { endpoint, currentEndpoint, thread, message, addressed } = input; + let destinationAccepting = true; + let provisionalTeamsSetupReply = false; + + let resource = await ensureResource(endpoint, thread, false, tx); + const enabledChannelCount = await tx + .select({ count: sql`count(*)::int` }) + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.enabled, true), + eq(chatEndpointResources.availability, "available"), + eq(chatEndpointResources.type, "channel"), + ), + ) + .then((rows) => rows[0]?.count ?? 0); + const canActivateFirstSetupChannel = + currentEndpoint.status === "verifying" && + resource.type === "channel" && + resource.availability === "available" && + enabledChannelCount === 0 && + addressed; + if (canActivateFirstSetupChannel && !resource.enabled) { + resource = await tx + .update(chatEndpointResources) + .set({ enabled: true, updatedAt: new Date() }) + .where(eq(chatEndpointResources.id, resource.id)) + .returning() + .then((rows) => rows[0] ?? resource); + } + const teamsRootMessageId = teamsThreadRootMessageId(thread.id); + provisionalTeamsSetupReply = + currentEndpoint.status === "verifying" && + resource.type === "channel" && + resource.availability === "available" && + enabledChannelCount === 0 && + !addressed && + Boolean(teamsRootMessageId) && + teamsRootMessageId !== message.id; + destinationAccepting = + nonDirectDestinationAllowed( + { ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats }, + resource, + ) || provisionalTeamsSetupReply; + return { destinationAccepting, provisionalTeamsSetupReply }; + } + return { admitTeamsDestination }; +} diff --git a/server/src/services/chat-channels/microsoft-teams/routes.ts b/server/src/services/chat-channels/microsoft-teams/routes.ts new file mode 100644 index 00000000000..541a40a75b6 --- /dev/null +++ b/server/src/services/chat-channels/microsoft-teams/routes.ts @@ -0,0 +1,55 @@ +import type { EndpointRow, DbTransaction, LifecycleRuntimeFence, InboundRuntimeContext } from "../runtime-types.js"; + +import { type ChatRuntime } from "../../chat-runtime.js"; + +import type { Db } from "@paperclipai/db"; + +import { + type ChatSdkRuntime, +} from "../../chat-sdk-runtime.js"; + +type TeamsRoutesOptions = { + runtime: Pick; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; +}; + + + +function microsoftTeamsServiceUrl(raw: unknown): string | null { + if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null; + const value = (raw as { serviceUrl?: unknown }).serviceUrl; + return typeof value === "string" && value ? value : null; +} +export function createTeamsRoutes(db: Db, options: TeamsRoutesOptions) { + const { runtime, runtimeCallbackEndpoint } = options; + + + async function recordCurrentMicrosoftTeamsRoute( + endpoint: EndpointRow, + context: InboundRuntimeContext | undefined, + threadId: string, + raw: unknown, + recordAcceptedActivity = true, + ): Promise { + if (endpoint.provider !== "microsoft-teams" || !context?.endpointRuntime) { + return; + } + const serviceUrl = microsoftTeamsServiceUrl(raw); + if (!serviceUrl) return; + await db.transaction(async (tx) => { + const current = await runtimeCallbackEndpoint(tx, endpoint.id, context, [ + "verifying", + "active", + ]); + if (!current || runtime.get(endpoint.id) !== context.endpointRuntime) { + return; + } + await context.endpointRuntime.recordMicrosoftTeamsRoute( + threadId, + serviceUrl, + recordAcceptedActivity ? raw : undefined, + ); + }); + } + return { recordCurrentMicrosoftTeamsRoute }; +} diff --git a/server/src/services/chat-channels/provider-catalog.ts b/server/src/services/chat-channels/provider-catalog.ts new file mode 100644 index 00000000000..5beb7879400 --- /dev/null +++ b/server/src/services/chat-channels/provider-catalog.ts @@ -0,0 +1,108 @@ +import type { + ChatAdapterCapabilities, + ChatProvider, +} from "@paperclipai/shared"; + +const PROVIDER_LABELS: Record = { + speko: "Speko", + "imessage-photon": "iMessage Photon", + agentmail: "AgentMail", + slack: "Slack", + github: "GitHub", + discord: "Discord", + "microsoft-teams": "Microsoft Teams", + telegram: "Telegram", +}; + +const CAPABILITIES: Record = { + speko: { threads: false, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: false, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false }, + "imessage-photon": { threads: false, directMessages: true, nativeStreaming: false, messageEdits: true, messageDeletes: false, reactions: false, files: true, cards: false, actions: true, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false }, + agentmail: { threads: true, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: true, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: true }, + slack: { + threads: true, + directMessages: true, + nativeStreaming: true, + messageEdits: true, + messageDeletes: true, + reactions: true, + files: true, + cards: true, + actions: true, + modals: true, + slashCommands: true, + ephemeralMessages: true, + proactiveDirectMessages: false, + }, + github: { + threads: true, + directMessages: false, + nativeStreaming: false, + messageEdits: true, + messageDeletes: true, + reactions: true, + files: false, + cards: true, + actions: false, + modals: false, + slashCommands: false, + ephemeralMessages: false, + proactiveDirectMessages: false, + }, + discord: { + threads: true, + directMessages: true, + nativeStreaming: false, + messageEdits: true, + messageDeletes: true, + reactions: true, + files: true, + cards: true, + actions: true, + modals: true, + // Automatic registration upgrades these only after a durable provider + // receipt. A draft or an uncertain/conflicting registration stays false. + slashCommands: false, + ephemeralMessages: false, + proactiveDirectMessages: false, + }, + "microsoft-teams": { + threads: true, + directMessages: true, + // Production webhook processing is deferred into Paperclip's durable + // ingress queue. The Teams adapter's request-scoped DM streamer is gone by + // the time agent output is published, so advertise the durable behavior we + // can actually provide. editMessage still lets one run coalesce its + // queued, working, and final states in place. + nativeStreaming: false, + // Paperclip supplements the pinned adapter's missing inbound Bot Framework + // messageUpdate/messageDelete dispatch after provider authentication. + messageEdits: true, + messageDeletes: true, + reactions: true, + files: true, + cards: true, + actions: true, + modals: true, + slashCommands: false, + ephemeralMessages: true, + proactiveDirectMessages: false, + }, + telegram: { + threads: true, + directMessages: true, + nativeStreaming: true, + messageEdits: true, + // Telegram's Bot API does not emit an update when a user deletes a + // message, so this capability cannot be offered truthfully. + messageDeletes: false, + reactions: true, + files: true, + cards: true, + actions: true, + modals: false, + slashCommands: false, + ephemeralMessages: false, + proactiveDirectMessages: false, + }, +}; +export { PROVIDER_LABELS, CAPABILITIES }; diff --git a/server/src/services/chat-channels/publications/authorization.ts b/server/src/services/chat-channels/publications/authorization.ts new file mode 100644 index 00000000000..9b3a82d58b5 --- /dev/null +++ b/server/src/services/chat-channels/publications/authorization.ts @@ -0,0 +1,344 @@ +import { createGitHubPublicationPolicy } from "../github/publication-policy.js"; + +import type { EndpointRow, DbTransaction, DbOrTransaction, LifecycleRuntimeFence } from "../runtime-types.js"; +import { createChatFailedRunRetry } from "../failed-run-retry.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import { authorizeSlackBoardPublication } from "../../slack-board-authority.js"; + +import { slackPublicationAllowed } from "../../connectors/slack-access.js"; + +import { + and, + eq, + gt, + inArray, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatConversations, + chatEndpointResources, + chatPublications, + heartbeatRuns, + issueComments, +} from "@paperclipai/db"; + +import { + authorizeNativeChatReviewPresentation, + NativeChatReviewPresentationContentionError, +} from "../../native-runtime/native-chat-review-presentation.js"; +import { isExternalChatWaitAuthorizationContention } from "../../native-runtime/chat-attachment-reuse.js"; + +import { isExplicitOperatorPublication } from "./policy.js"; +import { attachmentFailureNoticeFence } from "./policy.js"; +import type { createChatPublicationTaskControl } from "./task-control.js"; +type ConversationRow = typeof chatConversations.$inferSelect; +type PublicationAuthorizationOptions = { + authorizeInboundWakePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise; + authorizeRetainedChatSourcePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise; + fetchImpl: typeof globalThis.fetch; + githubCurrentOmissionPreparedText: ReturnType["githubCurrentOmissionPreparedText"]; + hasCommittedTaskControlCompletion: ReturnType["hasCommittedTaskControlCompletion"]; + lockCurrentPrincipalAuthorization: (tx: DbOrTransaction, endpoint: EndpointRow, principalId: string) => Promise<{ allowed: boolean; userId?: string | null }>; + nonDirectDestinationAllowed: (endpoint: EndpointRow, resource: typeof chatEndpointResources.$inferSelect | null | undefined) => boolean; + resolveCredentials: (endpoint: EndpointRow, tx?: DbOrTransaction) => Promise>; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; + FailedChatRetryPublicationReadError: ReturnType["FailedChatRetryPublicationReadError"]; +}; +export function createChatPublicationAuthorization(db: Db, options: PublicationAuthorizationOptions) { + const { authorizeInboundWakePublication, authorizeRetainedChatSourcePublication, fetchImpl, githubCurrentOmissionPreparedText, hasCommittedTaskControlCompletion, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed, resolveCredentials, runtimeCallbackEndpoint, FailedChatRetryPublicationReadError } = options; + + + async function claimPublicationTransportAuthorization(input: { + credentialLease: CredentialMutationLeaseGuard; + endpoint: EndpointRow; + publication: typeof chatPublications.$inferSelect; + runtimeContext: LifecycleRuntimeFence; + }): Promise<{ + authorizationActionId: string | null; + conversation: ConversationRow; + endpoint: EndpointRow; + preparedText: string | null; + } | null> { + return db.transaction(async (tx) => { + await input.credentialLease.assertOwned(tx); + const currentPublication = await tx + .select({ id: chatPublications.id }) + .from(chatPublications) + .where( + and( + eq(chatPublications.id, input.publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, input.publication.attempts + 1), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null); + if (!currentPublication) return null; + try { + if ( + !(await authorizeRetainedChatSourcePublication(tx, input.publication)) + ) + return null; + } catch (error) { + if (error instanceof NativeChatReviewPresentationContentionError) + throw error; + throw new FailedChatRetryPublicationReadError(); + } + const attachmentFailureFence = attachmentFailureNoticeFence( + input.publication.idempotencyKey, + ); + if ( + attachmentFailureFence && + (attachmentFailureFence.generation !== + input.runtimeContext.generation || + attachmentFailureFence.credentialFingerprint !== + input.runtimeContext.credentialFingerprint) + ) { + return null; + } + const endpoint = await runtimeCallbackEndpoint( + tx, + input.publication.endpointId, + input.runtimeContext, + ["verifying", "active"], + ); + const conversation = endpoint + ? await tx + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, endpoint.companyId), + eq(chatConversations.endpointId, endpoint.id), + eq(chatConversations.id, input.publication.conversationId), + inArray(chatConversations.state, [ + "active", + "waiting", + "completed", + ]), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const resource = + endpoint && conversation?.resourceId && !conversation.isDirectMessage + ? await tx + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.companyId, endpoint.companyId), + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.id, conversation.resourceId), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const attachmentFailureNoticeSuperseded = + attachmentFailureFence && conversation + ? await tx + .select({ id: chatConversations.id }) + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, conversation.companyId), + eq(chatConversations.endpointId, conversation.endpointId), + eq( + chatConversations.externalThreadId, + conversation.externalThreadId, + ), + gt( + chatConversations.sessionGeneration, + conversation.sessionGeneration, + ), + ), + ) + .limit(1) + .then((rows) => rows.length > 0) + : false; + const isTaskControl = + input.publication.idempotencyKey.startsWith("control:"); + const providerVisibleCompletion = + conversation?.state === "completed" && + !isExplicitOperatorPublication(input.publication) && + (await hasCommittedTaskControlCompletion(conversation.id, tx)); + const authorizationAction = + endpoint && isTaskControl + ? await tx + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, endpoint.companyId), + eq(chatActions.endpointId, endpoint.id), + eq(chatActions.conversationId, conversation?.id ?? ""), + eq(chatActions.kind, "task_control_authorization"), + eq( + chatActions.providerActionId, + `task-control-authorization:${input.publication.id}`, + ), + eq(chatActions.status, "issued"), + ), + ) + .for("update") + .then((rows) => rows[0] ?? null) + : null; + const destinationAllowed = + endpoint !== null && + conversation !== null && + (endpoint.provider !== "slack" || await slackPublicationAllowed(tx, endpoint.companyId, endpoint.id, input.publication.issueId, conversation.externalConversationId, null)) && + (conversation.isDirectMessage + ? endpoint.allowDirectMessages + : nonDirectDestinationAllowed(endpoint, resource)); + if ( + !endpoint || + !conversation || + (attachmentFailureFence && + ((conversation.state !== "active" && + conversation.state !== "waiting") || + attachmentFailureNoticeSuperseded)) || + providerVisibleCompletion || + (isTaskControl && !authorizationAction?.principalId) || + !destinationAllowed + ) { + return null; + } + let authorizationActionId: string | null = null; + if (endpoint.provider === "slack") { + const credentials = await resolveCredentials(endpoint, tx); + if (!await authorizeSlackBoardPublication(tx, endpoint, conversation, input.publication, credentials.botToken ?? "", fetchImpl)) return null; + } + if ( + input.publication.idempotencyKey.startsWith("wake:") && + !(await authorizeInboundWakePublication(tx, input.publication)) + ) + return null; + // A native response alongside an older completion review has its own + // narrow authorization. Recheck that exact proof, review and original + // requester at the transport claim, including file publications whose + // comment is the run's earlier internal attachment-selection comment. + if ( + input.publication.commentId && + !isExplicitOperatorPublication(input.publication) + ) { + const [origin] = await tx + .select({ + runId: heartbeatRuns.id, + resultJson: heartbeatRuns.resultJson, + }) + .from(issueComments) + .innerJoin( + heartbeatRuns, + and( + eq(heartbeatRuns.id, issueComments.createdByRunId), + eq(heartbeatRuns.companyId, issueComments.companyId), + ), + ) + .where( + and( + eq(issueComments.id, input.publication.commentId), + eq(issueComments.companyId, endpoint.companyId), + eq(issueComments.issueId, input.publication.issueId), + ), + ) + .limit(1); + if ( + origin?.resultJson?.finalizationReasonCode === + "governed_response_waiting" + ) { + try { + if ( + !(await authorizeNativeChatReviewPresentation( + tx as unknown as Db, + { + companyId: endpoint.companyId, + issueId: input.publication.issueId, + runId: origin.runId, + resultJson: origin.resultJson, + destination: { + endpointId: endpoint.id, + conversationId: conversation.id, + }, + }, + "nonblocking", + )) + ) + return null; + } catch (error) { + if (isExternalChatWaitAuthorizationContention(error)) + throw new NativeChatReviewPresentationContentionError(); + throw error; + } + } + } + const preparedText = await githubCurrentOmissionPreparedText( + tx, + input.publication, + endpoint, + ); + if (preparedText === false) return null; + if (authorizationAction?.principalId) { + const authorization = await lockCurrentPrincipalAuthorization( + tx, + endpoint, + authorizationAction.principalId, + ); + if (!authorization.allowed) return null; + const [claimedAuthorization] = await tx + .update(chatActions) + .set({ + status: "processing", + result: { + attempts: input.publication.attempts + 1, + authorizedUserId: authorization.userId, + credentialFingerprint: input.runtimeContext.credentialFingerprint, + runtimeGeneration: input.runtimeContext.generation, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, authorizationAction.id), + eq(chatActions.status, "issued"), + ), + ) + .returning({ id: chatActions.id }); + if (!claimedAuthorization) return null; + authorizationActionId = claimedAuthorization.id; + } + if (preparedText !== null) { + // Persist the exact provider text before I/O. A retry must reuse this + // body, not accumulate links or regenerate it from changed settings. + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ + payload: { ...input.publication.payload, text: preparedText }, + }) + .where(eq(chatPublications.id, input.publication.id)); + } + await input.credentialLease.assertOwned(tx); + // The publication's streaming row plus the optional authorization action + // are the durable transport claim. Their database locks are released + // before provider I/O. The renewable credential lease intentionally + // remains held so reconnect and reach mutations linearize after this + // committed authorization snapshot and transport attempt. + return { + authorizationActionId, + conversation, + endpoint, + preparedText, + }; + }); + } + return { claimPublicationTransportAuthorization }; +} diff --git a/server/src/services/chat-channels/publications/failures.ts b/server/src/services/chat-channels/publications/failures.ts new file mode 100644 index 00000000000..5c6bb302827 --- /dev/null +++ b/server/src/services/chat-channels/publications/failures.ts @@ -0,0 +1,328 @@ +import { isOutboundAttachmentHydrationError, isOutboundAttachmentValidationError } from "../outbound-attachments.js"; + +import type { EndpointRow, DbTransaction, LifecycleRuntimeFence } from "../runtime-types.js"; +import { createChatFailedRunRetry } from "../failed-run-retry.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import { + and, + eq, + inArray, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatConversations, + chatEndpointResources, + chatEndpoints, + chatPublications, + toolConnections, +} from "@paperclipai/db"; + +import { logger } from "../../../middleware/logger.js"; +import { redactSensitiveText } from "../../../redaction.js"; + +import { + NativeChatReviewPresentationContentionError, +} from "../../native-runtime/native-chat-review-presentation.js"; + +import { projectSafeChatPublication } from "../../chat-publication-projection.js"; +import { safeChatTaskUrl } from "../../chat-task-url.js"; + +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; + +import { ATTACHMENT_FAILURE_NOTICE_PREFIX, definiteAttachmentProviderRejection, attachmentFailureKind } from "./policy.js"; +type PublicationFailuresOptions = { + FailedChatRetryPublicationReadError: ReturnType["FailedChatRetryPublicationReadError"]; + MAX_ERROR_TEXT: number; + getTaskBaseUrl: () => string | null | undefined; + invalidateRuntime: (endpointId: string) => Promise; + runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise; +}; +export function createChatPublicationFailures(db: Db, options: PublicationFailuresOptions) { + const { FailedChatRetryPublicationReadError, MAX_ERROR_TEXT, getTaskBaseUrl, invalidateRuntime, runtimeCallbackEndpoint } = options; + + + async function finalizePublicationFailure( + publication: typeof chatPublications.$inferSelect, + error: unknown, + providerAccepted = false, + runtimeContext?: LifecycleRuntimeFence | null, + credentialLease?: CredentialMutationLeaseGuard, + ): Promise { + const attempts = publication.attempts + 1; + const disposition = providerAccepted + ? { + kind: "delivery_unknown" as const, + reason: + "Provider accepted the publication, but Paperclip could not confirm its durable result", + } + : error instanceof NativeChatReviewPresentationContentionError + ? { kind: "retry" as const, retryAfterMs: 250, reason: error.message } + : error instanceof FailedChatRetryPublicationReadError + ? { + kind: "retry" as const, + retryAfterMs: 1_000, + reason: error.message, + } + : isOutboundAttachmentValidationError(error) + ? { + kind: "failed" as const, + reason: error.message, + } + : isOutboundAttachmentHydrationError(error) + ? { + kind: "retry" as const, + retryAfterMs: Math.min( + 60_000, + 2 ** Math.max(0, attempts) * 1_000, + ), + reason: error.message, + } + : classifyChatPublicationError(error, attempts); + const failure = redactSensitiveText(disposition.reason).slice( + 0, + MAX_ERROR_TEXT, + ); + // Provider-directed rate limits are not delivery failures. Keep the + // durable publication retryable for the provider's full interval instead + // of exhausting the five-attempt transport budget while flood control is + // still active. + const terminalRetry = + disposition.kind === "retry" && + disposition.providerRateLimit !== true && + attempts >= 5; + const attachmentFailure = attachmentFailureKind(publication.payload); + const attachmentFailureRuntimeContext = + !providerAccepted && + disposition.kind === "failed" && + runtimeContext && + attachmentFailure && + definiteAttachmentProviderRejection(error) + ? runtimeContext + : null; + let invalidateCurrentRuntime = false; + const finalized = await db.transaction(async (tx) => { + await credentialLease?.assertOwned(tx); + const state = + disposition.kind === "retry" && !terminalRetry + ? "retry" + : disposition.kind === "delivery_unknown" + ? "delivery_unknown" + : disposition.kind === "resource_unavailable" + ? "cancelled" + : "failed"; + await notifyChatPublicationWork(tx); + const [ownedPublication] = await tx + .update(chatPublications) + .set({ + state, + nextAttemptAt: + disposition.kind === "retry" && !terminalRetry + ? new Date(Date.now() + disposition.retryAfterMs) + : null, + redactedError: failure, + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, attempts), + ), + ) + .returning({ id: chatPublications.id }); + if (!ownedPublication) return false; + + if (publication.idempotencyKey.startsWith("control:")) { + await tx + .update(chatActions) + .set({ + status: + state === "delivery_unknown" + ? "delivery_unknown" + : state === "cancelled" + ? "cancelled" + : "issued", + result: { + code: + state === "delivery_unknown" + ? "task_control_delivery_unknown" + : state === "cancelled" + ? "task_control_destination_unavailable" + : "task_control_transport_failed", + attempts, + retryable: state === "retry", + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.endpointId, publication.endpointId), + eq( + chatActions.providerActionId, + `task-control-authorization:${publication.id}`, + ), + eq(chatActions.status, "processing"), + sql`(${chatActions.result}->>'attempts')::int = ${attempts}`, + ), + ); + } + + const currentEndpoint = + runtimeContext && + (disposition.kind === "endpoint_attention" || + disposition.kind === "resource_unavailable") + ? await runtimeCallbackEndpoint( + tx, + publication.endpointId, + runtimeContext, + ["verifying", "active"], + ) + : null; + + if (disposition.kind === "endpoint_attention" && currentEndpoint) { + await tx + .update(chatEndpoints) + .set({ + status: "attention", + healthMessage: "Provider credentials or permissions need attention", + lastError: failure, + updatedAt: new Date(), + }) + .where(eq(chatEndpoints.id, currentEndpoint.id)); + await tx + .update(toolConnections) + .set({ + status: "disabled", + enabled: false, + healthStatus: "degraded", + healthMessage: "Provider credentials or permissions need attention", + lastError: failure, + healthCheckedAt: new Date(), + updatedAt: new Date(), + }) + .where(eq(toolConnections.id, currentEndpoint.connectionId)); + invalidateCurrentRuntime = true; + } else if ( + disposition.kind === "resource_unavailable" && + currentEndpoint + ) { + const binding = await tx + .select({ resourceId: chatConversations.resourceId }) + .from(chatConversations) + .where(eq(chatConversations.id, publication.conversationId)) + .then((result) => result[0] ?? null); + await tx + .update(chatConversations) + .set({ state: "unavailable", updatedAt: new Date() }) + .where( + and( + eq(chatConversations.id, publication.conversationId), + eq(chatConversations.endpointId, currentEndpoint.id), + ), + ); + if (binding?.resourceId) { + await tx + .update(chatEndpointResources) + .set({ availability: "unavailable", updatedAt: new Date() }) + .where( + and( + eq(chatEndpointResources.id, binding.resourceId), + eq(chatEndpointResources.endpointId, currentEndpoint.id), + ), + ); + } + } + + if (attachmentFailureRuntimeContext && attachmentFailure) { + const noticeEndpoint = await runtimeCallbackEndpoint( + tx, + publication.endpointId, + attachmentFailureRuntimeContext, + ["verifying", "active"], + ); + const noticeConversation = noticeEndpoint + ? await tx + .select({ + id: chatConversations.id, + state: chatConversations.state, + }) + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, publication.companyId), + eq(chatConversations.endpointId, publication.endpointId), + eq(chatConversations.id, publication.conversationId), + inArray(chatConversations.state, ["active", "waiting"]), + ), + ) + .for("no key update") + .then((rows) => rows[0] ?? null) + : null; + const providerCanSendTextNotice = + noticeEndpoint && + ["slack", "discord", "telegram"].includes(noticeEndpoint.provider) && + (attachmentFailure.provider === null || + noticeEndpoint.provider === attachmentFailure.provider); + if (providerCanSendTextNotice && noticeConversation) { + const taskUrl = safeChatTaskUrl( + getTaskBaseUrl(), + publication.issueId, + ); + const noticeText = + attachmentFailure.kind === "generated_response" + ? "Paperclip could not send the response attachment. The complete response remains on its Paperclip task for an operator to retry." + : "Paperclip could not send an attachment. The file remains on its Paperclip task for an operator to retry."; + const idempotencyKey = `${ATTACHMENT_FAILURE_NOTICE_PREFIX}${publication.id}:${attachmentFailureRuntimeContext.generation}:${attachmentFailureRuntimeContext.credentialFingerprint}`; + await notifyChatPublicationWork(tx); + await tx + .insert(chatPublications) + .values({ + companyId: publication.companyId, + endpointId: publication.endpointId, + conversationId: publication.conversationId, + issueId: publication.issueId, + commentId: null, + idempotencyKey, + payload: projectSafeChatPublication({ + classification: "external", + source: "safe_milestone", + text: taskUrl + ? `${noticeText} Open task: ${taskUrl}` + : noticeText, + }), + state: "pending", + }) + .onConflictDoNothing(); + } + } + + await credentialLease?.assertOwned(tx); + return true; + }); + if (finalized && invalidateCurrentRuntime) { + await credentialLease?.assertOwned(); + await invalidateRuntime(publication.endpointId).catch(() => undefined); + } + if (finalized) { + logger.warn( + { + endpointId: publication.endpointId, + publicationId: publication.id, + error: failure, + disposition: disposition.kind, + }, + "chat publication failed", + ); + } + return finalized; + } + return { finalizePublicationFailure }; +} diff --git a/server/src/services/chat-channels/publications/order.ts b/server/src/services/chat-channels/publications/order.ts new file mode 100644 index 00000000000..180dc43d449 --- /dev/null +++ b/server/src/services/chat-channels/publications/order.ts @@ -0,0 +1,36 @@ +import { + and, + eq, + lt, + or, + sql, +} from "drizzle-orm"; +import { type AnyPgColumn } from "drizzle-orm/pg-core"; + +type PublicationOrderTable = { + createdAt: AnyPgColumn; + id: AnyPgColumn; + payload: AnyPgColumn; +}; + +function publicationTransportOrderKey(table: PublicationOrderTable) { + return sql`coalesce(${table.payload}->'transportPart'->>'orderKey', ${table.id}::text)`; +} + +function publicationPrecedes( + earlier: PublicationOrderTable, + later: PublicationOrderTable, +) { + return or( + lt(earlier.createdAt, later.createdAt), + and( + eq(earlier.createdAt, later.createdAt), + lt( + publicationTransportOrderKey(earlier), + publicationTransportOrderKey(later), + ), + ), + ); +} + +export { publicationTransportOrderKey, publicationPrecedes }; diff --git a/server/src/services/chat-channels/publications/policy.ts b/server/src/services/chat-channels/publications/policy.ts new file mode 100644 index 00000000000..0abf10db4cd --- /dev/null +++ b/server/src/services/chat-channels/publications/policy.ts @@ -0,0 +1,132 @@ +import type { LifecycleRuntimeFence } from "../runtime-types.js"; + +import { + chatPublications, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + + function isExplicitOperatorPublication( + publication: Pick, + ): boolean { + return /^(?:explicit:|explicit-board:)/.test(publication.idempotencyKey); + } + +const ATTACHMENT_FAILURE_NOTICE_PREFIX = "attachment-failure-notice:"; + +function attachmentFailureNoticeFence( + idempotencyKey: string, +): LifecycleRuntimeFence | null { + const match = + /^attachment-failure-notice:[0-9a-f-]{36}:(\d+):([0-9a-f]{64})$/i.exec( + idempotencyKey, + ); + if (!match) return null; + const generation = Number(match[1]); + if (!Number.isSafeInteger(generation) || generation < 0) return null; + return { generation, credentialFingerprint: match[2]!.toLowerCase() }; +} + +function definiteAttachmentProviderRejection(error: unknown): boolean { + const pending: Array<{ depth: number; value: unknown }> = [ + { depth: 0, value: error }, + ]; + const seen = new Set(); + const statuses: number[] = []; + const names: string[] = []; + const codes: string[] = []; + while (pending.length > 0) { + const current = pending.shift(); + if ( + !current || + current.depth > 4 || + !current.value || + typeof current.value !== "object" || + seen.has(current.value) + ) { + continue; + } + seen.add(current.value); + const record = current.value as { + cause?: unknown; + code?: unknown; + details?: { code?: unknown; providerStatus?: unknown }; + innerHttpError?: { statusCode?: unknown }; + name?: unknown; + original?: unknown; + originalError?: unknown; + response?: { status?: unknown }; + status?: unknown; + statusCode?: unknown; + }; + for (const status of [ + record.status, + record.statusCode, + record.response?.status, + record.details?.providerStatus, + record.innerHttpError?.statusCode, + ]) { + if (typeof status === "number" && Number.isInteger(status)) { + statuses.push(status); + } + } + if (typeof record.name === "string") names.push(record.name); + for (const code of [record.code, record.details?.code]) { + if (typeof code === "string") codes.push(code); + } + for (const nested of [ + record.cause, + record.original, + record.originalError, + ]) { + pending.push({ depth: current.depth + 1, value: nested }); + } + } + if ( + names.some((name) => + ["ValidationError", "NotImplementedError"].includes(name), + ) || + codes.some((code) => + [ + "CHAT_ADAPTER_COMPATIBILITY_ERROR", + "CHAT_PROVIDER_PRETRANSPORT_REJECTED", + "NOT_IMPLEMENTED", + "VALIDATION_ERROR", + ].includes(code), + ) + ) { + return false; + } + // Slack's Web API can reject a file operation with `ok: false` in an HTTP + // 200 response. The SDK's structured platform-error code is still a + // definitive provider rejection; the main disposition classifier has + // already excluded auth, destination, and rate-limit outcomes. + if (codes.includes("slack_webapi_platform_error")) return true; + return statuses.some( + (status) => + status >= 400 && + status < 500 && + ![401, 403, 404, 410, 429].includes(status), + ); +} + +function attachmentFailureKind( + payload: SafeChatPublicationPayload, +): + | { kind: "generated_response"; provider: "discord" | "telegram" } + | { kind: "selected_file"; provider: null } + | null { + if (payload.transportPart?.mode === "discord_markdown_attachment") { + return { kind: "generated_response", provider: "discord" }; + } + if (payload.transportPart?.mode === "telegram_markdown_attachment") { + return { kind: "generated_response", provider: "telegram" }; + } + if (payload.attachmentIds?.length) { + return { kind: "selected_file", provider: null }; + } + return null; +} + +export { isExplicitOperatorPublication, ATTACHMENT_FAILURE_NOTICE_PREFIX, attachmentFailureNoticeFence, definiteAttachmentProviderRejection, attachmentFailureKind }; diff --git a/server/src/services/chat-channels/publications/preparation.ts b/server/src/services/chat-channels/publications/preparation.ts new file mode 100644 index 00000000000..54df34ef5bd --- /dev/null +++ b/server/src/services/chat-channels/publications/preparation.ts @@ -0,0 +1,150 @@ +import { createDiscordPublicationPreparation } from "../discord/publication-preparation.js"; +import { createTelegramPublicationPreparation } from "../telegram/publication-preparation.js"; +import { createChatPublicationReplacement } from "../publication-replacement.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { + eq, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatEndpoints, + chatPublications, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { + renderPublicationTransportText, + splitNativePublicationText, +} from "../../chat-publication-text-parts.js"; + +type PublicationPreparationOptions = { + runPublicationToReplace: ReturnType["runPublicationToReplace"]; + publicationTransportPreparationBarrier?: (input: { publicationId: string }) => Promise; +}; +export function createChatPublicationPreparation(db: Db, options: PublicationPreparationOptions) { + const { runPublicationToReplace } = options; + const { prepareDiscordPublication } = createDiscordPublicationPreparation(db, { runPublicationToReplace }); + const { prepareTelegramPublication } = createTelegramPublicationPreparation(db, { runPublicationToReplace }); + + + async function ensureDurablePublicationTransport( + publication: typeof chatPublications.$inferSelect, + ): Promise { + const persisted = publication.payload as SafeChatPublicationPayload; + if ( + !persisted || + typeof persisted !== "object" || + typeof persisted.text !== "string" + ) { + throw Object.assign( + new Error("Durable chat publication payload is malformed"), + { code: "VALIDATION_ERROR", name: "ValidationError" }, + ); + } + // Validate closed persisted wrappers even on retries that are already split. + renderPublicationTransportText(persisted); + await options.publicationTransportPreparationBarrier?.({ + publicationId: publication.id, + }); + const provider = await db + .select({ provider: chatEndpoints.provider }) + .from(chatEndpoints) + .where(eq(chatEndpoints.id, publication.endpointId)) + .then((rows) => rows[0]?.provider ?? null); + if ( + provider === "slack" || + provider === "github" || + provider === "microsoft-teams" + ) { + if ( + persisted.transportPart || + persisted.card || + persisted.interactionId || + persisted.attachmentIds?.length + ) + return publication; + const parts = splitNativePublicationText(provider, persisted.text); + if (parts.length === 1) return publication; + return db.transaction(async (tx) => { + const current = await tx + .select() + .from(chatPublications) + .where(eq(chatPublications.id, publication.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) return publication; + const currentPayload = current.payload as SafeChatPublicationPayload; + renderPublicationTransportText(currentPayload); + if ( + currentPayload.transportPart || + currentPayload.card || + currentPayload.interactionId || + currentPayload.attachmentIds?.length || + !["pending", "retry"].includes(current.state) + ) + return current; + const currentParts = + currentPayload.text === persisted.text + ? parts + : splitNativePublicationText(provider, currentPayload.text); + if (currentParts.length === 1) return current; + const batchId = current.id; + const updatedAt = new Date(); + const payloadFor = (index: number): SafeChatPublicationPayload => { + const part = currentParts[index]!; + const payload = { + ...currentPayload, + text: part.text, + transportPart: { + batchId, + index, + count: currentParts.length, + mode: "inline" as const, + orderKey: `${batchId}:${String(index).padStart(4, "0")}`, + ...(part.prefix ? { prefix: part.prefix } : {}), + ...(part.suffix ? { suffix: part.suffix } : {}), + }, + }; + if (index > 0) delete payload.progressState; + return payload; + }; + const rootPayload = payloadFor(0); + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: rootPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + await notifyChatPublicationWork(tx); + await tx.insert(chatPublications).values( + currentParts.slice(1).map((_part, offset) => ({ + companyId: current.companyId, + endpointId: current.endpointId, + conversationId: current.conversationId, + issueId: current.issueId, + commentId: current.commentId, + idempotencyKey: `${current.idempotencyKey}:${provider}-part:${offset + 1}`, + payload: payloadFor(offset + 1), + state: "pending" as const, + createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, + updatedAt, + })), + ); + return { ...current, payload: rootPayload, updatedAt }; + }); + } + if (provider === "discord") { + return prepareDiscordPublication(publication, persisted); + } + + if (provider !== "telegram") return publication; + return prepareTelegramPublication(publication, persisted); + } + return { ensureDurablePublicationTransport }; +} diff --git a/server/src/services/chat-channels/publications/queue.ts b/server/src/services/chat-channels/publications/queue.ts new file mode 100644 index 00000000000..7aa0537fe70 --- /dev/null +++ b/server/src/services/chat-channels/publications/queue.ts @@ -0,0 +1,389 @@ +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { unresolvedSlackPublicationCondition } from "../../connectors/slack-publication.js"; + +import { settleExternalConversation } from "../../slack-conversation-lifecycle.js"; + +import { + and, + asc, + eq, + gt, + inArray, + isNotNull, + isNull, + lte, + notExists, + notInArray, + or, + sql, +} from "drizzle-orm"; +import { alias } from "drizzle-orm/pg-core"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatEndpointLeases, + chatEndpoints, + chatPublications, + chatTeamsFileTransfers, +} from "@paperclipai/db"; + +import { logger } from "../../../middleware/logger.js"; + +import { publicationTransportOrderKey, publicationPrecedes } from "./order.js"; +import type { createChatPublicationWorker } from "./worker.js"; +type PublicationQueueOptions = { + PUBLICATION_ENDPOINT_CONCURRENCY: number; + enqueueInboundWakeupPublications: () => Promise; + processPendingInteractionWakeups: (limit?: number) => Promise; + reconcileTerminalConfirmationActions: (limit?: number) => Promise; + processSelectedPublication: ReturnType["processSelectedPublication"]; + publicationEndpointTasks: Map>; + teamsFileRetryAfter: Map; + redactError: (error: unknown) => string; + isShuttingDown: () => boolean; + wakePublications: () => void; +}; +export function createChatPublicationQueue(db: Db, options: PublicationQueueOptions) { + const { PUBLICATION_ENDPOINT_CONCURRENCY, enqueueInboundWakeupPublications, processPendingInteractionWakeups, reconcileTerminalConfirmationActions, processSelectedPublication, publicationEndpointTasks, teamsFileRetryAfter, redactError, wakePublications, isShuttingDown } = options; + + + function publicationEligibility({ now, busyEndpointIds = [], coolingTeamsPublicationIds = [], attemptedIds = [] }: { now?: Date; busyEndpointIds?: string[]; coolingTeamsPublicationIds?: string[]; attemptedIds?: string[] } = {}) { + const earlierPublication = alias(chatPublications, "earlier_chat_publications"); + return and( + sql`not (${unresolvedSlackPublicationCondition()})`, + // An explicit Board send promises both delivery and work. Keep + // its text/files pending until the durable wake has been accepted. + sql`not exists (select 1 from chat_actions a where a.company_id = ${chatPublications.companyId} and a.endpoint_id = ${chatPublications.endpointId} and a.conversation_id = ${chatPublications.conversationId} and a.kind = 'slack_board_message' and a.status = 'received' and a.payload->>'commentId' = ${chatPublications.commentId}::text)`, + or( + and( + sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`, + inArray(chatPublications.state, ["pending", "retry"]), + notExists( + db + .select({ id: chatTeamsFileTransfers.id }) + .from(chatTeamsFileTransfers) + .where( + and( + eq( + chatTeamsFileTransfers.companyId, + chatPublications.companyId, + ), + eq( + chatTeamsFileTransfers.publicationId, + chatPublications.id, + ), + ), + ), + ), + ), + // Dedicated protocol work shares the endpoint concurrency cap, + // never the ordinary publication claim or resend implementation. + sql`exists (${db + .select({ id: chatTeamsFileTransfers.id }) + .from(chatTeamsFileTransfers) + .where( + and( + eq( + chatTeamsFileTransfers.companyId, + chatPublications.companyId, + ), + eq( + chatTeamsFileTransfers.endpointId, + chatPublications.endpointId, + ), + eq( + chatTeamsFileTransfers.conversationId, + chatPublications.conversationId, + ), + eq( + chatTeamsFileTransfers.publicationId, + chatPublications.id, + ), + sql`exists (${db + .select({ id: chatEndpoints.id }) + .from(chatEndpoints) + .where( + and( + eq(chatEndpoints.id, chatPublications.endpointId), + eq( + chatEndpoints.companyId, + chatPublications.companyId, + ), + eq(chatEndpoints.provider, "microsoft-teams"), + eq(chatEndpoints.status, "active"), + ), + )})`, + or( + inArray(chatTeamsFileTransfers.phase, [ + "consent_pending", + "upload_pending", + "file_info_pending", + ]), + and( + eq(chatTeamsFileTransfers.phase, "consent_unknown"), + isNotNull(chatTeamsFileTransfers.responseActivityId), + sql`${chatTeamsFileTransfers.privateState}->'response' is not null`, + ), + ), + ), + )})`, + ), + busyEndpointIds.length > 0 + ? notInArray(chatPublications.endpointId, busyEndpointIds) + : undefined, + coolingTeamsPublicationIds.length > 0 + ? notInArray(chatPublications.id, coolingTeamsPublicationIds) + : undefined, + // Inactive endpoints must not consume the eligibility page or + // acquire an artificial retry deadline. When an operator resumes + // them, due work is immediately eligible; genuine provider retry + // deadlines below remain unchanged. + notExists( + db + .select({ id: chatEndpoints.id }) + .from(chatEndpoints) + .where( + and( + eq(chatEndpoints.id, chatPublications.endpointId), + inArray(chatEndpoints.status, ["paused", "attention"]), + ), + ), + ), + now ? or(isNull(chatPublications.nextAttemptAt), lte(chatPublications.nextAttemptAt, now)) : undefined, + attemptedIds.length > 0 + ? notInArray(chatPublications.id, attemptedIds) + : undefined, + notExists( + db + .select({ id: earlierPublication.id }) + .from(earlierPublication) + .where( + and( + eq( + earlierPublication.conversationId, + chatPublications.conversationId, + ), + publicationPrecedes(earlierPublication, chatPublications), + inArray(earlierPublication.state, [ + "pending", + "retry", + "streaming", + "delivery_unknown", + ]), + ), + ), + ), + ); + } + + async function processPendingPublications( + limit = 25, + { waitForCompletion = true, maintenance = true }: { waitForCompletion?: boolean; maintenance?: boolean } = {}, + ) { + if (isShuttingDown()) return 0; + if (maintenance) { + await enqueueInboundWakeupPublications(); + await reconcileTerminalConfirmationActions(limit); + } + const now = new Date(); + const staleBefore = new Date(now.getTime() - 60_000); + // A process that disappears after the provider accepted a post but before + // Paperclip persisted its message id leaves an ambiguous delivery. Never + // resend it automatically; an operator can explicitly replay after + // checking the provider conversation. + const quarantinedPublications = await db.transaction(async tx => { + const condition = and( + sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`, + eq(chatPublications.state, "streaming"), + lte(chatPublications.updatedAt, staleBefore), + // Teams owns separate staged I/O intents and a longer attempt lease. + // Only that protocol may recover/quarantine its in-flight stages. + notExists( + db + .select({ id: chatTeamsFileTransfers.id }) + .from(chatTeamsFileTransfers) + .where( + and( + eq( + chatTeamsFileTransfers.companyId, + chatPublications.companyId, + ), + eq(chatTeamsFileTransfers.publicationId, chatPublications.id), + ), + ), + ), + notExists( + db + .select({ id: chatEndpointLeases.id }) + .from(chatEndpointLeases) + .where( + and( + eq(chatEndpointLeases.companyId, chatPublications.companyId), + eq( + chatEndpointLeases.endpointId, + chatPublications.endpointId, + ), + sql`${chatEndpointLeases.leaseKey} = 'publication:' || ${chatPublications.id}::text || ':' || ${chatPublications.attempts}::text`, + gt(chatEndpointLeases.expiresAt, now), + ), + ), + ), + ); + const candidates = await tx.select({ id: chatPublications.id }).from(chatPublications).where(condition).for("update", { skipLocked: true }); + if (!candidates.length) return []; + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "delivery_unknown", + nextAttemptAt: null, + redactedError: + "Provider delivery could not be confirmed after the worker stopped. Check the external conversation before replaying.", + updatedAt: now, + }) + .where( + and(condition, inArray(chatPublications.id, candidates.map(row => row.id))), + ) + .returning({ id: chatPublications.id }); + }); + if (quarantinedPublications.length > 0) { + await db + .update(chatActions) + .set({ + status: "delivery_unknown", + result: { + code: "task_control_delivery_unknown", + retryable: false, + }, + updatedAt: now, + }) + .where( + and( + eq(chatActions.kind, "task_control_authorization"), + eq(chatActions.status, "processing"), + inArray( + chatActions.providerActionId, + quarantinedPublications.map( + (publication) => `task-control-authorization:${publication.id}`, + ), + ), + ), + ); + } + const attemptedIds: string[] = []; + const ownedTasks = new Set>(); + let failed = false; + let firstError: unknown; + try { + while (attemptedIds.length < limit && !failed && !isShuttingDown()) { + if (publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY) { + if (!waitForCompletion) break; + await Promise.race(publicationEndpointTasks.values()); + continue; + } + const busyEndpointIds = [...publicationEndpointTasks.keys()]; + for (const [id, retryAfter] of teamsFileRetryAfter) + if (retryAfter <= Date.now()) teamsFileRetryAfter.delete(id); + const coolingTeamsPublicationIds = [...teamsFileRetryAfter.keys()]; + // Select only each conversation's current head before applying the + // global limit. Re-query after every batch so one invocation can still + // drain a conversation's newly unblocked milestones in FIFO order. + const rows = await db + .select() + .from(chatPublications) + .where( + publicationEligibility({ now, busyEndpointIds, coolingTeamsPublicationIds, attemptedIds }), + ) + .orderBy( + asc(chatPublications.createdAt), + asc(publicationTransportOrderKey(chatPublications)), + ) + .limit(limit - attemptedIds.length); + if (rows.length === 0) { + // The query excluded the busy-endpoint snapshot above. A worker + // may have completed while PostgreSQL selected this empty page, + // exposing a new head that must be queried before declaring idle. + if (busyEndpointIds.some((id) => !publicationEndpointTasks.has(id))) + continue; + if (!waitForCompletion || publicationEndpointTasks.size === 0) break; + await Promise.race(publicationEndpointTasks.values()); + continue; + } + let started = false; + for (const selectedPublication of rows) { + if ( + publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY || + attemptedIds.length >= limit || + failed || + isShuttingDown() + ) + break; + if (publicationEndpointTasks.has(selectedPublication.endpointId)) + continue; + attemptedIds.push(selectedPublication.id); + started = true; + // Reserve synchronously before provider work; overlapping drains share + // this cap. Different conversations for one bot keep the existing + // endpoint-exclusive credential fence instead of timing out behind it. + const task = Promise.resolve() + .then(async () => { + await processSelectedPublication(selectedPublication); + await settleExternalConversation(db, selectedPublication.companyId, selectedPublication.issueId).catch((err) => { + logger.warn({ err, publicationId: selectedPublication.id }, "Slack conversation settlement deferred to reconciliation"); + }); + }) + .catch((error: unknown) => { + if (!failed) { + failed = true; + firstError = error; + } + if (!waitForCompletion) { + logger.error( + { + endpointId: selectedPublication.endpointId, + publicationId: selectedPublication.id, + error: redactError(error), + }, + "scheduled chat publication worker failed", + ); + } + }) + .finally(() => { + if ( + publicationEndpointTasks.get(selectedPublication.endpointId) === + task + ) { + publicationEndpointTasks.delete(selectedPublication.endpointId); + } + ownedTasks.delete(task); + wakePublications(); + }); + publicationEndpointTasks.set(selectedPublication.endpointId, task); + ownedTasks.add(task); + } + // Re-query while slots remain: one busy endpoint must not fill the + // candidate page and hide later eligible endpoints behind its backlog. + if (!started && publicationEndpointTasks.size > 0) { + if (!waitForCompletion) break; + await Promise.race(publicationEndpointTasks.values()); + } + } + } catch (error) { + if (!failed) { + failed = true; + firstError = error; + } + } finally { + // Observe rejection immediately above and join every started worker + // before this invocation returns or propagates its original failure. + if (waitForCompletion) await Promise.all(ownedTasks); + } + if (failed) throw firstError; + if (maintenance) await processPendingInteractionWakeups(limit); + return attemptedIds.length; + } + return { processPendingPublications, publicationEligibility }; +} diff --git a/server/src/services/chat-channels/publications/task-control.ts b/server/src/services/chat-channels/publications/task-control.ts new file mode 100644 index 00000000000..ebe1a2b4ee9 --- /dev/null +++ b/server/src/services/chat-channels/publications/task-control.ts @@ -0,0 +1,149 @@ +import type { DbOrTransaction } from "../runtime-types.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { + and, + eq, + gt, + inArray, + like, + or, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatConversations, + chatPublications, + issues, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { projectSafeChatPublication } from "../../chat-publication-projection.js"; + +import { publicationTransportOrderKey } from "./order.js"; +export function createChatPublicationTaskControl(db: Db) { + + + async function currentTaskControlPayload( + publication: typeof chatPublications.$inferSelect, + ): Promise { + const persisted = publication.payload as SafeChatPublicationPayload; + if (!publication.idempotencyKey.startsWith("control:status:")) { + return persisted; + } + const issue = await db + .select({ + identifier: issues.identifier, + status: issues.status, + title: issues.title, + }) + .from(issues) + .where( + and( + eq(issues.companyId, publication.companyId), + eq(issues.id, publication.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!issue) return persisted; + return projectSafeChatPublication({ + classification: "external", + source: "task_control", + text: `${issue.identifier}: ${issue.title} — ${issue.status}`, + }); + } + + function taskControlCompletesConversation( + publication: Pick, + ): boolean { + return /^control:(?:new|close):/.test(publication.idempotencyKey); + } + + async function hasCommittedTaskControlCompletion( + conversationId: string, + database: DbOrTransaction = db, + ): Promise { + return database + .select({ id: chatPublications.id }) + .from(chatPublications) + .where( + and( + eq(chatPublications.conversationId, conversationId), + eq(chatPublications.state, "published"), + or( + like(chatPublications.idempotencyKey, "control:new:%"), + like(chatPublications.idempotencyKey, "control:close:%"), + ), + ), + ) + .limit(1) + .then((rows) => rows.length > 0); + } + + async function commitTaskControlCompletion( + tx: Db, + publication: Pick< + typeof chatPublications.$inferSelect, + | "companyId" + | "conversationId" + | "createdAt" + | "endpointId" + | "id" + | "idempotencyKey" + | "payload" + >, + committedAt: Date, + ): Promise { + if (!taskControlCompletesConversation(publication)) return; + await tx + .update(chatConversations) + .set({ state: "completed", updatedAt: committedAt }) + .where( + and( + eq(chatConversations.companyId, publication.companyId), + eq(chatConversations.endpointId, publication.endpointId), + eq(chatConversations.id, publication.conversationId), + inArray(chatConversations.state, ["active", "waiting"]), + ), + ); + // The provider-visible completion is the external boundary. Anything + // enqueued later for this old generation stays internal even if its run + // finishes after /new or /close. A racing insert that commits after this + // transaction is rejected by the completed-conversation send guard. + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ + state: "cancelled", + nextAttemptAt: null, + redactedError: + "Conversation was completed by a task control before delivery", + updatedAt: committedAt, + }) + .where( + and( + eq(chatPublications.companyId, publication.companyId), + eq(chatPublications.endpointId, publication.endpointId), + eq(chatPublications.conversationId, publication.conversationId), + or( + gt(chatPublications.createdAt, publication.createdAt), + and( + eq(chatPublications.createdAt, publication.createdAt), + gt( + publicationTransportOrderKey(chatPublications), + publication.payload.transportPart?.orderKey ?? publication.id, + ), + ), + ), + inArray(chatPublications.state, ["pending", "retry", "streaming"]), + sql`not (${chatPublications.idempotencyKey} like 'explicit:%' or ${chatPublications.idempotencyKey} like 'explicit-board:%')`, + ), + ); + } + return { currentTaskControlPayload, hasCommittedTaskControlCompletion, commitTaskControlCompletion }; +} diff --git a/server/src/services/chat-channels/publications/transport.ts b/server/src/services/chat-channels/publications/transport.ts new file mode 100644 index 00000000000..e375fe805ed --- /dev/null +++ b/server/src/services/chat-channels/publications/transport.ts @@ -0,0 +1,439 @@ +import type { PublicationTransportInput } from "./types.js"; +import { createPhotonPublicationDelivery } from "../imessage-photon/publications.js"; +import { createOutboundAttachments } from "../outbound-attachments.js"; +import { discordMarkdownAttachment } from "../discord/messages.js"; +import { telegramMarkdownAttachment } from "../telegram/messages.js"; + +import type { EndpointRow, DbOrTransaction } from "../runtime-types.js"; + +import { createChatPublicationReplacement } from "../publication-replacement.js"; + +import { voiceSessionService } from "../../voice/voice-session-service.js"; + +import { voiceSessionStore } from "../../voice/voice-session-store.js"; +import { type ChatRuntime } from "../../chat-runtime.js"; +import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js"; + +import { chatGitHubConfigurations } from "@paperclipai/db"; + +import { githubResponseCommentService } from "../../chat-github-response-comments.js"; + +import { + and, + eq, + isNull, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatMessageLinks, + chatPublications, +} from "@paperclipai/db"; +import type { + ChatAdapterCapabilities, + ChatEndpointSetupState, + ChatProvider, + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { telegramAttachmentForUpload } from "../../chat-telegram-photo.js"; + +import { + conflict, + forbidden, +} from "../../../errors.js"; + +import { + type ChatSdkRuntime, +} from "../../chat-sdk-runtime.js"; + +import { safeChatTaskUrl } from "../../chat-task-url.js"; + +import { + renderPublicationTransportText, +} from "../../chat-publication-text-parts.js"; + +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; + +import { + prepareTeamsInlineImage, +} from "../../chat-teams-inline-images.js"; +import { + shouldStreamSafePublicationText, + streamSafePublicationText, +} from "../../chat-publication-stream.js"; + +import { + isChatQuestionFormOpenActionId, +} from "../../chat-question-forms.js"; + +import type { + AdapterPostableMessage, + Attachment, + FileUpload, +} from "chat"; +import { Actions, Button, Card, CardText, LinkButton } from "chat"; +import type { EndpointRecord } from "../runtime-types.js"; +import { isExplicitOperatorPublication } from "./policy.js"; +type PublicationTransportOptions = { + endpointRecord: (endpointId: string) => Promise; + fetchImpl: typeof globalThis.fetch; + getTaskBaseUrl: () => string | null | undefined; + publicationFiles: ReturnType["publicationFiles"]; + receiptReactionCompletionRunId: ReturnType["receiptReactionCompletionRunId"]; + runIdFromMilestonePublication: ReturnType["runIdFromMilestonePublication"]; + runtime: Pick; + runtimeFor: (endpoint: EndpointRow) => Promise; + runtimeGeneration: (setup: ChatEndpointSetupState) => number; + CAPABILITIES: Record; + voice: Pick, "pushReplies">; + voiceStore: Pick, "enqueuePublication">; + scheduleMessageProcessing: (task: () => Promise) => void; +}; +export function createChatPublicationTransport(db: Db, options: PublicationTransportOptions) { + const { endpointRecord, fetchImpl, getTaskBaseUrl, publicationFiles, receiptReactionCompletionRunId, runIdFromMilestonePublication, runtime, runtimeFor, runtimeGeneration, CAPABILITIES, voice, voiceStore, scheduleMessageProcessing } = options; + const { deliverPhotonPublication } = createPhotonPublicationDelivery(db, { endpointRecord, runtimeGeneration, runtime, receiptReactionCompletionRunId, getTaskBaseUrl }); + + +async function attemptProviderPublication( + send: () => Promise, +): Promise { + return await send(); +} + +async function editOrPostProviderPublication( + edit: () => Promise, + post: () => Promise, +): Promise { + try { + return await edit(); + } catch (error) { + // A definite 404 proves the old progress comment no longer exists. It is + // therefore safe to create a replacement without risking a duplicate. + // Ambiguous transport failures must still enter delivery_unknown. + if (classifyChatPublicationError(error, 1).kind !== "resource_unavailable") + throw error; + return await post(); + } +} + +function safeCardForPublication( + payload: SafeChatPublicationPayload, + provider: ChatProvider, +) { + if (!payload.card) return null; + const children = []; + if (payload.card.body) children.push(CardText(payload.card.body)); + const actions = (payload.card.actions ?? []).map((action) => + action.type === "callback" + ? Button({ + id: action.actionId, + label: action.label, + style: action.style ?? "default", + // Telegram encodes both id and value into callback_data, whose hard + // limit is 64 bytes. Its opaque action id resolves through the + // durable token ledger, so repeating the interaction UUID is both + // unnecessary and too large. Other adapters retain the value as a + // defense-in-depth binding supported by their larger envelopes. + value: provider === "telegram" ? undefined : payload.interactionId, + actionType: isChatQuestionFormOpenActionId(action.actionId) + ? "modal" + : undefined, + }) + : LinkButton({ label: action.label, url: action.url }), + ); + if (actions.length) children.push(Actions(actions)); + return Card({ title: payload.card.title, children }); +} + + async function recordExactOutboundPublicationLink( + tx: DbOrTransaction, + input: { + publication: typeof chatPublications.$inferSelect; + providerMessageId: string; + }, + ): Promise { + const inserted = await tx + .insert(chatMessageLinks) + .values({ + companyId: input.publication.companyId, + endpointId: input.publication.endpointId, + conversationId: input.publication.conversationId, + publicationId: input.publication.id, + commentId: input.publication.commentId, + providerMessageId: input.providerMessageId, + direction: "outbound", + }) + .onConflictDoNothing() + .returning({ id: chatMessageLinks.id }); + if (inserted.length) return true; + return tx + .select({ id: chatMessageLinks.id }) + .from(chatMessageLinks) + .where( + and( + eq(chatMessageLinks.companyId, input.publication.companyId), + eq(chatMessageLinks.endpointId, input.publication.endpointId), + eq(chatMessageLinks.conversationId, input.publication.conversationId), + eq(chatMessageLinks.publicationId, input.publication.id), + input.publication.commentId + ? eq(chatMessageLinks.commentId, input.publication.commentId) + : isNull(chatMessageLinks.commentId), + eq(chatMessageLinks.providerMessageId, input.providerMessageId), + eq(chatMessageLinks.direction, "outbound"), + ), + ) + .limit(1) + .then((rows) => rows.length > 0); + } + + async function postSafePublication(input: PublicationTransportInput) { + if (input.endpoint.provider === "speko") { + const receipt = await voiceStore.enqueuePublication(input.endpoint.companyId, input.publication.id); + scheduleMessageProcessing(() => voice.pushReplies(25, input.endpoint.companyId, input.endpoint.id).then(() => undefined)); + return receipt; + } + if (input.endpoint.provider === "github" && ["failed", "completed"].includes(input.payload.progressState ?? "") && + !input.payload.interactionId && !isExplicitOperatorPublication(input.publication) && + (await db.select({ endpointId: chatGitHubConfigurations.endpointId }).from(chatGitHubConfigurations).where(and( + eq(chatGitHubConfigurations.companyId, input.endpoint.companyId), eq(chatGitHubConfigurations.endpointId, input.endpoint.id))).limit(1)).length > 0) { + const runId = runIdFromMilestonePublication(input.publication); + if (!runId) throw forbidden("GitHub failure has no run binding"); + const receipt = await githubResponseCommentService(db, fetchImpl).failRun({ + companyId: input.publication.companyId, endpointId: input.endpoint.id, + issueId: input.publication.issueId, runId, body: input.payload.progressState === "completed" + ? "This turn ended without publishing a final response. See the Paperclip task for details." + : renderPublicationTransportText(input.payload), + }); + if (!receipt) throw conflict("GitHub response publication is busy"); + return { id: receipt.id, threadId: input.conversation.externalThreadId, raw: {}, text: input.payload.text }; + } + const endpointRuntime = await runtimeFor(input.endpoint); + const thread = endpointRuntime.thread(input.conversation.externalThreadId); + const card = safeCardForPublication(input.payload, input.endpoint.provider); + let attachments: Attachment[] = []; + let files: FileUpload[] = []; + let text = renderPublicationTransportText(input.payload); + if (input.payload.attachmentIds?.length) { + if ( + input.endpoint.provider === "microsoft-teams" && + !input.conversation.isDirectMessage + ) { + // The staged personal-file lane remains authoritative for consent. + // Channel/group pictures need no OneDrive upload or public asset URL: + // send only verified original picture bytes through the native adapter. + const candidates = await publicationFiles( + input.publication, + input.payload, + { teamsInlineImagesOnly: true }, + ); + const pictures = await Promise.all( + candidates.map(prepareTeamsInlineImage), + ); + if (pictures.length && pictures.every((picture) => picture !== null)) { + files = pictures; + } + } + const nativeFileSurface = + CAPABILITIES[input.endpoint.provider].files && + input.endpoint.provider !== "microsoft-teams"; + if (nativeFileSurface) { + const uploads = await publicationFiles( + input.publication, + input.payload, + ); + if (input.endpoint.provider === "telegram") { + attachments = await Promise.all( + uploads.map(telegramAttachmentForUpload), + ); + } else { + files = uploads; + } + } else if (files.length === 0) { + const taskUrl = safeChatTaskUrl( + getTaskBaseUrl(), + input.publication.issueId, + ); + if ( + input.endpoint.provider === "github" || + input.endpoint.provider === "microsoft-teams" + ) { + // GitHub lacks a bot upload contract. Teams reaches this fallback + // only without an admissible personal-consent destination. Replace + // our generated caption, preserving any future custom text. + const generatedFileLabel = /^Shared (.+)\.$/s.exec(text)?.[1] ?? null; + const limitation = + input.endpoint.provider === "github" + ? "This GitHub App connection cannot upload file bytes into comments." + : "Direct file delivery isn't available for this Teams conversation."; + if (generatedFileLabel) { + const saved = taskUrl + ? `File saved on the Paperclip task: ${generatedFileLabel}.` + : `File saved on the private Paperclip task: ${generatedFileLabel}.`; + text = taskUrl + ? `${saved} ${limitation} Download it: ${taskUrl}` + : `${saved} ${limitation}`; + } else { + const handoff = taskUrl + ? `Open the file on its Paperclip task: ${taskUrl}` + : "The file remains available only on the private Paperclip task."; + text = `${text}\n\n${limitation} ${handoff}`; + } + } else { + const attachmentFallback = taskUrl + ? `Open the task in Paperclip: ${taskUrl}` + : "Open the task in Paperclip to download the attachment."; + text = `${text}\n\n${attachmentFallback}`; + } + } + } + if (input.endpoint.provider === "imessage-photon") { + return deliverPhotonPublication(input, endpointRuntime, thread, text, files); + } + if ( + input.endpoint.provider === "discord" && + input.payload.transportPart?.mode === "discord_markdown_attachment" + ) { + files.push(discordMarkdownAttachment(text)); + text = + input.payload.transportPart.count > 1 + ? "Complete response attached." + : "Paperclip attached the complete response because it exceeds Discord’s message limit."; + } + if ( + input.endpoint.provider === "telegram" && + input.payload.transportPart?.mode === "telegram_markdown_attachment" + ) { + attachments.push(telegramMarkdownAttachment(text)); + text = + input.payload.transportPart.count > 1 + ? "Complete response attached." + : "Paperclip attached the complete response to preserve its Markdown formatting."; + } + if (card && CAPABILITIES[input.endpoint.provider].cards) { + return await attemptProviderPublication(async () => + input.replaceProviderMessageId + ? await editOrPostProviderPublication( + () => + thread.adapter.editMessage( + thread.id, + input.replaceProviderMessageId!, + { + card, + fallbackText: text, + ...(files.length ? { files } : {}), + }, + ), + () => + thread.post({ + card, + fallbackText: text, + ...(files.length ? { files } : {}), + }), + ) + : await thread.post({ + card, + fallbackText: text, + ...(files.length ? { files } : {}), + }), + ); + } + if (attachments.length) { + const attachmentMessage: AdapterPostableMessage = { + attachments, + markdown: text, + }; + return await attemptProviderPublication(async () => + input.replaceProviderMessageId + ? await editOrPostProviderPublication( + () => + thread.adapter.editMessage( + thread.id, + input.replaceProviderMessageId!, + attachmentMessage, + ), + () => thread.post(attachmentMessage), + ) + : await thread.post(attachmentMessage), + ); + } + if (files.length) { + // The pinned Slack adapter performs file upload and text publication as + // two provider calls when both fields are present. A rate limit or + // network failure on the second call would make an automatic retry + // duplicate the already-shared file. File publications have their own + // ordered outbox row, so Slack receives a single native upload here. + const fileMessage: AdapterPostableMessage = { + // Empty Slack markdown keeps the SDK call type-safe while its pinned + // adapter treats this as a file-only upload and skips chat.postMessage. + markdown: input.endpoint.provider === "slack" ? "" : text, + files, + }; + if ( + input.endpoint.provider === "slack" && + !input.replaceProviderMessageId && + !input.payload.card && + !input.payload.interactionId && + !input.publication.idempotencyKey.startsWith("control:") && + input.payload.attachmentIds?.length && + input.onSlackFileUploadAccepted + ) { + return await attemptProviderPublication( + async () => + await endpointRuntime.postSlackFilePublication( + thread.id, + fileMessage, + input.onSlackFileUploadAccepted!, + ), + ); + } + return await attemptProviderPublication(async () => + input.replaceProviderMessageId + ? await editOrPostProviderPublication( + () => + thread.adapter.editMessage( + thread.id, + input.replaceProviderMessageId!, + fileMessage, + ), + () => thread.post(fileMessage), + ) + : await thread.post(fileMessage), + ); + } + if ( + !input.replaceProviderMessageId && + !input.payload.transportPart && + CAPABILITIES[input.endpoint.provider].nativeStreaming && + (input.endpoint.provider !== "telegram" || + !input.conversation.isDirectMessage || + input.telegramDraftControl !== undefined) && + shouldStreamSafePublicationText(text) + ) { + return await attemptProviderPublication(async () => + input.telegramDraftControl + ? endpointRuntime.streamTelegramDraft( + input.conversation.externalThreadId, + streamSafePublicationText(text), + input.telegramDraftControl, + ) + : thread.post(streamSafePublicationText(text)), + ); + } + return await attemptProviderPublication(async () => + input.replaceProviderMessageId + ? await editOrPostProviderPublication( + () => + thread.adapter.editMessage( + thread.id, + input.replaceProviderMessageId!, + { markdown: text }, + ), + () => thread.post({ markdown: text }), + ) + : await thread.post({ markdown: text }), + ); + } + return { postSafePublication, recordExactOutboundPublicationLink }; +} diff --git a/server/src/services/chat-channels/publications/types.ts b/server/src/services/chat-channels/publications/types.ts new file mode 100644 index 00000000000..ee4bef967c6 --- /dev/null +++ b/server/src/services/chat-channels/publications/types.ts @@ -0,0 +1,18 @@ +import type { chatConversations, chatPublications } from "@paperclipai/db"; +import type { SafeChatPublicationPayload } from "@paperclipai/shared"; +import type { EndpointRow } from "../runtime-types.js"; +import type { TelegramDraftControl } from "../../chat-telegram-draft-stop.js"; +import type { SlackFileUploadAcceptedReceipt } from "../../chat-sdk-runtime.js"; +type ConversationRow = typeof chatConversations.$inferSelect; +export type PublicationTransportInput = { + endpoint: EndpointRow; + conversation: ConversationRow; + publication: typeof chatPublications.$inferSelect; + payload: SafeChatPublicationPayload; + replaceProviderMessageId?: string | null; + telegramDraftControl?: TelegramDraftControl; + beforePhotonWrite?(): Promise; + onSlackFileUploadAccepted?: ( + receipt: SlackFileUploadAcceptedReceipt, + ) => Promise; + }; diff --git a/server/src/services/chat-channels/publications/worker.ts b/server/src/services/chat-channels/publications/worker.ts new file mode 100644 index 00000000000..8a205ae5775 --- /dev/null +++ b/server/src/services/chat-channels/publications/worker.ts @@ -0,0 +1,1081 @@ +import { createGitHubPublicationPolicy } from "../github/publication-policy.js"; + +import { createTeamsFileDelivery } from "../microsoft-teams/file-delivery.js"; + +import { createReceiptReactions } from "../receipt-reactions.js"; + +import { createTelegramPublicationDrafts } from "../telegram/publication-drafts.js"; +import { createSlackFileReceipts } from "../slack/file-receipts.js"; + +import { createTelegramMaintenance } from "../telegram/maintenance.js"; +import type { EndpointRow, DbOrTransaction, RuntimeContext } from "../runtime-types.js"; + +import { createChatPublicationReplacement } from "../publication-replacement.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { + stageReceiptReactionRemovals, +} from "../../chat-receipt-reactions.js"; + +import { chatCredentialMutationLease, CREDENTIAL_MUTATION_LEASE_TTL_MS, type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import { slackExplicitPublicationDuplicate } from "../../connectors/slack-publication.js"; + +import { PhotonError } from "../../photon/cloud.js"; + +import { + randomUUID, +} from "node:crypto"; +import { + and, + eq, + inArray, + lt, + or, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatConversations, + chatEndpointLeases, + chatEndpointResources, + chatEndpoints, + chatMessageLinks, + chatPublications, + chatTeamsFileTransfers, + issueThreadInteractions, +} from "@paperclipai/db"; +import type { + ChatAdapterCapabilities, + ChatProvider, + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { + isTelegramDraftStopped, +} from "../../chat-telegram-draft-stop.js"; + +import { logger } from "../../../middleware/logger.js"; + +import { + renderPublicationTransportText, +} from "../../chat-publication-text-parts.js"; + +import { + stageSlackSessionSync, +} from "../../chat-slack-session-outbox.js"; + +import { classifyChatPublicationError } from "../../chat-publication-errors.js"; +import { + parseInboundWakePublicationKey, +} from "../../chat-inbound-wakeup-publications.js"; + +import { + shouldStreamSafePublicationText, +} from "../../chat-publication-stream.js"; + +import { publicationTransportOrderKey } from "./order.js"; +import { isExplicitOperatorPublication } from "./policy.js"; +import type { EndpointRecord } from "../runtime-types.js"; +import type { createChatPublicationAuthorization } from "./authorization.js"; +import type { createChatPublicationTaskControl } from "./task-control.js"; +import type { createChatPublicationFailures } from "./failures.js"; +import type { createChatPublicationPreparation } from "./preparation.js"; +import type { createChatPublicationTransport } from "./transport.js"; +type PublicationWorkerOptions = Pick, "runOwnershipMilestoneSupersessionReason" | "runPublicationToReplace" | "receiptReactionCompletionRunId" | "inboundWakePublicationToReplace" | "interactionResolutionPublicationToReplace" | "interactionPromptPublicationToReplace" | "progressSupersededByFastResponse" | "runProgressSupersededByPublishedInteraction" | "taskStatusPublicationToReplace" | "closeProgressToReplace"> & { + CAPABILITIES: Record; + authorizeRetainedChatSourcePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise; + claimPublicationTransportAuthorization: ReturnType["claimPublicationTransportAuthorization"]; + commitTaskControlCompletion: ReturnType["commitTaskControlCompletion"]; + currentTaskControlPayload: ReturnType["currentTaskControlPayload"]; + endpointRecord: (endpointId: string) => Promise; + ensureDurablePublicationTransport: ReturnType["ensureDurablePublicationTransport"]; + finalizePublicationFailure: ReturnType["finalizePublicationFailure"]; + hasCommittedTaskControlCompletion: ReturnType["hasCommittedTaskControlCompletion"]; + hasTelegramStopSubscription: ReturnType["hasTelegramStopSubscription"]; + nonDirectDestinationAllowed: (endpoint: EndpointRow, resource: typeof chatEndpointResources.$inferSelect | null | undefined) => boolean; + postSafePublication: ReturnType["postSafePublication"]; + processReceiptReaction: ReturnType["processReceiptReaction"]; + recordSlackFileUploadReceipt: ReturnType["recordSlackFileUploadReceipt"]; + redactError: (error: unknown) => string; + reserveTelegramPublicationDraft: ReturnType["reserveTelegramPublicationDraft"]; + runtimeContextForRecord: (record: EndpointRecord) => RuntimeContext; + scheduleMessageProcessing: (task: () => Promise) => void; + settleGitHubAutomaticPublication: ReturnType["settleGitHubAutomaticPublication"]; + stageSlackFilePublicationSuccess: ReturnType["stageSlackFilePublicationSuccess"]; + tryProcessTeamsFilePublication: ReturnType["tryProcessTeamsFilePublication"]; + withCredentialMutationLease: ReturnType; +}; +export function createChatPublicationWorker(db: Db, options: PublicationWorkerOptions) { + const { CAPABILITIES, authorizeRetainedChatSourcePublication, claimPublicationTransportAuthorization, closeProgressToReplace, commitTaskControlCompletion, currentTaskControlPayload, endpointRecord, ensureDurablePublicationTransport, finalizePublicationFailure, hasCommittedTaskControlCompletion, hasTelegramStopSubscription, inboundWakePublicationToReplace, interactionPromptPublicationToReplace, interactionResolutionPublicationToReplace, nonDirectDestinationAllowed, postSafePublication, processReceiptReaction, progressSupersededByFastResponse, recordSlackFileUploadReceipt, redactError, reserveTelegramPublicationDraft, runOwnershipMilestoneSupersessionReason, runProgressSupersededByPublishedInteraction, runPublicationToReplace, runtimeContextForRecord, scheduleMessageProcessing, settleGitHubAutomaticPublication, stageSlackFilePublicationSuccess, taskStatusPublicationToReplace, tryProcessTeamsFilePublication, withCredentialMutationLease, receiptReactionCompletionRunId } = options; + + + async function stageTerminalReceiptReactionRemovals( + tx: Db, + input: { + endpoint: EndpointRow; + publication: typeof chatPublications.$inferSelect; + payload: SafeChatPublicationPayload; + runtimeContext: RuntimeContext; + closedProgressRunId?: string; + }, + ): Promise { + if ( + !["discord", "slack", "telegram", "github"].includes(input.endpoint.provider) + ) return []; + const runId = input.closedProgressRunId ?? + (await receiptReactionCompletionRunId(tx, input.publication, input.payload)); + if (!runId) return []; + return stageReceiptReactionRemovals(tx, { + endpoint: input.endpoint, + binding: input.publication, + runId, + runtimeContext: input.runtimeContext, + }); + } + + async function settlePublicationPreparationFailure( + publication: typeof chatPublications.$inferSelect, + error: unknown, + ): Promise { + const attempts = publication.attempts + 1; + const classified = classifyChatPublicationError(error, attempts); + // No provider I/O has started at this boundary. Unknown transport or + // database failures are therefore safe to retry; only a definite local + // validation rejection (or repeated exhaustion) is terminal. + const terminal = classified.kind === "failed" || attempts >= 5; + const retryAfterMs = + classified.kind === "retry" + ? classified.retryAfterMs + : Math.min(60_000, 2 ** Math.max(0, attempts) * 1_000); + const result = await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + const [settled] = await tx + .update(chatPublications) + .set({ + state: terminal ? "failed" : "retry", + attempts, + nextAttemptAt: terminal ? null : new Date(Date.now() + retryAfterMs), + redactedError: terminal + ? "Publication payload could not be prepared for provider delivery" + : "Publication preparation failed before provider delivery", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, publication.state), + eq(chatPublications.attempts, publication.attempts), + ), + ) + .returning({ id: chatPublications.id }); + if (!settled) return false; + if (terminal && publication.idempotencyKey.startsWith("control:")) { + await tx + .update(chatActions) + .set({ + status: "cancelled", + result: { + code: "task_control_publication_invalid", + attempts, + retryable: false, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.endpointId, publication.endpointId), + eq( + chatActions.providerActionId, + `task-control-authorization:${publication.id}`, + ), + eq(chatActions.status, "issued"), + ), + ); + } + return true; + }); + if (result) { + logger.warn( + { + endpointId: publication.endpointId, + publicationId: publication.id, + error: redactError(error), + terminal, + }, + "chat publication preparation failed", + ); + } + return result; + } + + async function withPublicationAttemptLease( + publication: typeof chatPublications.$inferSelect, + operation: (lease: CredentialMutationLeaseGuard) => Promise, + ): Promise { + // A scheduled sweep may overlap a slow provider call (also on a standby + // server). Claim and exact-attempt ownership must become visible together: + // endpoint-wide activity is not evidence that this publication is alive. + const token = randomUUID(); + const leaseKey = `publication:${publication.id}:${publication.attempts + 1}`; + const now = new Date(); + const owned = and( + eq(chatEndpointLeases.companyId, publication.companyId), + eq(chatEndpointLeases.endpointId, publication.endpointId), + eq(chatEndpointLeases.leaseKey, leaseKey), + eq(chatEndpointLeases.token, token), + ); + const claimed = await db.transaction(async (tx) => { + // Transfer issuance locks this same publication. Check after acquiring + // that lock, not through a pre-lock snapshot that could miss a new lane. + const current = await tx + .select({ id: chatPublications.id }) + .from(chatPublications) + .where( + and( + eq(chatPublications.companyId, publication.companyId), + eq(chatPublications.id, publication.id), + eq(chatPublications.state, publication.state), + eq(chatPublications.attempts, publication.attempts), + ), + ) + .for("update") + .then((rows) => rows[0]); + if (!current) return false; + const transfer = await tx + .select({ id: chatTeamsFileTransfers.id }) + .from(chatTeamsFileTransfers) + .where( + and( + eq(chatTeamsFileTransfers.companyId, publication.companyId), + eq(chatTeamsFileTransfers.publicationId, current.id), + ), + ) + .then((rows) => rows[0]); + if (transfer) return false; + await notifyChatPublicationWork(tx); + const [row] = await tx + .update(chatPublications) + .set({ + state: "streaming", + attempts: publication.attempts + 1, + updatedAt: now, + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, publication.state), + eq(chatPublications.attempts, publication.attempts), + ), + ) + .returning({ id: chatPublications.id }); + if (!row) return false; + await tx.insert(chatEndpointLeases).values({ + companyId: publication.companyId, + endpointId: publication.endpointId, + leaseKey, + token, + expiresAt: new Date(now.getTime() + CREDENTIAL_MUTATION_LEASE_TTL_MS), + }); + return true; + }); + if (!claimed) return; + let leaseLoss: Error | null = null; + const assertOwned: CredentialMutationLeaseGuard["assertOwned"] = async ( + database = db, + ) => { + if (leaseLoss) throw leaseLoss; + try { + // Use a savepoint when the caller already owns a transaction. The + // decision clock is sampled only after the lease row lock resolves; + // an UPDATE predicate using a pre-lock clock can revive expired work. + await database.transaction(async (tx) => { + const current = await tx + .select({ expiresAt: chatEndpointLeases.expiresAt }) + .from(chatEndpointLeases) + .where(owned) + .for("update") + .then((rows) => rows[0] ?? null); + const checkedAt = new Date(); + if ( + leaseLoss || + !current || + current.expiresAt.getTime() <= checkedAt.getTime() + ) { + throw new Error("Publication attempt lease is no longer owned"); + } + await tx + .update(chatEndpointLeases) + .set({ + expiresAt: new Date( + checkedAt.getTime() + CREDENTIAL_MUTATION_LEASE_TTL_MS, + ), + updatedAt: checkedAt, + }) + .where(owned); + }); + } catch (error) { + leaseLoss = Object.assign( + new Error( + "Chat publication attempt ownership was lost before completion", + { cause: error }, + ), + { code: "CHAT_PUBLICATION_LEASE_LOST" }, + ); + throw leaseLoss; + } + }; + let renewal: Promise | null = null; + const timer = setInterval(() => { + if (renewal || leaseLoss) return; + renewal = assertOwned() + .catch(() => undefined) + .finally(() => { + renewal = null; + }); + }, CREDENTIAL_MUTATION_LEASE_TTL_MS / 3); + timer.unref?.(); + try { + await operation({ assertOwned }); + } finally { + clearInterval(timer); + await renewal; + await db.delete(chatEndpointLeases).where(owned); + } + } + + async function processSelectedPublication( + selectedPublication: typeof chatPublications.$inferSelect, + ): Promise { + if (await settleGitHubAutomaticPublication(selectedPublication)) return; + let publication: typeof chatPublications.$inferSelect; + try { + publication = + await ensureDurablePublicationTransport(selectedPublication); + } catch (error) { + try { + await settlePublicationPreparationFailure(selectedPublication, error); + } catch (settlementError) { + logger.error( + { + endpointId: selectedPublication.endpointId, + publicationId: selectedPublication.id, + error: redactError(settlementError), + }, + "could not durably settle chat publication preparation failure", + ); + } + return; + } + const explicitSlackDelivery = await slackExplicitPublicationDuplicate(db, publication); + if (explicitSlackDelivery === "unresolved") { + // Keep the durable publication pending until the explicit send is settled; + // a missing provider receipt does not authorize a duplicate. + return; + } + if (explicitSlackDelivery === "delivered") { + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx.update(chatPublications).set({ state: "cancelled", redactedError: "Identical response already delivered by Slack tool", updatedAt: new Date() }).where(and(eq(chatPublications.id, publication.id), inArray(chatPublications.state, ["pending", "retry"]))) +; + }); + return; + } + const earlierOpenPublication = await db + .select({ id: chatPublications.id }) + .from(chatPublications) + .where( + and( + eq(chatPublications.conversationId, publication.conversationId), + or( + lt(chatPublications.createdAt, publication.createdAt), + and( + eq(chatPublications.createdAt, publication.createdAt), + lt( + publicationTransportOrderKey(chatPublications), + publication.payload.transportPart?.orderKey ?? publication.id, + ), + ), + ), + inArray(chatPublications.state, [ + "pending", + "retry", + "streaming", + "delivery_unknown", + ]), + ), + ) + .limit(1) + .then((result) => result[0] ?? null); + if (earlierOpenPublication) return; + if (await tryProcessTeamsFilePublication(publication)) return; + await withPublicationAttemptLease(publication, async (publicationLease) => { + let providerAccepted = false; + let publicationRuntimeContext: RuntimeContext | null = null; + let failureHandledWithinCredentialLease = false; + let slackFileReceiptActionId: string | null = null; + const settlePublicationFailure = async ( + error: unknown, + credentialLease?: CredentialMutationLeaseGuard, + ) => { + try { + await finalizePublicationFailure( + publication, + error, + providerAccepted, + publicationRuntimeContext, + credentialLease ?? publicationLease, + ); + } catch (finalizationError) { + // If the provider accepted the message, a failed receipt write + // must remain streaming until the stale scan quarantines it. A + // later lifecycle mutation must not turn this into an implicit + // replay or allow a stale failure to overwrite the new runtime. + logger.error( + { + endpointId: publication.endpointId, + publicationId: publication.id, + error: redactError(finalizationError), + }, + "could not durably finalize chat publication failure", + ); + } + }; + try { + const recordForLease = await endpointRecord(publication.endpointId); + if (!recordForLease) + throw new Error("Chat publication binding is unavailable"); + const currentPublicationRuntimeContext = + runtimeContextForRecord(recordForLease); + publicationRuntimeContext = currentPublicationRuntimeContext; + const [endpoint, conversation] = await Promise.all([ + db + .select() + .from(chatEndpoints) + .where(eq(chatEndpoints.id, publication.endpointId)) + .then((result) => result[0] ?? null), + db + .select() + .from(chatConversations) + .where(eq(chatConversations.id, publication.conversationId)) + .then((result) => result[0] ?? null), + ]); + if (!endpoint || !conversation) + throw new Error("Chat publication binding is unavailable"); + const providerVisibleCompletion = + conversation.state === "completed" && + !isExplicitOperatorPublication(publication) && + (await hasCommittedTaskControlCompletion(conversation.id)); + if (endpoint.status === "paused" || endpoint.status === "attention") { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "pending", + attempts: publication.attempts, + // Pause can race the eligibility read. The query excludes + // inactive endpoints on the next pass, so retain only the + // original provider deadline rather than delaying Resume. + nextAttemptAt: publication.nextAttemptAt, + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + // The setup conversation is a real end-to-end test: once provider + // credentials are verified, its safe agent response must be able to + // reach the provider before the operator confirms the final wizard + // step. Draft, paused, revoked, and archived endpoints remain closed. + if ( + !["active", "verifying"].includes(endpoint.status) || + ["unavailable", "endpoint_removed"].includes(conversation.state) || + providerVisibleCompletion + ) { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "cancelled", + redactedError: providerVisibleCompletion + ? "Conversation was completed before delivery" + : "External destination is no longer active", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + if (conversation.isDirectMessage) { + if (!endpoint.allowDirectMessages) { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "cancelled", + redactedError: "Direct messages are disabled in Paperclip", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + } else { + const resource = conversation.resourceId + ? await db + .select() + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.id, conversation.resourceId), + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.companyId, endpoint.companyId), + ), + ) + .then((result) => result[0] ?? null) + : null; + if (!nonDirectDestinationAllowed(endpoint, resource)) { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "cancelled", + redactedError: "Destination is disabled in Paperclip", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + } + const interactionId = publication.payload.interactionId; + const isInteractionPrompt = + Boolean(interactionId) && + publication.idempotencyKey === + `interaction:${interactionId}:${publication.endpointId}`; + if (isInteractionPrompt && publication.issueId) { + const currentInteraction = await db + .select({ status: issueThreadInteractions.status }) + .from(issueThreadInteractions) + .where( + and( + eq(issueThreadInteractions.id, interactionId!), + eq(issueThreadInteractions.companyId, publication.companyId), + eq(issueThreadInteractions.issueId, publication.issueId), + ), + ) + .then((result) => result[0] ?? null); + if (!currentInteraction || currentInteraction.status !== "pending") { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "cancelled", + attempts: publication.attempts, + nextAttemptAt: null, + redactedError: "Interaction resolved before provider delivery", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + } + // Milestone selection and insertion can overlap across sweeps. An old + // snapshot inserted after newer attention must not overwrite it simply + // because it now occupies the next FIFO slot. + const progressSupersessionReason = + (await runOwnershipMilestoneSupersessionReason(publication)) ?? + ((await progressSupersededByFastResponse(publication)) ? "Acknowledged by contextual fast response" : null) ?? + ((await runProgressSupersededByPublishedInteraction(publication)) + ? "Run progress was superseded by its provider interaction" + : null); + if (progressSupersessionReason) { + await publicationLease.assertOwned(); + await db.transaction(async (tx) => { + await notifyChatPublicationWork(tx); + return tx + .update(chatPublications) + .set({ + state: "cancelled", + attempts: publication.attempts, + nextAttemptAt: null, + redactedError: progressSupersessionReason, + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + }); + return; + } + // Status is sampled when it reaches the head of the provider lane, + // not when the command was admitted. If an already-streaming final + // publication won the race, this reply reflects Paperclip's latest + // authoritative task state after that earlier send commits. + let payload = await currentTaskControlPayload(publication); + await withCredentialMutationLease( + recordForLease.endpoint, + async (credentialGuard) => { + const credentialLease: CredentialMutationLeaseGuard = { + assertOwned: async (database = db) => { + await credentialGuard.assertOwned(database); + await publicationLease.assertOwned(database); + }, + }; + try { + if (await settleGitHubAutomaticPublication(publication, credentialLease)) return; + // A prior failure's provider ID may have been reused while this + // worker waited for the endpoint lane. Select against current + // outbound links only after owning that lane, before the final + // source authorization claim and provider I/O. + const closedProgress = await closeProgressToReplace( + publication, + endpoint, + ); + const replaceProviderMessageId = endpoint.provider !== "imessage-photon" && CAPABILITIES[endpoint.provider] + .messageEdits + ? (closedProgress?.providerMessageId ?? + (await interactionResolutionPublicationToReplace( + publication, + payload, + )) ?? + (await interactionPromptPublicationToReplace( + publication, + payload, + )) ?? + (await runPublicationToReplace(publication, payload)) ?? + (await inboundWakePublicationToReplace( + publication, + payload, + )) ?? + (await taskStatusPublicationToReplace(publication, payload))) + : null; + const authorizationClaim = + await claimPublicationTransportAuthorization({ + credentialLease, + endpoint: recordForLease.endpoint, + publication, + runtimeContext: currentPublicationRuntimeContext, + }); + if (!authorizationClaim) { + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ + state: "cancelled", + nextAttemptAt: null, + redactedError: + "Task control requester or destination is no longer authorized", + updatedAt: new Date(), + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ); + await tx + .update(chatActions) + .set({ + status: "cancelled", + result: { + code: "task_control_authorization_changed", + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.endpointId, publication.endpointId), + eq( + chatActions.providerActionId, + `task-control-authorization:${publication.id}`, + ), + eq(chatActions.status, "issued"), + ), + ); + await credentialLease.assertOwned(tx); + }); + return; + } + // The durable claim above is the authorization linearization + // point. Provider I/O runs without a database transaction or + // row lock. The renewable lease preserves credential/runtime + // identity until conditional settlement. + if (authorizationClaim.preparedText !== null) + payload = { ...payload, text: authorizationClaim.preparedText }; + const telegramDraft = + authorizationClaim.endpoint.provider === "telegram" && + authorizationClaim.conversation.isDirectMessage && + !replaceProviderMessageId && + !payload.transportPart && + !payload.card && + !payload.attachmentIds?.length && + shouldStreamSafePublicationText( + renderPublicationTransportText(payload), + ) && + (await hasTelegramStopSubscription( + authorizationClaim.endpoint, + currentPublicationRuntimeContext, + )) + ? await reserveTelegramPublicationDraft({ + endpoint: authorizationClaim.endpoint, + conversation: authorizationClaim.conversation, + publication, + text: renderPublicationTransportText(payload), + credentialLease, + runtimeContext: currentPublicationRuntimeContext, + }) + : null; + const sent = await postSafePublication({ + endpoint: authorizationClaim.endpoint, + conversation: authorizationClaim.conversation, + publication, + payload, + replaceProviderMessageId, + telegramDraftControl: telegramDraft?.control, + beforePhotonWrite: () => db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + if (!(await authorizeRetainedChatSourcePublication(tx, publication))) throw new PhotonError("rejected", "Publication source authorization changed"); + }), + onSlackFileUploadAccepted: async (receipt) => { + // uploadV2 has completed at this point. Mark acceptance + // before the durable callback so a local write failure is + // still quarantined and can never trigger an implicit upload. + providerAccepted = true; + slackFileReceiptActionId = await recordSlackFileUploadReceipt( + { + credentialLease, + endpoint: authorizationClaim.endpoint, + conversation: authorizationClaim.conversation, + publication, + receipt, + runtimeContext: currentPublicationRuntimeContext, + }, + ); + }, + }); + if (isTelegramDraftStopped(sent)) { + // Only the durable stop transaction can produce this outcome. + // It settled this exact publication; never fabricate a provider + // receipt, complete its task, or enqueue a retry from this path. + if (!telegramDraft) + throw new Error("Unexpected Telegram draft stop result"); + const stopped = await db + .select({ id: chatActions.id }) + .from(chatActions) + .innerJoin( + chatPublications, + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "cancelled"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ) + .where( + and( + eq(chatActions.id, telegramDraft.actionId), + eq(chatActions.status, "cancelled"), + sql`${chatActions.result}->>'phase' = 'stopped'`, + ), + ); + if (stopped.length !== 1) + throw new Error( + "Telegram draft stop was not durably committed", + ); + return; + } + providerAccepted = true; + // A resolved SDK call is not a usable delivery receipt. Teams, + // for example, returns an empty ID for an empty HTTP success + // body. Preserve uncertainty instead of publishing a broken + // message link or automatically sending the content again. + if ( + typeof sent?.id !== "string" || + !sent.id || + sent.id.length > 2048 || + sent.id.trim() !== sent.id || + /[\u0000-\u001f\u007f]/.test(sent.id) + ) { + throw new Error( + "Provider did not return a usable message receipt", + ); + } + const { authorizationActionId } = authorizationClaim; + const receiptRemovalActionIds: string[] = []; + await db.transaction(async (tx) => { + await credentialLease.assertOwned(tx); + const committedAt = new Date(); + await notifyChatPublicationWork(tx); + const [completedPublication] = await tx + .update(chatPublications) + .set({ + state: "published", + payload, + providerMessageId: sent.id, + publishedAt: committedAt, + redactedError: null, + updatedAt: committedAt, + }) + .where( + and( + eq(chatPublications.id, publication.id), + eq(chatPublications.state, "streaming"), + eq(chatPublications.attempts, publication.attempts + 1), + ), + ) + .returning({ id: chatPublications.id }); + if (!completedPublication) { + throw new Error( + "Chat publication ownership changed before commit", + ); + } + if (telegramDraft) { + const completedDraft = await tx + .update(chatActions) + .set({ + status: "processed", + result: { + phase: "published", + providerMessageId: sent.id, + }, + updatedAt: committedAt, + }) + .where( + and( + eq(chatActions.id, telegramDraft.actionId), + eq(chatActions.status, "processing"), + sql`${chatActions.result}->>'phase' = 'final_sending'`, + ), + ) + .returning({ id: chatActions.id }); + if (completedDraft.length !== 1) + throw new Error( + "Telegram final draft ownership changed before commit", + ); + } + if (slackFileReceiptActionId) { + const [completedReceipt] = await tx + .update(chatActions) + .set({ + status: "processed", + result: { + code: "slack_file_upload_identity_confirmed", + attempts: 1, + }, + updatedAt: committedAt, + }) + .where( + and( + eq(chatActions.id, slackFileReceiptActionId), + eq(chatActions.kind, "slack_file_upload_receipt"), + eq(chatActions.status, "received"), + sql`(${chatActions.payload}->>'publicationId')::uuid = ${publication.id}`, + sql`(${chatActions.payload}->>'publicationAttempt')::int = ${publication.attempts + 1}`, + ), + ) + .returning({ id: chatActions.id }); + if (!completedReceipt) { + throw new Error( + "Slack file upload receipt ownership changed before commit", + ); + } + } + if (slackFileReceiptActionId) { + await stageSlackFilePublicationSuccess(tx, { + committedAt, + endpoint: authorizationClaim.endpoint, + publication, + providerMessageId: sent.id, + runtimeContext: currentPublicationRuntimeContext, + stageSessionSync: + !isExplicitOperatorPublication(publication) && + !publication.idempotencyKey.startsWith("control:status:"), + }); + } else { + // A queue notice references its causal inbound comment for + // authorization, but is not an authored copy of that comment. + // Reactions/readers must not attribute bot status to the user. + const linkedCommentId = parseInboundWakePublicationKey( + publication.idempotencyKey, + ) + ? null + : publication.commentId; + const messageLinkInsert = tx.insert(chatMessageLinks).values({ + companyId: publication.companyId, + endpointId: publication.endpointId, + conversationId: publication.conversationId, + publicationId: publication.id, + commentId: linkedCommentId, + providerMessageId: sent.id, + direction: "outbound", + }); + if (replaceProviderMessageId) { + await messageLinkInsert.onConflictDoUpdate({ + target: [ + chatMessageLinks.endpointId, + chatMessageLinks.conversationId, + chatMessageLinks.providerMessageId, + ], + set: { + publicationId: publication.id, + commentId: linkedCommentId, + }, + }); + } else { + await messageLinkInsert.onConflictDoNothing(); + } + await tx + .update(chatEndpoints) + .set({ + lastPublicationAt: committedAt, + updatedAt: committedAt, + }) + .where( + and( + eq(chatEndpoints.id, authorizationClaim.endpoint.id), + inArray(chatEndpoints.status, ["verifying", "active"]), + sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer, 0) = ${currentPublicationRuntimeContext.generation}`, + ), + ); + } + if (authorizationActionId) { + const processedAuthorization = await tx + .update(chatActions) + .set({ + status: "processed", + result: { code: "task_control_authorized_and_sent" }, + updatedAt: committedAt, + }) + .where( + and( + eq(chatActions.id, authorizationActionId), + eq(chatActions.status, "processing"), + sql`(${chatActions.result}->>'attempts')::int = ${publication.attempts + 1}`, + ), + ) + .returning({ id: chatActions.id }); + if (!processedAuthorization.length) { + throw new Error( + "Task-control authorization ownership changed before commit", + ); + } + } + await commitTaskControlCompletion( + tx as unknown as Db, + publication, + committedAt, + ); + if ( + !slackFileReceiptActionId && + authorizationClaim.endpoint.provider === "slack" && + !isExplicitOperatorPublication(publication) && + !parseInboundWakePublicationKey(publication.idempotencyKey) && + !publication.idempotencyKey.startsWith("control:status:") + ) { + await stageSlackSessionSync(tx, { + companyId: publication.companyId, + endpointId: publication.endpointId, + conversationId: publication.conversationId, + runtimeGeneration: + currentPublicationRuntimeContext.generation, + credentialFingerprint: + currentPublicationRuntimeContext.credentialFingerprint, + }); + } + receiptRemovalActionIds.push( + ...(await stageTerminalReceiptReactionRemovals( + tx as unknown as Db, + { + endpoint: authorizationClaim.endpoint, + publication, + payload, + runtimeContext: currentPublicationRuntimeContext, + closedProgressRunId: closedProgress?.runId, + }, + )), + ); + await credentialLease.assertOwned(tx); + }); + // Receipt cleanup is non-critical provider I/O. Dispatch it + // after the terminal reply and durable action commit so a + // slow reaction endpoint cannot hold the publication lane. + // If this process stops first, the action outbox resumes the + // idempotent removal on a later reconciliation sweep. + for (const actionId of receiptRemovalActionIds) { + scheduleMessageProcessing(() => + processReceiptReaction(actionId), + ); + } + } catch (error) { + // Authentication, membership, and destination failures mutate + // endpoint/runtime state. Settle them before releasing the + // credential lease so a concurrent reconnect or provider + // recovery cannot be overwritten by this stale attempt. + failureHandledWithinCredentialLease = true; + // If ownership was already lost, leave the publication in its + // conservative streaming state for quarantine. Mutating health + // from this obsolete runtime would overwrite the new owner. + await settlePublicationFailure(error, credentialLease); + } + }, + ); + } catch (error) { + if (!failureHandledWithinCredentialLease) + await settlePublicationFailure(error); + } + }); + } + return { processSelectedPublication, stageTerminalReceiptReactionRemovals }; +} diff --git a/server/src/services/chat-channels/runtime-types.ts b/server/src/services/chat-channels/runtime-types.ts index 809d5050e0c..c0c9dce4631 100644 --- a/server/src/services/chat-channels/runtime-types.ts +++ b/server/src/services/chat-channels/runtime-types.ts @@ -1,3 +1,5 @@ +import type { Message, Thread } from "chat"; +import type { ChatSdkMessageCallbackEvent } from "../chat-sdk-runtime.js"; import type { Db, chatEndpoints } from "@paperclipai/db"; import type { ToolCredentialSecretRef, ChatEndpointSetupState, ChatEndpointCallbackSurfaces } from "@paperclipai/shared"; import type { ChatEndpointRuntime } from "../voice/voice-runtime.js"; @@ -66,3 +68,26 @@ export type PreparedProviderInventory = { inventory: import("../chat-provider-inventory.js").ChatProviderInventoryResult | null; managementUrl?: string; }; + +export type LiveInboundMessage = { + endpoint: EndpointRow; + message: Message; + receiptReactionSupported: boolean; + providerUrl: string | null; + runtimeContext?: InboundRuntimeContext; + thread: Thread; + trigger: ChatSdkMessageCallbackEvent["trigger"]; +}; + +export type DiscordGatewayOwnership = { + companyId: string; + context: RuntimeContext; + endpointId: string; + expiresAt: Date; + leaseKey: "discord_gateway_runtime" | "photon_receiver_runtime"; + renewTimer: ReturnType | null; + renewal: Promise | null; + stopPromise: Promise | null; + stopping: boolean; + token: string; +}; diff --git a/server/src/services/chat-channels/sdk-state.ts b/server/src/services/chat-channels/sdk-state.ts new file mode 100644 index 00000000000..5fb2f86db91 --- /dev/null +++ b/server/src/services/chat-channels/sdk-state.ts @@ -0,0 +1,88 @@ +import { + and, + eq, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatSdkState, +} from "@paperclipai/db"; + +import type { + ChatSdkStateCompareAndSetInput, + ChatSdkStateDeleteInput, + ChatSdkStatePersistence, + ChatSdkStateScope, +} from "../chat-sdk-state.js"; + +export function createChatSdkStatePersistence(db: Db): ChatSdkStatePersistence { + return { + async read(scope: ChatSdkStateScope, key: string) { + return db + .select({ + value: chatSdkState.value, + version: chatSdkState.version, + expiresAt: chatSdkState.expiresAt, + }) + .from(chatSdkState) + .where( + and( + eq(chatSdkState.companyId, scope.companyId), + eq(chatSdkState.endpointId, scope.endpointId), + eq(chatSdkState.stateKey, key), + ), + ) + .then((rows) => rows[0] ?? null); + }, + async compareAndSet(input: ChatSdkStateCompareAndSetInput) { + if (input.expectedVersion === null) { + const inserted = await db + .insert(chatSdkState) + .values({ + companyId: input.companyId, + endpointId: input.endpointId, + stateKey: input.key, + version: 1, + value: input.value, + expiresAt: input.expiresAt, + }) + .onConflictDoNothing() + .returning({ id: chatSdkState.id }); + return inserted.length === 1; + } + const updated = await db + .update(chatSdkState) + .set({ + version: input.expectedVersion + 1, + value: input.value, + expiresAt: input.expiresAt, + updatedAt: new Date(), + }) + .where( + and( + eq(chatSdkState.companyId, input.companyId), + eq(chatSdkState.endpointId, input.endpointId), + eq(chatSdkState.stateKey, input.key), + eq(chatSdkState.version, input.expectedVersion), + ), + ) + .returning({ id: chatSdkState.id }); + return updated.length === 1; + }, + async deleteIfVersion(input: ChatSdkStateDeleteInput) { + const deleted = await db + .delete(chatSdkState) + .where( + and( + eq(chatSdkState.companyId, input.companyId), + eq(chatSdkState.endpointId, input.endpointId), + eq(chatSdkState.stateKey, input.key), + eq(chatSdkState.version, input.expectedVersion), + ), + ) + .returning({ id: chatSdkState.id }); + return deleted.length === 1; + }, + }; +} diff --git a/server/src/services/chat-channels/slack/setup.ts b/server/src/services/chat-channels/slack/setup.ts new file mode 100644 index 00000000000..3f442d11a1d --- /dev/null +++ b/server/src/services/chat-channels/slack/setup.ts @@ -0,0 +1,89 @@ +import { slackCommandForAgent } from "./commands.js"; +import { canonicalCallbackUrl, slackCallbackMatchesPublicUrl } from "./callback-verification.js"; +import type { EndpointRow, SlackCallbackSurface, InternalSetupState } from "../runtime-types.js"; + +import type { + ChatEndpointCallbackSurfaces, +} from "@paperclipai/shared"; + +export function slackSetupState(input: { + endpoint: Pick< + EndpointRow, + | "provider" + | "providerAccountId" + | "publicId" + | "status" + | "setup" + | "botUsername" + >; + webhookUrl: string | null; + step: EndpointRow["setup"]["step"]; + assignedAgentName?: string | null; +}) { + const { endpoint, webhookUrl, step, assignedAgentName } = input; + + const observations = (endpoint.setup as InternalSetupState) + .slackCallbackSurfaces; + const currentUrl = webhookUrl ? canonicalCallbackUrl(webhookUrl) : null; + const surfaceState = ( + surface: SlackCallbackSurface, + ): ChatEndpointCallbackSurfaces[SlackCallbackSurface] => { + const observation = observations?.[surface]; + if (!observation) return { status: "unverified" }; + return { + status: + currentUrl !== null && + slackCallbackMatchesPublicUrl(observation.url, currentUrl) + ? "current" + : "stale", + observedAt: observation.observedAt, + }; + }; + const callbackSurfaces: ChatEndpointCallbackSurfaces = { + events: surfaceState("events"), + interactivity: surfaceState("interactivity"), + slashCommands: surfaceState("slashCommands"), + }; + return { + step, + testStartedAt: endpoint.setup.testStartedAt ?? null, + testSkipped: endpoint.setup.testSkipped ?? false, + authorizationUrl: "https://api.slack.com/apps?new_app=1", + providerUrl: "https://app.slack.com/", + webhookUrl, + webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null, + callbackSurfaces, + callbacksNeedUpdate: Object.values(callbackSurfaces).some( + (surface) => surface.status === "stale", + ), + slackApp: endpoint.setup.slackApp, + slackSetupMethod: endpoint.setup.slackSetupMethod, + slackAvatar: endpoint.setup.slackAvatar, + slackManagerError: endpoint.setup.slackManagerError, + slackAccount: endpoint.setup.slackAccount, + // Slack registers the slash command in the provider configuration. + // Keep that identity immutable when the assigned agent is renamed; + // deriving it remains only a compatibility path for older rows. + command: + typeof endpoint.setup.command === "string" + ? endpoint.setup.command + : assignedAgentName + ? slackCommandForAgent(assignedAgentName, endpoint.publicId) + : null, + } as const; + +} + +import type { ResolvedChatSdkProviderConfig } from "../../chat-sdk-runtime.js"; + +export function slackRuntimeConfig(endpoint: EndpointRow, userName: string, credentials: Record): ResolvedChatSdkProviderConfig { + return { + provider: "slack", + userName, + credentials: { + botToken: credentials.botToken, + signingSecret: credentials.signingSecret, + botUserId: endpoint.botExternalId ?? undefined, + }, + }; +} diff --git a/server/src/services/chat-channels/speko/setup.ts b/server/src/services/chat-channels/speko/setup.ts new file mode 100644 index 00000000000..cda78300620 --- /dev/null +++ b/server/src/services/chat-channels/speko/setup.ts @@ -0,0 +1,37 @@ +import type { EndpointRow } from "../runtime-types.js"; + +import { syncSpekoVoiceTools } from "../../voice/speko-agent-tools.js"; +import { configureSpekoSessionTools } from "../../voice/speko-tool-setup.js"; +import { createSpekoProvider } from "../../voice/speko-provider.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import type { Db } from "@paperclipai/db"; + +import { + forbidden, + unprocessable, +} from "../../../errors.js"; + +import type { DbTransaction } from "../runtime-types.js"; +type SpekoSetupOptions = { + fetchImpl: typeof globalThis.fetch; + getWebhookPublicBaseUrl: () => string | null; + allowLocalVoiceBoard?: boolean; +}; +export function createSpekoSetup(db: Db, options: SpekoSetupOptions) { + const { fetchImpl, getWebhookPublicBaseUrl } = options; + + async function installSpekoVoiceTools(tx: DbTransaction, endpoint: EndpointRow, actorUserId?: string | null) { + if (!actorUserId) throw forbidden("An authenticated connection owner is required to install phone tools"); + await syncSpekoVoiceTools(tx, endpoint, actorUserId, true, options.allowLocalVoiceBoard === true); + } + + async function configureSpekoTools(endpoint: EndpointRow, credentials: Record, credentialLease: CredentialMutationLeaseGuard, previousSpekoSigningSecret: string | undefined) { + const webhookPublicBaseUrl = getWebhookPublicBaseUrl(); + if (!webhookPublicBaseUrl?.startsWith("https://")) throw unprocessable("Speko needs an explicitly configured public HTTPS callback origin"); + await configureSpekoSessionTools(db, { companyId: endpoint.companyId, endpointId: endpoint.id, agentId: credentials.agentId, callbackUrl: `${webhookPublicBaseUrl}/api/voice-webhooks/${endpoint.publicId}/tools`, signingSecret: credentials.signingSecret, previousSigningSecret: previousSpekoSigningSecret, client: createSpekoProvider(credentials.apiKey, fetchImpl), assertOwned: () => credentialLease.assertOwned() }); + } + + return { installSpekoVoiceTools, configureSpekoTools }; +} diff --git a/server/src/services/chat-channels/telegram/confirmation-actions.ts b/server/src/services/chat-channels/telegram/confirmation-actions.ts new file mode 100644 index 00000000000..8e5c9971b8f --- /dev/null +++ b/server/src/services/chat-channels/telegram/confirmation-actions.ts @@ -0,0 +1,656 @@ +import type { DbTransaction, RuntimeContext, LifecycleRuntimeFence } from "../runtime-types.js"; + +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; + +import { + and, + asc, + eq, + inArray, + ne, + sql, +} from "drizzle-orm"; + +import type { Db } from "@paperclipai/db"; + +import { + chatActions, + chatConversations, + chatEndpoints, + chatExternalPrincipals, + chatPublications, + issueThreadInteractions, + issues, +} from "@paperclipai/db"; + +import { + type ChatSdkCallbackEvent, +} from "../../chat-sdk-runtime.js"; + +import { + logActivity, +} from "../../activity-log.js"; + +import { projectSafeChatPublication } from "../../chat-publication-projection.js"; + +import { issueThreadInteractionService } from "../../issue-thread-interactions.js"; + +import type { EndpointRecord } from "../runtime-types.js"; +import type { ActionEvent } from "chat"; +import { nativeTelegramConfirmation } from "../../chat-interaction-publications.js"; +type TelegramConfirmationInput = { + record: EndpointRecord; + event: ChatSdkCallbackEvent; + runtimeContext: RuntimeContext; + conversation: typeof chatConversations.$inferSelect; + principal: { principal: typeof chatExternalPrincipals.$inferSelect; userId: string }; + issued: { publication: typeof chatPublications.$inferSelect }; + interaction: Awaited["listForIssue"]>>[number]; + deny: (known?: { conversationId?: string | null; principalId?: string | null }) => Promise; + safelyKnown: { conversationId: string; principalId: string }; +}; +type TelegramConfirmationOptions = { + requireCurrentExternalActionAuthorization: (tx: DbTransaction, input: { conversationId: string; endpointId: string; expectedUserId: string; principalId: string; runtimeContext: LifecycleRuntimeFence }) => Promise; + isExternalActionAuthorizationChange: (error: unknown) => boolean; + scheduleMessageProcessing: (task: () => Promise) => void; + processPendingPublications: () => Promise; + confirmationResolutionPersistBarrier?: () => Promise; +}; +export function createTelegramConfirmationActions(db: Db, options: TelegramConfirmationOptions) { + const { requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, scheduleMessageProcessing, processPendingPublications } = options; + async function handleTelegramConfirmation(input: TelegramConfirmationInput) { + const { record, event, runtimeContext, conversation, principal, issued, interaction, deny, safelyKnown } = input; + + if ( + event.provider !== "telegram" || + !nativeTelegramConfirmation(interaction) + ) { + return deny(safelyKnown); + } + const action = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.companyId, record.endpoint.companyId), + eq(chatActions.endpointId, record.endpoint.id), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.kind, "confirmation_response"), + eq(chatActions.providerActionId, event.event.actionId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!action || action.status !== "issued") return deny(safelyKnown); + const tokenPayload = action.payload; + const tokenExpiresAt = + typeof tokenPayload.expiresAt === "string" + ? Date.parse(tokenPayload.expiresAt) + : Number.NaN; + if (!Number.isFinite(tokenExpiresAt) || tokenExpiresAt <= Date.now()) { + await db + .update(chatActions) + .set({ + status: "expired", + result: { code: "confirmation_action_token_expired" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "issued"), + ), + ); + return deny(safelyKnown); + } + if ( + tokenPayload.version !== 1 || + tokenPayload.publicationId !== issued.publication.id || + tokenPayload.interactionId !== interaction.id || + (tokenPayload.decision !== "accept" && + tokenPayload.decision !== "reject") + ) { + return deny(safelyKnown); + } + + const issue = await db + .select({ + id: issues.id, + companyId: issues.companyId, + projectId: issues.projectId, + goalId: issues.goalId, + status: issues.status, + assigneeAgentId: issues.assigneeAgentId, + }) + .from(issues) + .where( + and( + eq(issues.companyId, record.endpoint.companyId), + eq(issues.id, conversation.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!issue) { + return deny(safelyKnown); + } + + const decision = tokenPayload.decision; + try { + if (decision === "accept") { + await issueThreadInteractionService(db).acceptInteraction( + issue, + interaction.id, + {}, + { userId: principal.userId }, + { + beforeResolveInTransaction: async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + }, + afterResolveInTransaction: async (tx, resolved) => { + const [owned] = await tx + .update(chatActions) + .set({ + principalId: principal.principal.id, + status: "processing", + payload: { + ...tokenPayload, + messageId: event.event.messageId, + actionId: event.event.actionId, + value: null, + }, + result: { + code: "confirmation_resolution_committed_by_provider", + interactionId: resolved.id, + interactionStatus: resolved.status, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "issued"), + ), + ) + .returning({ id: chatActions.id }); + if (!owned) { + throw new Error( + "External confirmation action ownership changed before interaction commit", + ); + } + }, + }, + ); + } else { + await issueThreadInteractionService(db).rejectInteraction( + issue, + interaction.id, + {}, + { userId: principal.userId }, + { + beforeResolveInTransaction: async (tx) => { + await requireCurrentExternalActionAuthorization(tx, { + conversationId: conversation.id, + endpointId: record.endpoint.id, + expectedUserId: principal.userId!, + principalId: principal.principal.id, + runtimeContext, + }); + }, + afterResolveInTransaction: async (tx, resolved) => { + const [owned] = await tx + .update(chatActions) + .set({ + principalId: principal.principal.id, + status: "processing", + payload: { + ...tokenPayload, + messageId: event.event.messageId, + actionId: event.event.actionId, + value: null, + }, + result: { + code: "confirmation_resolution_committed_by_provider", + interactionId: resolved.id, + interactionStatus: resolved.status, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "issued"), + ), + ) + .returning({ id: chatActions.id }); + if (!owned) { + throw new Error( + "External confirmation action ownership changed before interaction commit", + ); + } + }, + }, + ); + } + } catch (error) { + if (isExternalActionAuthorizationChange(error)) { + return deny(safelyKnown); + } + const currentStatus = await db + .select({ status: issueThreadInteractions.status }) + .from(issueThreadInteractions) + .where(eq(issueThreadInteractions.id, interaction.id)) + .then((rows) => rows[0]?.status ?? null); + if (currentStatus !== null && currentStatus !== "pending") { + await settleTerminalConfirmationAction(action.id); + const completedAction = await db + .select({ status: chatActions.status, result: chatActions.result }) + .from(chatActions) + .where(eq(chatActions.id, action.id)) + .then((rows) => rows[0] ?? null); + if ( + completedAction?.status === "processed" && + completedAction.result?.code === + "confirmation_resolution_committed_by_provider" + ) { + return; + } + return deny(safelyKnown); + } + throw error; + } + + await options.confirmationResolutionPersistBarrier?.(); + if (!(await settleTerminalConfirmationAction(action.id))) { + throw new Error( + "External confirmation action ownership changed before settlement", + ); + } + + scheduleMessageProcessing(async () => { + await processPendingPublications(); + }); + return; + + } + + + async function settleTerminalConfirmationAction( + actionId: string, + ): Promise { + const action = await db + .select() + .from(chatActions) + .where( + and( + eq(chatActions.id, actionId), + eq(chatActions.kind, "confirmation_response"), + ), + ) + .then((rows) => rows[0] ?? null); + if (action?.status === "processed") return true; + if (!action || action.status !== "processing" || !action.conversationId) + return false; + const token = action.payload; + if ( + token.version !== 1 || + typeof token.publicationId !== "string" || + typeof token.interactionId !== "string" || + (token.decision !== "accept" && token.decision !== "reject") + ) { + await db + .update(chatActions) + .set({ + status: "failed", + result: { code: "confirmation_action_payload_invalid" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ); + return false; + } + const [conversation, publication, endpoint] = await Promise.all([ + db + .select() + .from(chatConversations) + .where( + and( + eq(chatConversations.companyId, action.companyId), + eq(chatConversations.endpointId, action.endpointId), + eq(chatConversations.id, action.conversationId), + ), + ) + .then((rows) => rows[0] ?? null), + db + .select() + .from(chatPublications) + .where( + and( + eq(chatPublications.companyId, action.companyId), + eq(chatPublications.endpointId, action.endpointId), + eq(chatPublications.id, token.publicationId as string), + ), + ) + .then((rows) => rows[0] ?? null), + db + .select() + .from(chatEndpoints) + .where( + and( + eq(chatEndpoints.companyId, action.companyId), + eq(chatEndpoints.id, action.endpointId), + ), + ) + .then((rows) => rows[0] ?? null), + ]); + if ( + !conversation || + !publication || + !endpoint || + endpoint.provider !== "telegram" || + publication.conversationId !== conversation.id || + publication.payload.interactionId !== token.interactionId + ) { + await db + .update(chatActions) + .set({ + status: "failed", + result: { code: "confirmation_action_binding_invalid" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ); + return false; + } + const interaction = ( + await issueThreadInteractionService(db).listForIssue(conversation.issueId) + ).find((candidate) => candidate.id === token.interactionId); + if ( + !interaction || + interaction.kind !== "request_confirmation" || + !nativeTelegramConfirmation(interaction) + ) { + await db + .update(chatActions) + .set({ + status: "failed", + result: { code: "confirmation_interaction_invalid" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ); + return false; + } + if ( + interaction.status !== "accepted" && + interaction.status !== "rejected" + ) { + if (interaction.status === "pending") return false; + const [retired] = await db + .update(chatActions) + .set({ + status: "expired", + result: { + code: "confirmation_interaction_resolved_elsewhere", + interactionId: interaction.id, + interactionStatus: interaction.status, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ) + .returning({ id: chatActions.id }); + if (retired) return true; + const winner = await db + .select({ status: chatActions.status }) + .from(chatActions) + .where(eq(chatActions.id, action.id)) + .then((rows) => rows[0] ?? null); + return winner?.status === "expired" || winner?.status === "processed"; + } + const decisionMatches = + (token.decision === "accept" && interaction.status === "accepted") || + (token.decision === "reject" && interaction.status === "rejected"); + if (!decisionMatches) { + await db + .update(chatActions) + .set({ + status: "expired", + result: { code: "interaction_resolved_by_sibling_action" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ); + return false; + } + const issue = await db + .select({ + id: issues.id, + companyId: issues.companyId, + assigneeAgentId: issues.assigneeAgentId, + status: issues.status, + }) + .from(issues) + .where( + and( + eq(issues.companyId, action.companyId), + eq(issues.id, conversation.issueId), + ), + ) + .then((rows) => rows[0] ?? null); + if (!issue || !interaction.resolvedByUserId) { + await db + .update(chatActions) + .set({ + status: "failed", + result: { code: "confirmation_resolution_context_missing" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ); + return false; + } + const resolvedByUserId = interaction.resolvedByUserId; + const shouldWake = + Boolean(issue.assigneeAgentId) && + issue.status !== "done" && + issue.status !== "cancelled" && + (interaction.continuationPolicy === "wake_assignee" || + (interaction.continuationPolicy === "wake_assignee_on_accept" && + interaction.status === "accepted")); + const resolutionLabel = + interaction.status === "accepted" ? "Accepted" : "Rejected"; + const resolvedByThisProviderAction = + action.result?.code === "confirmation_resolution_committed_by_provider"; + return db.transaction(async (tx) => { + const [completed] = await tx + .update(chatActions) + .set({ + status: "processed", + result: { + code: resolvedByThisProviderAction + ? "confirmation_resolution_committed_by_provider" + : "confirmation_resolution_reconciled", + interactionId: interaction.id, + interactionStatus: interaction.status, + }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.id, action.id), + eq(chatActions.status, "processing"), + ), + ) + .returning({ id: chatActions.id }); + if (!completed) { + // A publication sweep may reconcile the durable action while the + // original provider callback is still in flight. Treat that CAS loss + // as success once the winner has processed the same action so the + // provider does not receive a spurious 5xx and redeliver it. + const winner = await tx + .select({ status: chatActions.status }) + .from(chatActions) + .where(eq(chatActions.id, action.id)) + .then((rows) => rows[0] ?? null); + return winner?.status === "processed"; + } + await tx + .update(chatActions) + .set({ + status: "expired", + result: { code: "interaction_resolved_by_sibling_action" }, + updatedAt: new Date(), + }) + .where( + and( + eq(chatActions.companyId, issue.companyId), + eq(chatActions.endpointId, action.endpointId), + eq(chatActions.conversationId, conversation.id), + eq(chatActions.kind, "confirmation_response"), + inArray(chatActions.status, ["issued", "processing"]), + ne(chatActions.id, action.id), + eq( + sql`${chatActions.payload}->>'interactionId'`, + interaction.id, + ), + ), + ); + await notifyChatPublicationWork(tx); + await tx + .insert(chatPublications) + .values({ + companyId: issue.companyId, + endpointId: action.endpointId, + conversationId: conversation.id, + issueId: issue.id, + idempotencyKey: `interaction-resolution:${interaction.id}:${action.endpointId}`, + payload: projectSafeChatPublication({ + classification: "external", + source: "issue_interaction", + text: `${resolutionLabel}: ${interaction.payload.prompt}`, + interaction: { + id: interaction.id, + card: { + kind: "confirmation", + title: interaction.payload.prompt, + body: resolutionLabel, + actions: [], + }, + }, + }), + state: "pending", + }) + .onConflictDoNothing(); + if (shouldWake && issue.assigneeAgentId) { + await tx + .insert(chatActions) + .values({ + companyId: issue.companyId, + endpointId: action.endpointId, + conversationId: conversation.id, + principalId: action.principalId, + kind: "interaction_wakeup", + providerActionId: `interaction_wakeup:${interaction.id}`, + payload: { + version: 1, + interactionId: interaction.id, + interactionKind: interaction.kind, + interactionStatus: interaction.status, + issueId: issue.id, + agentId: issue.assigneeAgentId, + sourceCommentId: interaction.sourceCommentId ?? null, + sourceRunId: interaction.sourceRunId ?? null, + requestedByUserId: resolvedByUserId, + requestedByActorType: "user", + requestedByActorId: resolvedByUserId, + }, + status: "issued", + }) + .onConflictDoNothing(); + } + // A provider callback can lose the pending -> terminal interaction race + // to the Paperclip UI. In that case this transaction only retires the + // now-redundant provider action and must not fabricate a second external + // resolution event attributed to the board winner. + if (resolvedByThisProviderAction) { + await logActivity(tx as unknown as Db, { + companyId: issue.companyId, + actorType: "user", + actorId: resolvedByUserId, + action: + interaction.status === "accepted" + ? "issue.thread_interaction_accepted" + : "issue.thread_interaction_rejected", + entityType: "issue", + entityId: issue.id, + details: { + source: "external_chat", + endpointId: action.endpointId, + provider: endpoint.provider, + conversationId: conversation.id, + publicationId: publication.id, + providerMessageId: publication.providerMessageId, + interactionId: interaction.id, + interactionKind: interaction.kind, + interactionStatus: interaction.status, + resolutionActorKind: "user", + requestedResolverPolicy: interaction.requestedResolverPolicy, + effectiveResolverPolicy: interaction.effectiveResolverPolicy, + }, + }); + } + return true; + }); + } + + async function reconcileTerminalConfirmationActions( + limit = 100, + ): Promise { + const candidates = await db + .select({ id: chatActions.id }) + .from(chatActions) + .where( + and( + eq(chatActions.kind, "confirmation_response"), + eq(chatActions.status, "processing"), + ), + ) + .orderBy(asc(chatActions.updatedAt), asc(chatActions.id)) + .limit(limit); + let settled = 0; + for (const candidate of candidates) { + if (await settleTerminalConfirmationAction(candidate.id)) settled += 1; + } + return settled; + } + return { settleTerminalConfirmationAction, reconcileTerminalConfirmationActions, handleTelegramConfirmation }; +} diff --git a/server/src/services/chat-channels/telegram/destination-admission.ts b/server/src/services/chat-channels/telegram/destination-admission.ts new file mode 100644 index 00000000000..03edf2a6b4b --- /dev/null +++ b/server/src/services/chat-channels/telegram/destination-admission.ts @@ -0,0 +1,62 @@ +import type { Thread } from "chat"; +import { and, eq, ne, sql } from "drizzle-orm"; +import { chatEndpointResources } from "@paperclipai/db"; +import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js"; +import { nonDirectDestinationAllowed } from "../inbound/surfaces.js"; + +type DestinationAdmissionInput = { + endpoint: EndpointRow; + currentEndpoint: Pick; + thread: Thread; + addressed: boolean; +}; +type DestinationAdmissionOptions = { + ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise; +}; +export function createTelegramDestinationAdmission(options: DestinationAdmissionOptions) { + const { ensureResource } = options; + async function admitTelegramDestination(input: DestinationAdmissionInput, tx: DbTransaction) { + const { endpoint, currentEndpoint, thread, addressed } = input; + let destinationAccepting = true; + + let resource = await ensureResource(endpoint, thread, false, tx); + const enabledResourceCount = await tx + .select({ count: sql`count(*)::int` }) + .from(chatEndpointResources) + .where( + and( + eq(chatEndpointResources.endpointId, endpoint.id), + eq(chatEndpointResources.enabled, true), + eq(chatEndpointResources.availability, "available"), + ne(chatEndpointResources.type, "direct_message"), + ne(chatEndpointResources.type, "group_chat"), + ), + ) + .then((rows) => rows[0]?.count ?? 0); + const canActivateFirstSetupGroup = + currentEndpoint.status === "verifying" && + resource.availability === "available" && + enabledResourceCount === 0 && + addressed; + if (canActivateFirstSetupGroup && !resource.enabled) { + resource = await tx + .update(chatEndpointResources) + .set({ enabled: true, updatedAt: new Date() }) + .where(eq(chatEndpointResources.id, resource.id)) + .returning() + .then((rows) => rows[0] ?? resource); + } + // Telegram's privacy-mode contract is explicit: in a group or forum + // topic, every admitted turn must address the bot. A direct reply to a + // bot message is normalized as a mention by the pinned adapter, while + // unrelated subscribed traffic remains unaddressed and is filtered. + destinationAccepting = + addressed && + nonDirectDestinationAllowed( + { ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats }, + resource, + ); + return destinationAccepting; + } + return { admitTelegramDestination }; +} diff --git a/server/src/services/chat-channels/telegram/publication-preparation.ts b/server/src/services/chat-channels/telegram/publication-preparation.ts new file mode 100644 index 00000000000..638a63d8cd3 --- /dev/null +++ b/server/src/services/chat-channels/telegram/publication-preparation.ts @@ -0,0 +1,192 @@ +import { createChatPublicationReplacement } from "../publication-replacement.js"; +import { notifyChatPublicationWork } from "../../chat-work-notifications.js"; +import { + eq, + sql, +} from "drizzle-orm"; +import type { Db } from "@paperclipai/db"; +import { + chatPublications, +} from "@paperclipai/db"; +import type { + SafeChatPublicationPayload, +} from "@paperclipai/shared"; + +import { + splitTelegramPublicationText, + telegramMarkdownRequiresAttachment, +} from "../../chat-publication-stream.js"; +type PublicationPreparationOptions = { + runPublicationToReplace: ReturnType["runPublicationToReplace"]; +}; +export function createTelegramPublicationPreparation(db: Db, options: PublicationPreparationOptions) { + const { runPublicationToReplace } = options; + async function prepareTelegramPublication(publication: typeof chatPublications.$inferSelect, persisted: SafeChatPublicationPayload): Promise {if ( + persisted.transportPart || + persisted.card || + persisted.interactionId || + persisted.attachmentIds?.length + ) { + return publication; + } + if (telegramMarkdownRequiresAttachment(persisted.text)) { + // Telegram parses each message independently. A fixed-size split can + // turn the second half of a code fence, link, or list into unrelated + // plain text even though concatenating the source parts is lossless. + // Preserve structured long Markdown as one document. When this is a run + // final, first complete its existing progress placeholder, then send the + // file through a separate ordered outbox row because Telegram cannot add + // an attachment while editing that placeholder. + const replacesProgressMessage = Boolean( + await runPublicationToReplace(publication, persisted), + ); + return db.transaction(async (tx) => { + const current = await tx + .select() + .from(chatPublications) + .where(eq(chatPublications.id, publication.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) return publication; + const currentPayload = current.payload as SafeChatPublicationPayload; + if ( + currentPayload.transportPart || + currentPayload.card || + currentPayload.interactionId || + currentPayload.attachmentIds?.length || + !["pending", "retry"].includes(current.state) || + !telegramMarkdownRequiresAttachment(currentPayload.text) + ) { + return current; + } + + const batchId = current.id; + const updatedAt = new Date(); + const transportPart = ( + index: number, + count: number, + mode: "inline" | "telegram_markdown_attachment", + ) => ({ + batchId, + count, + index, + mode, + orderKey: `${batchId}:${String(index).padStart(4, "0")}`, + }); + if (!replacesProgressMessage) { + const attachmentPayload: SafeChatPublicationPayload = { + ...currentPayload, + transportPart: transportPart(0, 1, "telegram_markdown_attachment"), + }; + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: attachmentPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + return { ...current, payload: attachmentPayload, updatedAt }; + } + + const handoffPayload: SafeChatPublicationPayload = { + ...currentPayload, + text: "This response needs a separate attachment because it exceeds the message limit.", + transportPart: transportPart(0, 2, "inline"), + }; + delete handoffPayload.attachmentIds; + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: handoffPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + + const attachmentPayload: SafeChatPublicationPayload = { + ...currentPayload, + transportPart: transportPart(1, 2, "telegram_markdown_attachment"), + }; + delete attachmentPayload.progressState; + await notifyChatPublicationWork(tx); + await tx.insert(chatPublications).values({ + companyId: current.companyId, + endpointId: current.endpointId, + conversationId: current.conversationId, + issueId: current.issueId, + commentId: current.commentId, + idempotencyKey: `telegram-markdown-attachment:${current.id}`, + payload: attachmentPayload, + state: "pending", + createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, + updatedAt, + }); + return { ...current, payload: handoffPayload, updatedAt }; + }); + } + const parts = splitTelegramPublicationText(persisted.text); + if (parts.length === 1) return publication; + + return db.transaction(async (tx) => { + const current = await tx + .select() + .from(chatPublications) + .where(eq(chatPublications.id, publication.id)) + .for("update") + .then((rows) => rows[0] ?? null); + if (!current) return publication; + const currentPayload = current.payload as SafeChatPublicationPayload; + if ( + currentPayload.transportPart || + currentPayload.card || + currentPayload.interactionId || + currentPayload.attachmentIds?.length || + !["pending", "retry"].includes(current.state) + ) { + return current; + } + const currentParts = splitTelegramPublicationText(currentPayload.text); + if (currentParts.length === 1) return current; + + const batchId = current.id; + const updatedAt = new Date(); + const transportPart = (index: number) => ({ + batchId, + count: currentParts.length, + index, + orderKey: `${batchId}:${String(index).padStart(4, "0")}`, + }); + const rootPayload: SafeChatPublicationPayload = { + ...currentPayload, + text: currentParts[0]!, + transportPart: transportPart(0), + }; + await notifyChatPublicationWork(tx); + await tx + .update(chatPublications) + .set({ payload: rootPayload, updatedAt }) + .where(eq(chatPublications.id, current.id)); + + const { progressState: _progressState, ...nonProgressPayload } = + currentPayload; + await notifyChatPublicationWork(tx); + await tx.insert(chatPublications).values( + currentParts.slice(1).map((text, offset) => { + const index = offset + 1; + return { + companyId: current.companyId, + endpointId: current.endpointId, + conversationId: current.conversationId, + issueId: current.issueId, + commentId: current.commentId, + idempotencyKey: `${current.idempotencyKey}:telegram-part:${index}`, + payload: { + ...nonProgressPayload, + text, + transportPart: transportPart(index), + }, + state: "pending" as const, + createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`, + updatedAt, + }; + }), + ); + return { ...current, payload: rootPayload, updatedAt }; + });} + return { prepareTelegramPublication }; +} diff --git a/server/src/services/chat-channels/telegram/setup.ts b/server/src/services/chat-channels/telegram/setup.ts new file mode 100644 index 00000000000..5e297103629 --- /dev/null +++ b/server/src/services/chat-channels/telegram/setup.ts @@ -0,0 +1,110 @@ +import { createTelegramMaintenance } from "./maintenance.js"; +import type { EndpointRow, RuntimeContext } from "../runtime-types.js"; + +import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js"; + +import type { Db } from "@paperclipai/db"; + +import type { + ConfigureChatEndpointInput, +} from "@paperclipai/shared"; + +import { + unprocessable, +} from "../../../errors.js"; + +import type { EndpointRecord } from "../runtime-types.js"; + +type TelegramSetupOptions = { + fetchImpl: typeof globalThis.fetch; + getWebhookPublicBaseUrl: () => string | null; + PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number; + runtimeContextForRecord: (record: EndpointRecord) => RuntimeContext; + recordTelegramStopSubscription: ReturnType["recordTelegramStopSubscription"]; + stageTelegramMaintenance: ReturnType["stageTelegramMaintenance"]; + processTelegramMaintenance: ReturnType["processTelegramMaintenance"]; +}; +export function createTelegramSetup(db: Db, options: TelegramSetupOptions) { + const { fetchImpl, getWebhookPublicBaseUrl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, runtimeContextForRecord, recordTelegramStopSubscription, stageTelegramMaintenance, processTelegramMaintenance } = options; + + async function registerTelegramWebhook(endpoint: EndpointRow, input: ConfigureChatEndpointInput, credentials: Record, credentialLease: CredentialMutationLeaseGuard, next: EndpointRecord) { + const webhookUrl = `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/telegram`; + const infoResponse = await fetchImpl( + `https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/getWebhookInfo`, + { signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS) }, + ); + const infoResult = (await infoResponse.json()) as { + ok?: boolean; + description?: string; + result?: { url?: string }; + }; + if (!infoResponse.ok || !infoResult.ok) { + throw unprocessable( + `Telegram could not inspect the existing webhook: ${infoResult.description ?? infoResponse.status}`, + ); + } + await credentialLease.assertOwned(); + const response = await fetchImpl( + `https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/setWebhook`, + { + method: "POST", + signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS), + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + url: webhookUrl, + secret_token: credentials.webhookSecret, + allowed_updates: [ + "message", + "edited_message", + "callback_query", + "stopped_message_generation", + "message_reaction", + "my_chat_member", + ], + // A first-time connection may inherit stale updates from a bot + // that was used before Paperclip owned it. Reconnect is + // different: Telegram may be holding legitimate updates while + // the old callback URL is unavailable. Preserve that backlog + // even when an operator is rotating the public origin. + drop_pending_updates: input.action === "configure", + }), + }, + ); + const result = (await response.json()) as { + ok?: boolean; + description?: string; + result?: unknown; + }; + if (!response.ok || !result.ok) + throw unprocessable( + `Telegram could not register the webhook: ${result.description ?? response.status}`, + ); + await credentialLease.assertOwned(); + if (result.ok === true && result.result === true) { + // Telegram's successful boolean receipt acknowledges the exact URL, + // secret and allowed_updates request above. GET alone cannot prove + // secret_token, which Telegram deliberately does not return. + await recordTelegramStopSubscription( + next.endpoint, + runtimeContextForRecord(next), + credentialLease, + ); + } + // Command discovery is optional for ingress, but its provider mutation + // is still a durable, auditable action. A transient or rate-limited + // failure is retried by the maintenance worker without blocking the + // otherwise-valid webhook connection. + const commandAction = await stageTelegramMaintenance( + endpoint, + "register_commands", + ); + if (commandAction) + await processTelegramMaintenance( + commandAction.id, + false, + credentialLease, + ); + } + + return { registerTelegramWebhook }; +}