Files
PaperClipAI/server/src/services/chat-channels.ts
T
DottaandPaperclip 09a2f0b7b0 refactor(server): isolate provider message admission and runtime policies (#16289)
## Thinking Path

> - Paperclip helps people manage AI agents for work.
> - Chat channels turn external messages into shared task workflows.
> - The main service still contains some provider decisions.
> - These decisions belong with the existing channel modules.
> - This pull request isolates provider message-admission and runtime
policies.
> - Shared authorization, persistence, and runtime ownership keep their
current order.

## Linked Issues or Issue Description

Refs: #16235. Stack part 3/3. Base: `master`. #16276 and #16283 are
merged.

**What existing behavior does this improve?**

Provider ownership and readability in the chat channel service.

**Current behavior**

The entry point combines shared workflows with provider decisions.

**Proposed behavior**

Provider modules own their decisions. Shared modules own product state
and authorization.

**Reason and benefit**

Maintainers can find provider behavior in its channel folder.

**Breaking changes**

None intended. Public service methods and durable data shapes stay
compatible.

## What Changed

- Move Discord SDK admission and its barrier to the Discord message
module.
- Move Photon live-message eviction and startup recovery policy to its
runtime callbacks module.
- Move Slack mention eligibility and Teams setup-channel eligibility to
their message modules.
- Keep Photon deferred ingress and Speko conversation policy with their
provider modules.
- Keep ordered SDK message batching in a shared runtime module with a
live per-message ingress callback.
- Keep receiver leases, cache lifetime, startup failure cleanup,
authorization locks, and transaction ownership shared.
- Reduce `chat-channels.ts` from 3,570 to 3,548 lines.

## Verification

- The same 46 runtime/admission integration cases and all 18 Photon
channel cases passed before and after in the owned Linux container.
- The existing first-channel Teams orphan-admission case passed before
and after moving its eligibility predicate.
- The same 94 SDK adapter and lifecycle-normalization tests passed
locally.
- Server typecheck and module-boundary checks passed.
- Token comparison confirms ordered SDK batching and every processor
argument, plus Photon invalidation and its live runtime lookup.
- The shared factory creates no cache or worker. It reads the ingress
policy for each message and awaits processing in order.
- The extraction rebased without conflicts onto master after #16276 and
#16283 merged. Git range-diff reports the same patch. Final head
`7b63b569e2a0dbcc959310f565f77f367fa64aa0` has the same patch by
range-diff. The 94 SDK/lifecycle tests, server typecheck, and boundary
check passed on the final head. All CI gates pass on that exact head.
Greptile reviewed it at 5/5 with no inline findings or unresolved review
threads.
- One Runner CI test failed its original short timing window. The
unchanged test passed earlier-head CI and local reproduction. The single
failed-job retry passed.
- Cumulative `pnpm -r typecheck` and `pnpm build` passed. All 1,321
channel integration cases passed. The normal UI, CLI, database, adapter,
plugin, and serialized API test selections ran.
- The first Linux `pnpm test:run` returned 1. Original-source runs
reproduced the local failures. Container-only repairs supplied missing
tools, process reaping, and sufficient Node heap. The existing
mocked-fetch CLI telemetry fixture passed with the image opt-out flag
overridden only for that fixture.
- One local venue limitation remains: adapter-utils T24 fails unchanged
on both original and final source in the Linux container. Its entire
56-case suite passes before and after on the host. All other failed or
aborted suites passed in repaired reruns. No test assertion or timeout
changed.
- These fixtures prove application integration. They do not prove live
provider execution.

## Risks

- A captured option value or changed callback order could affect
deferred ingress or stale-runtime handling.
- The ingress callback and runtime getter remain live. Core maps and
receiver lease operations keep one owner.
- No schema migration or public API change is included.

## Model Used

OpenAI GPT-6, running in Codex with reasoning, tool use, and code
execution. The exact deployment identifier and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 10:50:34 -05:00

3553 lines
163 KiB
TypeScript

import { createChatSdkMessageReceiver } from "./chat-channels/runtime/sdk-messages.js";
import { admitDiscordSdkMessage } from "./chat-channels/discord/messages.js";
import { resolveInboundConversationBinding } from "./chat-channels/inbound/conversation-binding.js";
import { createGitHubMaintenance } from "./chat-channels/github/maintenance.js";
import { createSpekoChannelService, authorizeSpekoPublicVoiceTask, replaceSpekoChannelRuntime, SPEKO_CONVERSATION_IS_LINEAR } from "./chat-channels/speko/runtime.js";
import { prepareDiscordRuntimeCommand } from "./chat-channels/discord/runtime-commands.js";
import { createSlackSetupServices } from "./chat-channels/slack/setup-services.js";
import { isSupportedTelegramWebhookBaseUrl } from "./chat-channels/telegram/setup.js";
import { initializeGitHubWebhookRuntime } from "./chat-channels/github/webhook-runtime.js";
import { githubInboundTaskTitle, githubInboundCommentText, githubAssignedAgentRequiresLowTrust, retainGitHubDeliveryAuthority, retainGitHubReviewReceipt } from "./chat-channels/github/task-preparation.js";
import { applyChatTaskReviewContainment } from "./chat-channels/inbound/task-trust.js";
import { photonConversationBindingIsComplete, photonControlTargetsConversation, photonReplyMetadata, recordPhotonInboundActivity } from "./chat-channels/imessage-photon/messages.js";
import { deferTeamsSetupReply } from "./chat-channels/microsoft-teams/setup-replies.js";
import { slackOrphanFollowUp, stripSlackTitleMentions, slackMessageMeetsMentionRequirement } from "./chat-channels/slack/messages.js";
import { createChatPublicationResolution } from "./chat-channels/publications/resolution.js";
import { createChatPublicationRequests } from "./chat-channels/publications/requests.js";
import { createTeamsBoardSendPreparation } from "./chat-channels/microsoft-teams/board-send-preparation.js";
import { createInboundWakeAuthority, inboundWakeScope } from "./chat-channels/inbound/wakeup-authorization.js";
import { createInboundWakeups } from "./chat-channels/inbound/wakeups.js";
import { createInboundWakeNotices } from "./chat-channels/inbound/wakeup-notices.js";
import { createDiscordRuntimeReconciliation } from "./chat-channels/discord/runtime-reconciliation.js";
import { createPhotonRuntimeCallbacks, discardPhotonInboundMessages, photonRuntimeStatusMayAdvance, PHOTON_MESSAGE_INGRESS_ONLY } from "./chat-channels/imessage-photon/runtime-callbacks.js";
import { createChatReceiverOwnership } from "./chat-channels/runtime/ownership.js";
import { createGitHubWebhookScope, githubWebhookCanRecoverRevokedInstallation } from "./chat-channels/github/webhook-scope.js";
import { createGitHubWebhookAuthentication, retryableGitHubWebhookResponse } from "./chat-channels/github/webhook-authentication.js";
import { createSlackWebhookCallbacks } from "./chat-channels/slack/webhook-callbacks.js";
import { createGitHubWebhookEvents } from "./chat-channels/github/webhook-events.js";
import { createSlackTaskStartResolution } from "./chat-channels/slack/task-start-resolution.js";
import { createSlackBoardMessages } from "./chat-channels/slack/board-messages.js";
import { PROVIDER_LABELS, CAPABILITIES } from "./chat-channels/provider-catalog.js";
import { createMessageResourceAdmission } from "./chat-channels/inbound/resource-admission.js";
import type { DiscordGatewayOwnership } from "./chat-channels/runtime-types.js";
import { createInboundAdmission } from "./chat-channels/inbound/admission.js";
import { createInboundEnvelope } from "./chat-channels/inbound/envelope.js";
import { sameTeamsThreadIdentity, durableExternalThreadIdentity, canonicalProviderResourceId, nonDirectDestinationAllowed, linearControlCommand, actionThreadMatchesConversation } from "./chat-channels/inbound/surfaces.js";
import { createChatInboundAuthority } from "./chat-channels/inbound/authorization.js";
import { createTeamsRoutes } from "./chat-channels/microsoft-teams/routes.js";
import type { LiveInboundMessage } from "./chat-channels/runtime-types.js";
import { DELIVERY_PROCESSING_STALE_MS, normalizedDeliveryThreadId, normalizedLifecycleEffect, deliveryReady, pendingInboundWakeupCondition, lifecycleRuntimeFence, normalizedLifecycleWithRuntimeFence } from "./chat-channels/inbound/delivery-ledger.js";
import { createChatMessageLifecycle } from "./chat-channels/inbound/lifecycle.js";
import { createChatConversationDeliveryQueue } from "./chat-channels/inbound/conversation-queue.js";
import { createChatPublicationTaskControl } from "./chat-channels/publications/task-control.js";
import { createChatPublicationTransport } from "./chat-channels/publications/transport.js";
import { createChatPublicationAuthorization } from "./chat-channels/publications/authorization.js";
import { createChatPublicationFailures } from "./chat-channels/publications/failures.js";
import { createChatPublicationPreparation } from "./chat-channels/publications/preparation.js";
import { createChatPublicationWorker } from "./chat-channels/publications/worker.js";
import { createChatPublicationQueue } from "./chat-channels/publications/queue.js";
import { isExplicitOperatorPublication } from "./chat-channels/publications/policy.js";
import { createChatCredentials } from "./chat-channels/credentials.js";
import { createChatEndpoints } from "./chat-channels/endpoints.js";
import { createChatConfiguration } from "./chat-channels/configuration.js";
import { createPhotonSetupInspection } from "./chat-channels/imessage-photon/setup.js";
import { createChatSdkStatePersistence } from "./chat-channels/sdk-state.js";
export { createChatSdkStatePersistence } from "./chat-channels/sdk-state.js";
import { createChatInteractionCallbacks } from "./chat-channels/interactions/callbacks.js";
import { createChatInteractionContinuations } from "./chat-channels/interactions/continuations.js";
import { createGitHubInstallationLifecycle } from "./chat-channels/github/installation-lifecycle.js";
import { githubChatReviewService } from "./chat-github-reviews.js";
import { createGitHubAppSetup } from "./chat-channels/github/app-setup.js";
import { createSlackTaskAuthorityRegistration } from "./chat-channels/slack/task-authority.js";
import { createGitHubPublicationPolicy } from "./chat-channels/github/publication-policy.js";
import { createTelegramAttachmentFailure } from "./chat-channels/telegram/attachment-failure.js";
import { createSlackCommands } from "./chat-channels/slack/commands.js";
import { createTelegramCommands, telegramInboundControls, telegramGuidanceResponse, telegramForumControlResponse } from "./chat-channels/telegram/commands.js";
import { createTeamsFileDelivery } from "./chat-channels/microsoft-teams/file-delivery.js";
import { createGitHubSetupSecret } from "./chat-channels/github/setup-secret.js";
import { createSlackInventory } from "./chat-channels/slack/inventory.js";
import { REQUIRED_GITHUB_EVENTS, UNAVOIDABLE_GITHUB_EVENTS } from "./chat-channels/github/credentials.js";
import { createPhotonCredentialChecks } from "./chat-channels/imessage-photon/credentials.js";
import { createSetupProbes } from "./chat-channels/setup-probes.js";
import { createChatActivity } from "./chat-channels/activity.js";
import { createProviderEffects, type ProviderEffectTarget } from "./chat-channels/provider-effects.js";
import { createTelegramCallbackNoticeAuthority } from "./chat-channels/telegram/callback-notices.js";
import { createOutboundAttachments } from "./chat-channels/outbound-attachments.js";
export { hydrateOutboundAttachment } from "./chat-channels/outbound-attachments.js";
import { createInboundAttachments } from "./chat-channels/inbound-attachments.js";
import { createInboundReactions } from "./chat-channels/reactions.js";
import { createReceiptReactions } from "./chat-channels/receipt-reactions.js";
import { createDiscordRootThreads } from "./chat-channels/discord/root-threads.js";
import { createDiscordGatewayEvents } from "./chat-channels/discord/gateway-events.js";
import { createTeamsFileAuthority } from "./chat-channels/microsoft-teams/file-transfer-authority.js";
import { createPhotonEvents } from "./chat-channels/imessage-photon/events.js";
import { createTelegramPublicationDrafts } from "./chat-channels/telegram/publication-drafts.js";
import { createSlackFileReceipts } from "./chat-channels/slack/file-receipts.js";
import { createSlackSessionStatus } from "./chat-channels/slack/session-status.js";
import { createDiscordCommands } from "./chat-channels/discord/commands.js";
import { createSlackTaskStart } from "./chat-channels/slack/task-start.js";
import { createGitHubWebhookIngress } from "./chat-channels/github/webhook-ingress.js";
import { createSlackSessionStop } from "./chat-channels/slack/session-stop.js";
import { createTelegramMaintenance } from "./chat-channels/telegram/maintenance.js";
import type { EndpointRow, DbTransaction, DbOrTransaction, RuntimeContext, LifecycleRuntimeFence, InboundRuntimeContext, InternalSetupState } from "./chat-channels/runtime-types.js";
import { createChatFailedRunRetry } from "./chat-channels/failed-run-retry.js";
import { createChatPublicationReplacement } from "./chat-channels/publication-replacement.js";
import { createChatIdentityLinks } from "./chat-channels/identity-links.js";
import { createChatResourceReconciliation } from "./chat-channels/resource-reconciliation.js";
import { scheduleSlackLifecycleResourceHydration } from "./chat-channels/slack/resources.js";
import {
MAX_INBOUND_TEXT,
stableExternalPrincipalId,
} from "./chat-channels/message-lifecycle.js";
import { telegramLifecycleEventFromPayload, telegramIgnoresInboundMessage } from "./chat-channels/telegram/messages.js";
import {
teamsWebhookLifecycleEventFromPayload, teamsRootMentionCanEnableSetupDestination, teamsOrphanFollowUp, teamsAttachmentReferenceText,
} from "./chat-channels/microsoft-teams/messages.js";
import {
githubSupplementalLifecycleFromRequest,
} from "./chat-channels/github/messages.js";
import { nextChatActionAt } from "./chat-action-work.js";
import type { ScheduledChatAction } from "./chat-work-notifications.js";
import { notifyChatDeliveryWork, notifyChatPublicationWork } from "./chat-work-notifications.js";
import { voiceSessionStore } from "./voice/voice-session-store.js";
import { createChatRuntime, type ChatRuntime } from "./chat-runtime.js";
import type { ChatEndpointRuntime } from "./voice/voice-runtime.js";
import { fastResponseService, fastResponseSourceCurrent } from "./fast-responses.js";
import { fastResponseRequests } from "@paperclipai/db";
import { chatCredentialMutationLease } from "./chat-credential-mutation-lease.js";
import type { AgentAvatarRequest } from "./agent-avatars.js";
import { canPublishIssueToChatAudience } from "./authorization.js";
import { buildChatCommunicationGuidance } from "./chat-communication-guidance.js";
import { assertChatTaskCreationAccess, resolveChatExecutionDefaults } from "./chat-execution-defaults.js";
import { githubAutomaticAdmission } from "./chat-github-events.js";
import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared";
import { resumeExternalConversation } from "./slack-conversation-state.js";
import { runtimeCanonicalOrigin } from "./cloud-runtime-identity.js";
import { photonReplyReference } from "./photon/adapter.js";
import { createHash } from "node:crypto";
import {
and,
asc,
desc,
eq,
inArray,
isNull,
lte,
ne,
notExists,
notInArray,
or,
sql,
} from "drizzle-orm";
import {
recordChatWebhookReceipt,
recordChatWebhookStage,
} from "./chat-webhook-diagnostics.js";
import {
readChatControlChronology,
} from "./chat-control-chronology.js";
import type { Db } from "@paperclipai/db";
import { agentWakeupRequests, agents, chatActions, chatConversations, chatDeliveries, chatEndpointResources, chatEndpoints, chatExternalPrincipals, chatIdentityLinks, chatMessageLinks, chatPublications, chatTeamsFileTransfers, companyMemberships, heartbeatRuns, issueComments, issueThreadInteractions, issues, toolConnections } from "@paperclipai/db";
import type { ChatEventKind, ChatEndpointSetupState, ChatProvider, ExternalChannelBindingSummary, SafeChatPublicationPayload, ToolCredentialSecretRef } from "@paperclipai/shared";
import {
isUuidLike,
LOW_TRUST_REVIEW_PRESET,
} from "@paperclipai/shared";
import {
type TeamsFileTransferOptions,
} from "./chat-teams-file-transfers.js";
import { parseChatWebhookPublicBaseUrl } from "../chat-webhook-public-url.js";
import { conflict, forbidden, notFound, unprocessable } from "../errors.js";
import { logger } from "../middleware/logger.js";
import { redactSensitiveText } from "../redaction.js";
import type { StorageService } from "../storage/types.js";
import {
type ChatSdkCallbackEvent,
type ChatSdkMessageCallbackEvent,
type ChatSdkProvider,
type ChatSdkRuntime,
type DiscordGatewayCallbackEvent,
} from "./chat-sdk-runtime.js";
import {
logActivity,
publishActivity,
type ActivityPublication,
} from "./activity-log.js";
import { type IssueAssignmentWakeupDeps } from "./issue-assignment-wakeup.js";
import { issueService } from "./issues.js";
import {
NativeChatReviewPresentationContentionError,
} from "./native-runtime/native-chat-review-presentation.js";
import { isExternalChatWaitAuthorizationContention } from "./native-runtime/chat-attachment-reuse.js";
import { projectSafeChatPublication } from "./chat-publication-projection.js";
import {
githubAttachmentLimitOmissions,
} from "./chat-github-attachments.js";
import { getExternalChannelBindingSummary } from "./chat-channel-binding.js";
import {
parseChatProviderLifecycle,
type ChatProviderLifecycleEffect,
} from "./chat-provider-lifecycle.js";
import { parseInboundWakePublicationKey } from "./chat-inbound-wakeup-publications.js";
import {
questionResponseDeliveryService,
type QuestionResponseDeliveryServiceOptions,
} from "./question-response-delivery.js";
import { secretService } from "./secrets.js";
import type {
Message,
SlashCommandEvent,
Thread,
} from "chat";
const REQUIRED_CREDENTIALS: Record<
Exclude<ChatProvider, "github">,
readonly string[]
> = {
speko: ["apiKey", "signingSecret", "agentId"],
"imessage-photon": ["projectSecret"],
agentmail: [],
slack: ["botToken", "signingSecret"],
discord: ["botToken", "applicationId", "guildId"],
"microsoft-teams": ["clientId", "tenantId", "clientSecret"],
telegram: ["botToken"],
};
const SUPPORTED_GITHUB_WEBHOOK_EVENTS = new Set<string>([
"issues",
"pull_request",
"pull_request_review",
"ping",
...REQUIRED_GITHUB_EVENTS,
...UNAVOIDABLE_GITHUB_EVENTS,
]);
const SUPPLIED_CREDENTIAL_KEYS: Record<ChatProvider, readonly string[]> = {
speko: ["apiKey", "signingSecret", "agentId"],
"imessage-photon": ["projectSecret"],
agentmail: [],
slack: ["botToken", "signingSecret"],
github: ["appId", "privateKey"],
discord: ["botToken", "applicationId", "guildId"],
"microsoft-teams": ["clientId", "tenantId", "clientSecret"],
telegram: ["botToken"],
};
const MAX_ERROR_TEXT = 2_000;
// A Discord root mention is durably staged before the adapter creates its
// provider thread. Give the bounded provider retry loop ample time to finish;
// if the process disappears, the delivery worker verifies that thread over
// Discord's read-only API before it permits any Paperclip task mutation.
const DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS = 5 * 60_000;
const DELIVERY_LEASE_TTL_MS = 90_000;
const DELIVERY_DRAIN_LIMIT = 100;
const ORPHAN_FOLLOW_UP_GRACE_MS = 5_000;
const ORPHAN_FOLLOW_UP_MAX_ATTEMPTS = 12;
const PUBLICATION_ENDPOINT_CONCURRENCY = 4;
function leasedChatProvider(provider: string): boolean { return provider === "discord" || provider === "imessage-photon"; }
const DISCORD_GATEWAY_LEASE_POLL_MS = 100;
const PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS = 25_000;
const GITHUB_WEBHOOK_RESPONSE_BUDGET_MS = 8_000;
const CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS = 30_000;
// Providers can deliver adjacent callbacks on separate HTTP requests out of
// order. Hold the first callback briefly so a rapid burst can be sorted by the
// provider's own timestamp and sequence before any run is started.
const INGRESS_REORDER_WINDOW_MS: Partial<Record<ChatProvider, number>> = {
// Both providers deliver adjacent comments as independent HTTP requests and
// do not guarantee callback arrival order. A fixed, non-sliding window lets
// the durable drain sort a short burst before the first agent wake starts.
slack: 750,
github: 750,
discord: 750,
// Bot Framework can dispatch adjacent activities over independent requests.
// Keep Teams bursts inside the same provider-timestamp ordering boundary as
// Slack and GitHub so a faster later reply cannot wake the agent first.
"microsoft-teams": 750,
// Telegram's webhook max_connections setting permits concurrent delivery.
// Its message timestamp has only one-second resolution, so the drain also
// uses message_id/update_id below to order callbacks within this fixed,
// non-sliding window without serializing separate chats.
telegram: 750,
};
type ResourceRow = typeof chatEndpointResources.$inferSelect;
function runtimeGeneration(setup: ChatEndpointSetupState): number {
const value = Number((setup as InternalSetupState).runtimeGeneration ?? 0);
return Number.isSafeInteger(value) && value >= 0 ? value : 0;
}
function credentialFingerprint(refs: ToolCredentialSecretRef[]): string {
const stable = refs
.map((ref) => ({
configPath: ref.configPath,
secretId: ref.secretId,
versionSelector: ref.versionSelector ?? "latest",
}))
.sort((left, right) =>
`${left.configPath}:${left.secretId}:${left.versionSelector}`.localeCompare(
`${right.configPath}:${right.secretId}:${right.versionSelector}`,
),
);
return createHash("sha256").update(JSON.stringify(stable)).digest("hex");
}
type ConversationRow = typeof chatConversations.$inferSelect;
type DeliveryRow = typeof chatDeliveries.$inferSelect;
export interface ChatChannelServiceOptions {
/**
* Provider webhooks await only the durable ingress write, then finish task
* processing outside the provider response budget. Tests may leave this off
* when they need direct callback assertions.
*/
deferWebhookProcessing?: boolean;
/** Derived from instance deployment mode, never a provider request. */
allowLocalVoiceBoard?: boolean;
/** Test override for GitHub's end-to-end webhook response budget. */
githubWebhookResponseBudgetMs?: number;
/** Testable boundary after reading a GitHub webhook and before authentication. */
githubWebhookAuthenticationBarrier?: () => Promise<void>;
/** Testable boundary before a durable GitHub receipt re-enters provider handling. */
githubWebhookReplayBarrier?: () => Promise<void>;
/** Test override for the credential-mutation lease renewal cadence. */
credentialMutationLeaseRenewalIntervalMs?: number;
/** Narrow fault-injection boundary for credential-lease ownership checks. */
renewCredentialMutationLease?: (input: {
companyId: string;
endpointId: string;
expiresAt: Date;
leaseKey: string;
token: string;
}) => Promise<boolean>;
fetch?: typeof globalThis.fetch;
/** Test-only private upload transport; production retains guarded egress. */
teamsFileUploadRequest?: TeamsFileTransferOptions["uploadRequest"];
heartbeat: IssueAssignmentWakeupDeps & {
cancelRun?: (
runId: string,
reason?: string,
options?: {
errorCode?: string;
eventMessage?: string;
eventPayload?: Record<string, unknown>;
resultJson?: Record<string, unknown>;
},
) => Promise<unknown>;
};
/** Production bridge for resuming a native run that owns the question. */
resolveNativeQuestion?: QuestionResponseDeliveryServiceOptions["resolveNativeQuestion"];
publicBaseUrl?: string | null;
/** Canonical loopback browser return for Cloud-assisted GitHub registration. */
githubWizardOrigin?: string | null;
/** Optional verified ingress origin; never used for board or identity links. */
webhookPublicBaseUrl?: string | null;
runtime?: ChatSdkRuntime | ChatRuntime;
/** Testable scheduler hook; its callback settles after the tracked work finishes. */
scheduleDeferredWork?: (task: () => void | Promise<void>) => void;
/** Test boundary after selecting due Slack status work and before claiming. */
slackSessionSyncSelectionBarrier?: () => Promise<void>;
/** Narrow fault-injection boundary for the one-time setup-secret audit. */
setupSecretActivityLogger?: typeof logActivity;
/** Testable barrier after fail-closed state and before secret-ref mutation. */
setupSecretCredentialPersistBarrier?: () => Promise<void>;
/** Testable boundary immediately before setup-secret's final lease check. */
setupSecretFinalOwnershipBarrier?: () => Promise<void>;
/** Testable boundary after setup qualification and before activation. */
setupTestActivationBarrier?: () => Promise<void>;
/** Testable boundary after identity preflight and before its database claim. */
nativeBotIdentityClaimBarrier?: () => Promise<void>;
/** Testable crash boundary after interaction commit and before chat settlement. */
confirmationResolutionPersistBarrier?: () => Promise<void>;
/** Testable boundary after a question callback claim and before resolution. */
questionResolutionPersistBarrier?: () => Promise<void>;
/** Testable boundary before the final question-form opener authorization fence. */
questionFormOpenAuthorizationBarrier?: () => Promise<void>;
/** Testable boundary immediately before the final inbound reach row lock. */
reachAuthorizationBarrier?: () => Promise<void>;
/** Testable boundary after receipt-reaction durability and before transport. */
receiptReactionTransportBarrier?: () => Promise<void>;
/** Testable boundary between the first reaction-link and streaming lookups. */
reactionLinkPreflightBarrier?: () => Promise<void>;
/** Testable boundary after replay locks its endpoint and before conversation. */
reactionReplayEndpointLockBarrier?: () => Promise<void>;
/** Testable boundary after replay locks its conversation. */
reactionReplayConversationLockBarrier?: () => Promise<void>;
/** Narrow fault-injection boundary before durable publication preparation. */
publicationTransportPreparationBarrier?: (input: {
publicationId: string;
}) => Promise<void>;
/** Test override for the conversation-delivery lease renewal cadence. */
conversationLeaseRenewalIntervalMs?: number;
/** Narrow fault-injection boundary for renewing a conversation lease. */
renewConversationDeliveryLease?: (input: {
endpointId: string;
leaseKey: string;
token: string;
expiresAt: Date;
}) => Promise<boolean>;
/** Test override for Discord Gateway leader-lease expiry. */
discordGatewayLeaseTtlMs?: number;
/** Test override for Discord Gateway leader-lease renewal cadence. */
discordGatewayLeaseRenewalIntervalMs?: number;
/** Test override for lifecycle handoff while another replica is stopping. */
discordGatewayLeaseWaitMs?: number;
/** Narrow fault-injection boundary before renewing a Discord Gateway lease. */
renewDiscordGatewayLease?: (input: {
companyId: string;
credentialFingerprint: string;
endpointId: string;
expiresAt: Date;
generation: number;
leaseKey: string;
token: string;
}) => Promise<boolean>;
/** Testable barrier inside the serialized Discord Gateway event drain. */
discordGatewayEventBarrier?: (
event: DiscordGatewayCallbackEvent,
) => Promise<void>;
/** Testable boundary after initial Gateway ownership and before admission. */
discordGatewayMessageAdmissionBarrier?: () => Promise<void>;
/** Testable barrier before the credential-leased Discord root creation. */
discordRootThreadTransportBarrier?: () => Promise<void>;
/** Testable boundary after a Slack task-admission worker claims its attempt. */
slackTaskAdmissionClaimBarrier?: (input: {
actionId: string;
claimId: string;
}) => Promise<void>;
renderSlackAvatar?: (request: AgentAvatarRequest) => Promise<Buffer>;
slackManaged?: import("./connectors/slack/setup/registration.js").SlackSetupOptions["managed"];
storage?: StorageService;
}
function safeTitle(text: string, fallback: string): string {
const line = stripSlackTitleMentions(text)
.replace(/@[\w.-]+(?:\[bot\])?/gi, "")
.trim()
.split(/\r?\n/)[0];
return (line || fallback).slice(0, 160);
}
function sanitizeFilename(value: string | undefined): string | null {
if (!value) return null;
const leaf = value.replaceAll("\\", "/").split("/").pop()?.trim();
return leaf ? leaf.replace(/[\u0000-\u001f\u007f]/g, "").slice(0, 255) : null;
}
function redactError(error: unknown): string {
const text = error instanceof Error ? error.message : String(error);
// Telegram authenticates Bot API calls with a token embedded in the URL
// path. Generic key/value redaction cannot recognize that shape, so scrub
// both literal and URL-encoded forms before an error reaches logs, health
// state, or an HTTP response.
const withoutTelegramBotTokens = text
.replace(/(\/bot)\d{5,}(?::|%3A)[A-Za-z0-9_-]{20,}/gi, "$1***REDACTED***")
.replace(/\b\d{5,}:[A-Za-z0-9_-]{20,}\b/g, "***REDACTED***");
return redactSensitiveText(withoutTelegramBotTokens).slice(0, MAX_ERROR_TEXT);
}
function absoluteBaseUrl(value: string | null | undefined): string | null {
if (!value) return null;
try {
const url = new URL(value);
return url.protocol === "https:" ||
url.hostname === "localhost" ||
url.hostname === "127.0.0.1"
? url.origin
: null;
} catch {
return null;
}
}
export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
const runtime = options.runtime ?? createChatRuntime();
const githubManualMessages = new WeakMap<object, { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }>();
const githubAutomaticMessages = new WeakMap<object, { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }>();
const githubIssueMessages = new WeakMap<object, { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }>();
const runtimeVersions = new Map<string, string>();
const runtimeLocalEpochs = new Map<string, number>();
// One bounded publication lane per endpoint; credential/reconnect fencing
// still serializes a bot's sends while unrelated bots can make progress.
const publicationEndpointTasks = new Map<string, Promise<void>>();
// Scheduling-only cooldown for rejected/contended staged work. Do not alter
// an unverified transfer or its publication merely because a worker saw it.
const teamsFileRetryAfter = new Map<string, number>();
const runtimeContexts = new WeakMap<object, RuntimeContext>();
const runtimeInitializations = new Map<
string,
{
discordGatewayOwned: boolean;
promise: Promise<ChatEndpointRuntime>;
version: string;
}
>();
const { recordCurrentMicrosoftTeamsRoute } = createTeamsRoutes(db, { runtime, runtimeCallbackEndpoint });
const persistence = createChatSdkStatePersistence(db);
const fetchImpl = options.fetch ?? globalThis.fetch;
const { inspectPhotonCredentials, verifyPhotonCredentials } = createPhotonCredentialChecks(fetchImpl);
const { hydrateSlackResourceLabel } = createSlackInventory(db, { endpointRecord, resolveCredentials: (...args) => resolveCredentials(...args), fetchImpl });
const configuredPublicBaseUrl = absoluteBaseUrl(options.publicBaseUrl);
const configuredWebhookPublicBaseUrl = parseChatWebhookPublicBaseUrl(options.webhookPublicBaseUrl);
// A warm Cloud instance is constructed before it receives its signed claim.
// Resolve its live identity when producing URLs, not once at service startup.
// An explicit webhook ingress remains separate from board/identity links.
const getPublicBaseUrl = () => runtimeCanonicalOrigin() ?? configuredPublicBaseUrl;
const { listPrincipals, createLinkIntent, previewIdentityLink, requestIdentityAccess, confirmIdentityLink, revokeLink } = createChatIdentityLinks(db, {
endpointRecord, getPublicBaseUrl, stageProviderEffect: (...args) => stageProviderEffect(...args), scheduleProviderEffect,
confirmationRuntimeFor: async (endpoint) => runtimeContexts.get((await runtimeFor(endpoint)) as object),
});
// Task links must validate the original configured URL before normalization
// can remove credentials or other evidence that makes it unsafe to publish.
const getTaskBaseUrl = () => runtimeCanonicalOrigin() ?? options.publicBaseUrl;
const getWebhookPublicBaseUrl = () => configuredWebhookPublicBaseUrl ?? getPublicBaseUrl();
const voiceStore = voiceSessionStore(db, { allowLocalBoard: options.allowLocalVoiceBoard === true });
const { findAuthorizedIdentityLink, ensurePrincipal, sponsorAllowsGuest, lockCurrentPrincipalAuthorization, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, ensureResource } = createChatInboundAuthority(db, { voiceStore, runtimeCallbackEndpoint });
const voice = createSpekoChannelService(db, { endpointRecord, resolveCredentialRefs: (...args) => resolveCredentialRefs(...args), fetchImpl,
allowLocalBoard: options.allowLocalVoiceBoard === true,
onQuestionAnswered: (interactionId) => scheduleMessageProcessing(async () => {
await questionResponses.deliver(interactionId);
await processPendingPublications();
}),
});
const issuesSvc = issueService(db);
const { runIdFromMilestonePublication, runOwnershipMilestoneSupersessionReason, runPublicationToReplace, receiptReactionCompletionRunId, inboundWakePublicationToReplace, interactionResolutionPublicationToReplace, interactionPromptPublicationToReplace, progressSupersededByFastResponse, runProgressSupersededByPublishedInteraction, taskStatusPublicationToReplace, closeProgressToReplace } = createChatPublicationReplacement(db, { endpointRecord, authorizeRetainedChatSourcePublication });
const secrets = secretService(db);
const { normalizedCredentials, verifyCredentials, nativeBotIdentityMatches, legacyGitHubLabelsMatchVerifiedCredentials, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, persistCredentials, clearCredentials, resolveCredentialRefs, resolveCredentials, prepareProviderInventory } = createChatCredentials(db, {
secrets, PROVIDER_LABELS, REQUIRED_CREDENTIALS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS,
fetchImpl, redactError, credentialFingerprint, verifyPhotonCredentials,
});
const { inspectPhoton } = createPhotonSetupInspection(db, { endpointRecord, inspectPhotonCredentials });
const questionResponses = questionResponseDeliveryService(db, {
heartbeat:
options.heartbeat as QuestionResponseDeliveryServiceOptions["heartbeat"],
resolveNativeQuestion: options.resolveNativeQuestion,
});
const backgroundMessageTasks = new Set<Promise<void>>();
const scheduledConversationDrains = new Map<string, number>();
const liveInboundMessages = new Map<string, LiveInboundMessage>();
const discordGatewayOwnerships = new Map<string, DiscordGatewayOwnership>();
const discordGatewayStopTasks = new Set<Promise<void>>();
let shuttingDown = false;
const { discordGatewayNotOwnedError, isDiscordGatewayNotOwnedError, releaseDiscordGatewayLeaseRow, stopDiscordGatewayOwnership, renewDiscordGatewayOwnershipForMessageAdmission, ensureDiscordGatewayRuntimeIsCurrent, acquireDiscordGatewayOwnership, discordGatewayRuntimeIsCurrent } = createChatReceiverOwnership(db, { runtime, runtimeVersions, runtimeCallbackEndpoint, redactError, leasedChatProvider, discordGatewayOwnerships, discordGatewayStopTasks, isShuttingDown: () => shuttingDown, get discordGatewayLeaseTtlMs() { return options.discordGatewayLeaseTtlMs; }, get discordGatewayLeaseRenewalIntervalMs() { return options.discordGatewayLeaseRenewalIntervalMs; }, get discordGatewayLeaseWaitMs() { return options.discordGatewayLeaseWaitMs; }, get renewDiscordGatewayLease() { return options.renewDiscordGatewayLease; } });
const { receiveSdkMessages } = createChatSdkMessageReceiver({
endpointRecord, processMessage,
ingressOnly: (provider) => provider === "imessage-photon"
? PHOTON_MESSAGE_INGRESS_ONLY
: options.deferWebhookProcessing === true,
});
function localRuntimeEpoch(endpointId: string): number {
return runtimeLocalEpochs.get(endpointId) ?? 0;
}
function runtimeContextForRecord(
record: NonNullable<Awaited<ReturnType<typeof endpointRecord>>>,
): RuntimeContext {
const generation = runtimeGeneration(record.endpoint.setup);
const refsFingerprint = credentialFingerprint(record.credentialSecretRefs);
const localEpoch = localRuntimeEpoch(record.endpoint.id);
return {
credentialFingerprint: refsFingerprint,
generation,
localEpoch,
version: `${generation}:${refsFingerprint}:${localEpoch}`,
};
}
async function invalidateRuntime(endpointId: string): Promise<boolean> {
discardPhotonInboundMessages(liveInboundMessages, () => runtime.get(endpointId));
runtimeLocalEpochs.set(endpointId, localRuntimeEpoch(endpointId) + 1);
runtimeVersions.delete(endpointId);
const discordOwnership = discordGatewayOwnerships.get(endpointId);
if (discordOwnership) {
const hadRuntime = runtime.get(endpointId) !== null;
await stopDiscordGatewayOwnership(discordOwnership);
return hadRuntime;
}
return await runtime.removeEndpoint(endpointId);
}
function scheduleMessageProcessing(task: () => Promise<void>) {
if (shuttingDown) return;
const schedule = options.scheduleDeferredWork ?? setImmediate;
schedule(() => {
if (shuttingDown) return;
const pending = task().catch((error) => {
logger.error(
{ error: redactError(error) },
"deferred external chat message processing failed",
);
});
backgroundMessageTasks.add(pending);
void pending.finally(() => backgroundMessageTasks.delete(pending));
return pending;
});
}
function conversationDrainKey(endpointId: string, threadId: string) {
return `${endpointId}:${createHash("sha256").update(durableExternalThreadIdentity(threadId)).digest("hex")}`;
}
function scheduleConversationDrain(
endpointId: string,
threadId: string,
drainAt = Date.now(),
) {
if (shuttingDown) return;
const key = conversationDrainKey(endpointId, threadId);
const scheduledAt = scheduledConversationDrains.get(key);
if (scheduledAt !== undefined) return;
scheduledConversationDrains.set(key, drainAt);
scheduleMessageProcessing(async () => {
try {
while (!shuttingDown) {
const target = scheduledConversationDrains.get(key);
if (target === undefined) return;
const remaining = target - Date.now();
if (remaining > 0) {
await new Promise((resolve) => setTimeout(resolve, remaining));
continue;
}
const shouldContinue = await drainConversationDeliveries(
endpointId,
threadId,
);
if (shouldContinue) {
scheduledConversationDrains.set(key, Date.now());
continue;
}
return;
}
} finally {
scheduledConversationDrains.delete(key);
}
});
}
async function endpointRecord(endpointId: string) {
return db
.select({
endpoint: chatEndpoints,
assignedAgentName: agents.name,
connectionName: toolConnections.name,
applicationId: toolConnections.applicationId,
credentialSecretRefs: toolConnections.credentialSecretRefs,
})
.from(chatEndpoints)
.innerJoin(
agents,
and(
eq(agents.id, chatEndpoints.assignedAgentId),
eq(agents.companyId, chatEndpoints.companyId),
),
)
.innerJoin(
toolConnections,
and(
eq(toolConnections.id, chatEndpoints.connectionId),
eq(toolConnections.companyId, chatEndpoints.companyId),
),
)
.where(eq(chatEndpoints.id, endpointId))
.then((rows) => rows[0] ?? null);
}
async function stageAuthorizedTaskControlPublication(
tx: DbOrTransaction,
input: {
companyId: string;
conversationId: string;
endpointId: string;
idempotencyKey: string;
issueId: string;
payload: SafeChatPublicationPayload;
principalId: string;
},
) {
await notifyChatPublicationWork(tx);
const [inserted] = await tx
.insert(chatPublications)
.values({
companyId: input.companyId,
endpointId: input.endpointId,
conversationId: input.conversationId,
issueId: input.issueId,
idempotencyKey: input.idempotencyKey,
payload: input.payload,
state: "pending",
})
.onConflictDoNothing()
.returning({ id: chatPublications.id });
const publication =
inserted ??
(await tx
.select({ id: chatPublications.id })
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, input.companyId),
eq(chatPublications.idempotencyKey, input.idempotencyKey),
),
)
.then((rows) => rows[0] ?? null));
if (!publication) {
throw new Error("Task-control publication was not persisted");
}
await tx
.insert(chatActions)
.values({
companyId: input.companyId,
endpointId: input.endpointId,
conversationId: input.conversationId,
principalId: input.principalId,
kind: "task_control_authorization",
providerActionId: `task-control-authorization:${publication.id}`,
payload: { publicationId: publication.id },
status: "issued",
})
.onConflictDoNothing();
return publication;
}
function scheduleProviderEffect(
actionId: string,
liveTarget?: ProviderEffectTarget,
) {
scheduleMessageProcessing(async () => {
await processProviderEffect(actionId, liveTarget);
});
}
async function runtimeCallbackEndpoint(
tx: DbTransaction,
endpointId: string,
context: LifecycleRuntimeFence,
allowedStatuses: EndpointRow["status"][],
): Promise<EndpointRow | null> {
const endpoint = await tx
.select()
.from(chatEndpoints)
.where(eq(chatEndpoints.id, endpointId))
// Serialize endpoint mutations without taking the stronger row lock
// that conflicts with the Chat SDK state table's endpoint foreign-key
// check. FOR UPDATE can self-deadlock an SDK-state write performed by a
// provider callback on another pooled connection.
.for("no key update")
.then((rows) => rows[0] ?? null);
if (
!endpoint ||
!allowedStatuses.includes(endpoint.status) ||
runtimeGeneration(endpoint.setup) !== context.generation
)
return null;
const refs = await tx
.select({ refs: toolConnections.credentialSecretRefs })
.from(toolConnections)
.where(
and(
eq(toolConnections.companyId, endpoint.companyId),
eq(toolConnections.id, endpoint.connectionId),
),
)
.then((rows) => rows[0]?.refs ?? []);
if (credentialFingerprint(refs) !== context.credentialFingerprint) {
return null;
}
const runtimeContext = context as RuntimeContext;
if (
leasedChatProvider(endpoint.provider) &&
runtimeContext.discordGatewayOwned === true &&
runtimeContext.endpointRuntime !== undefined &&
!discordGatewayRuntimeIsCurrent(endpointId, runtimeContext)
) {
return null;
}
return endpoint;
}
async function runtimeCallbackRecord(
endpointId: string,
context: RuntimeContext,
allowedStatuses: EndpointRow["status"][],
) {
const record = await endpointRecord(endpointId);
if (
record && leasedChatProvider(record.endpoint.provider) &&
!(await ensureDiscordGatewayRuntimeIsCurrent(endpointId, context))
) {
return null;
}
return record &&
allowedStatuses.includes(record.endpoint.status) &&
runtimeGeneration(record.endpoint.setup) === context.generation &&
credentialFingerprint(record.credentialSecretRefs) ===
context.credentialFingerprint &&
runtime.get(endpointId) === context.endpointRuntime &&
(!leasedChatProvider(record.endpoint.provider) ||
discordGatewayRuntimeIsCurrent(endpointId, context))
? record
: null;
}
const withCredentialMutationLease = chatCredentialMutationLease(db, options);
const { list, get, create, update } = createChatEndpoints(db, {
endpointRecord, CAPABILITIES, PROVIDER_LABELS, withCredentialMutationLease,
getPublicBaseUrl, getWebhookPublicBaseUrl,
get slackRegistration() { return slackRegistration; },
});
const { generateSetupSecret } = createGitHubSetupSecret(db, {
endpointRecord, withCredentialMutationLease, resolveCredentials, persistCredentials,
runtimeGeneration, credentialFingerprint, invalidateRuntime, redactError,
get setupSecretActivityLogger() { return options.setupSecretActivityLogger; },
get setupSecretCredentialPersistBarrier() { return options.setupSecretCredentialPersistBarrier; },
get setupSecretFinalOwnershipBarrier() { return options.setupSecretFinalOwnershipBarrier; },
});
const { setupTestStatus, test } = createSetupProbes(db, { endpointRecord, withCredentialMutationLease, fetchImpl, findAuthorizedIdentityLink, runtimeGeneration, lockCurrentPrincipalAuthorization, get, setupTestActivationBarrier: () => options.setupTestActivationBarrier?.() ?? Promise.resolve() });
const { listConversations, listActivity, listActivityPage } = createChatActivity(db, { endpointRecord, teamsTransferProtocol: (...args) => teamsTransferProtocol(...args) });
const { lockTelegramCallbackNoticeAuthority } = createTelegramCallbackNoticeAuthority({ nonDirectDestinationAllowed, lockCurrentPrincipalAuthorization });
const { stageProviderEffect, processProviderEffect, processPendingProviderEffects } = createProviderEffects(db, { endpointRecord, runtimeCallbackEndpoint, withCredentialMutationLease, runtimeFor, invalidateRuntime, lockCurrentPrincipalAuthorization, redactError, MAX_ERROR_TEXT, nonDirectDestinationAllowed, lockTelegramCallbackNoticeAuthority });
const { publicationFiles } = createOutboundAttachments(db, { get storage() { return options.storage; } });
const { authorizeInboundWakeup, authorizeInboundWorkStart, authorizeFastResponse } = createInboundWakeAuthority(db, { runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization });
const { ingestAttachments, attachmentOmissionDetail } = createInboundAttachments(db, { authorizeInboundWakeup, lifecycleRuntimeFence, sanitizeFilename, issuesSvc, isExternalActionAuthorizationChange, redactError, get storage() { return options.storage; } });
const { stageInboundWakeup, acceptInboundWakeup, settleRejectedInboundWakeups, processInboundWakeup } = createInboundWakeups(db, { authorizeInboundWakeup, authorizeInboundWorkStart, attachmentOmissionDetail, isExternalActionAuthorizationChange, redactError, fetchImpl, get heartbeat() { return options.heartbeat; } });
const { settleTelegramAttachmentOnlyFailure } = createTelegramAttachmentFailure(db, {
attachmentOmissionDetail, runtimeContextForRecord, endpointRecord, stageProviderEffect, scheduleProviderEffect,
});
const { handleReaction, processPendingReactionDeliveries } = createInboundReactions(db, { lockCurrentPrincipalAuthorization, runtimeCallbackRecord, runtimeCallbackEndpoint, ensurePrincipal, durableExternalThreadIdentity, recordCurrentMicrosoftTeamsRoute, lifecycleRuntimeFence, deliveryReady, redactError, nonDirectDestinationAllowed, reactionLinkPreflightBarrier: () => options.reactionLinkPreflightBarrier?.() ?? Promise.resolve(), reactionReplayEndpointLockBarrier: () => options.reactionReplayEndpointLockBarrier?.() ?? Promise.resolve(), reactionReplayConversationLockBarrier: () => options.reactionReplayConversationLockBarrier?.() ?? Promise.resolve() });
const { addReceiptReaction, processReceiptReaction, processPendingReceiptReactions } = createReceiptReactions(db, { endpointRecord, runtimeContextForRecord, withCredentialMutationLease, runtimeFor, fetchImpl, runtime, runtimeGeneration, credentialFingerprint, redactError, MAX_ERROR_TEXT, receiptReactionTransportBarrier: () => options.receiptReactionTransportBarrier?.() ?? Promise.resolve() });
const { admitDiscordRootMention, discordRootThreadPending, reconcileDiscordRootThread, promoteDiscordRootReceipt } = createDiscordRootThreads(db, { ensureDiscordGatewayRuntimeIsCurrent, withCredentialMutationLease, runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization, processMessage, redactError, scheduleConversationDrain, lifecycleRuntimeFence, runtimeFor, DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS, discordRootThreadTransportBarrier: () => options.discordRootThreadTransportBarrier?.() ?? Promise.resolve() });
const { handleDiscordGatewayEvent } = createDiscordGatewayEvents(db, { ensureDiscordGatewayRuntimeIsCurrent, discordGatewayRuntimeIsCurrent, runtimeCallbackRecord, runtimeCallbackEndpoint, applyProviderLifecycleEffect, runtimeGeneration, discordGatewayEventBarrier: (event) => options.discordGatewayEventBarrier?.(event) ?? Promise.resolve() });
const { handlePhotonFailure, handlePhotonEvent } = createPhotonEvents(db, { runtimeCallbackRecord, runtimeCallbackEndpoint, invalidateRuntime, ensurePrincipal, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, scheduleMessageProcessing, processPendingPublications, questionResponses, redactError, renewDiscordGatewayOwnershipForMessageAdmission, handleSdkMessage });
const { handleTelegramGenerationStopped, reserveTelegramPublicationDraft } = createTelegramPublicationDrafts(db, { runtimeCallbackRecord, runtimeCallbackEndpoint, runtimeFor, runtime, runtimeContexts, credentialFingerprint, authorizeRetainedChatSourcePublication });
const { recordSlackFileUploadReceipt, stageSlackFilePublicationSuccess, processPendingSlackFileUploadReceipts, nextSlackReceiptAt } = createSlackFileReceipts(db, { endpointRecord, withCredentialMutationLease, runtimeCallbackEndpoint, nonDirectDestinationAllowed, runtimeFor, isExplicitOperatorPublication, recordExactOutboundPublicationLink });
const { processPendingSlackSessionSyncs } = createSlackSessionStatus(db, { endpointRecord, withCredentialMutationLease, runtimeCallbackEndpoint, nonDirectDestinationAllowed, runIdFromMilestonePublication, resolveCredentials, fetchImpl, redactError, get slackSessionSyncSelectionBarrier() { return options.slackSessionSyncSelectionBarrier; } });
const { reconcileDiscordCommands, handleDiscordNativeCommand } = createDiscordCommands(db, { endpointRecord, runtimeContextForRecord, runtimeGeneration, credentialFingerprint, resolveCredentialRefs, fetchImpl, withCredentialMutationLease, runtimeCallbackRecord, ensurePrincipal, runtimeCallbackEndpoint, nonDirectDestinationAllowed, lockCurrentPrincipalAuthorization, getDiscordGatewayOwnership: (endpointId) => discordGatewayOwnerships.get(endpointId), discordGatewayRuntimeIsCurrent, stageAuthorizedTaskControlPublication, scheduleMessageProcessing, processPendingPublications });
const { latestSlackDmControlThreadId, finalizeSlackTaskStartFailure, admitConfirmedSlackTaskStart, processSlackTaskStart, processPendingSlackTaskStarts } = createSlackTaskStart(db, { endpointRecord, runtimeGeneration, credentialFingerprint, invalidateRuntime, runtimeFor, processMessage, runtimeContextForRecord, redactError, withCredentialMutationLease, sponsorAllowsGuest, nonDirectDestinationAllowed, runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization, get slackTaskAdmissionClaimBarrier() { return options.slackTaskAdmissionClaimBarrier; }, maxErrorText: MAX_ERROR_TEXT });
const { resolveSlackTaskStart } = createSlackTaskStartResolution(db, { endpointRecord, withCredentialMutationLease, runtimeFor, runtimeContextForRecord, runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization, sponsorAllowsGuest, nonDirectDestinationAllowed, finalizeSlackTaskStartFailure, admitConfirmedSlackTaskStart, get deferWebhookProcessing() { return options.deferWebhookProcessing; } });
const { processPendingSlackBoardMessages } = createSlackBoardMessages(db, { endpointRecord, resolveCredentials, fetchImpl, get heartbeat() { return options.heartbeat; } });
const { handleSlackSlashCommand } = createSlackCommands(db, {
runtime, runtimeFor, handleSdkMessage, stageProviderEffect, scheduleProviderEffect, ensurePrincipal,
runtimeCallbackEndpoint, getPublicBaseUrl, canonicalProviderResourceId, nonDirectDestinationAllowed,
sponsorAllowsGuest, latestSlackDmControlThreadId, scheduleMessageProcessing, processSlackTaskStart,
get deferWebhookProcessing() { return options.deferWebhookProcessing; },
});
const { handleTelegramSlashCommand } = createTelegramCommands({ runtime, runtimeFor, handleSdkMessage });
const { stageGitHubWebhookIngress, githubIngressAcceptedResponse, processGitHubWebhookIngress, processFailedGitHubWebhookDeliveries, processPendingGitHubWebhookIngress } = createGitHubWebhookIngress(db, { endpointRecord, runtimeContextForRecord, resolveCredentialRefs, runtimeGeneration, credentialFingerprint, getWebhookPublicBaseUrl, redactError, handleWebhook, runtimeCallbackEndpoint, fetchImpl, persistence, withCredentialMutationLease, isShuttingDown: () => shuttingDown, get githubWebhookReplayBarrier() { return options.githubWebhookReplayBarrier; }, supportedWebhookEvents: SUPPORTED_GITHUB_WEBHOOK_EVENTS });
const { stageSlackSessionStop, processSlackSessionStop, processPendingSlackSessionStops, handleSlackSessionStopPayload } = createSlackSessionStop(db, { scheduleMessageProcessing, get deferWebhookProcessing() { return options.deferWebhookProcessing; }, runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed, stageAuthorizedTaskControlPublication, getPublicBaseUrl, get heartbeat() { return options.heartbeat; }, redactError });
const { stageTelegramMaintenance, processTelegramMaintenance, processPendingTelegramMaintenance, processPendingTelegramActions, processTelegramSubscriptionDiscovery, scheduleTelegramSubscriptionDiscovery, hasPendingTelegramSubscriptionDiscovery, hasTelegramStopSubscription, recordTelegramStopSubscription } = createTelegramMaintenance(db, { endpointRecord, runtimeContextForRecord, withCredentialMutationLease, resolveCredentials, clearCredentials, runtimeCallbackEndpoint, getWebhookPublicBaseUrl, fetchImpl, runtimeGeneration, credentialFingerprint, redactError, credentialCheckTimeoutMs: PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS });
const { reconcileGitHubWebhookRepository, reconcileProviderResourceRows, listResources, listGitHubRepositories, replaceResources, toggleAllGitHubRepositories, hasNewerProcessedProviderLifecycleEffect, applyProviderAvailability } = createChatResourceReconciliation(db, { endpointRecord, withCredentialMutationLease, runtimeGeneration, maxErrorText: MAX_ERROR_TEXT });
const { applyGitHubInstallationLifecycle } = createGitHubInstallationLifecycle(db, {
endpointRecord, resolveCredentials, runtimeCallbackEndpoint, runtimeGeneration, credentialFingerprint,
verifyCredentials, legacyGitHubLabelsMatchVerifiedCredentials, nativeBotIdentityMatches, assertNativeBotIdentityAvailable,
prepareProviderInventory, persistCredentials, reconcileProviderResourceRows, redactError,
normalizedLifecycleWithRuntimeFence, MAX_ERROR_TEXT,
});
const { handleGitHubWebhookEvent } = createGitHubWebhookEvents(db, { processMessage, recordLifecycleDelivery: (...args) => recordLifecycleDelivery(...args), githubAutomaticMessages, githubIssueMessages, fetchImpl, runtimeCallbackEndpoint });
const { authenticateGitHubWebhook } = createGitHubWebhookAuthentication(db, { endpointRecord, runtimeContextForRecord, withCredentialMutationLease, resolveCredentials, reconcileGitHubWebhookRepository, stageGitHubWebhookIngress, processGitHubWebhookIngress, githubIngressAcceptedResponse, scheduleMessageProcessing, backgroundMessageTasks, redactError, get githubWebhookAuthenticationBarrier() { return options.githubWebhookAuthenticationBarrier; }, get deferWebhookProcessing() { return options.deferWebhookProcessing; } });
const { inspectSlackWebhookScope, recordSlackWebhookCallback } = createSlackWebhookCallbacks(db, { resolveCredentials, runtimeCallbackEndpoint, getWebhookPublicBaseUrl, scheduleMessageProcessing, slackRegistration: { notifyVerified: (endpointId) => slackRegistration.notifyVerified(endpointId) } });
const { reconcileDiscordRuntime } = createDiscordRuntimeReconciliation(db, { reconcileDiscordCommands, endpointRecord, runtimeContexts, invalidateRuntime, runtimeFor, runtime, runtimeCallbackEndpoint, credentialFingerprint, discordGatewayRuntimeIsCurrent });
async function runtimeFor(
endpoint: EndpointRow,
optionsForRuntime: {
requireDiscordOwnership?: boolean;
waitForDiscordOwnership?: boolean;
} = {},
): Promise<ChatEndpointRuntime> {
for (;;) {
const record = await endpointRecord(endpoint.id);
if (!record) throw notFound("Chat endpoint not found");
if (
record.endpoint.status === "archived" ||
record.endpoint.status === "paused" ||
record.endpoint.status === "draft"
) {
throw conflict("Chat endpoint runtime is not available in this state", {
code: "chat_endpoint_runtime_unavailable",
});
}
const context = runtimeContextForRecord(record);
const current = runtime.get(endpoint.id);
const currentDiscordOwnership = discordGatewayOwnerships.get(endpoint.id);
const currentOwnsDiscordGateway =
currentDiscordOwnership !== undefined &&
!currentDiscordOwnership.stopping &&
currentDiscordOwnership.context.version === context.version &&
currentDiscordOwnership.context.endpointRuntime === current &&
currentDiscordOwnership.expiresAt.getTime() > Date.now();
if (
current &&
runtimeVersions.get(endpoint.id) === context.version &&
(!leasedChatProvider(record.endpoint.provider) ||
optionsForRuntime.requireDiscordOwnership !== true ||
currentOwnsDiscordGateway)
)
return current;
const pending = runtimeInitializations.get(endpoint.id);
if (pending) {
try {
const instance = await pending.promise;
if (
pending.version === context.version &&
(optionsForRuntime.requireDiscordOwnership !== true ||
pending.discordGatewayOwned)
) {
return instance;
}
if (
pending.version === context.version &&
optionsForRuntime.requireDiscordOwnership === true
) {
await new Promise((resolve) =>
setTimeout(resolve, DISCORD_GATEWAY_LEASE_POLL_MS),
);
continue;
}
} catch (error) {
if (pending.version === context.version) {
if (
optionsForRuntime.waitForDiscordOwnership === true &&
isDiscordGatewayNotOwnedError(error)
) {
await new Promise((resolve) =>
setTimeout(resolve, DISCORD_GATEWAY_LEASE_POLL_MS),
);
continue;
}
throw error;
}
}
continue;
}
const initialization: {
discordGatewayOwned: boolean;
promise: Promise<ChatEndpointRuntime>;
version: string;
} = {
discordGatewayOwned: false,
promise: undefined as unknown as Promise<ChatEndpointRuntime>,
version: context.version,
};
const promise = (async () => {
const discordOwnership =
leasedChatProvider(record.endpoint.provider)
? await acquireDiscordGatewayOwnership(
record.endpoint,
context,
optionsForRuntime.waitForDiscordOwnership === true,
)
: null;
if (
leasedChatProvider(record.endpoint.provider) &&
!discordOwnership &&
optionsForRuntime.requireDiscordOwnership === true
) {
throw discordGatewayNotOwnedError();
}
initialization.discordGatewayOwned = discordOwnership !== null;
context.discordGatewayOwned = discordOwnership !== null;
let instance: ChatEndpointRuntime | null = null;
try {
const stale = runtime.get(endpoint.id);
if (stale) {
runtimeVersions.delete(endpoint.id);
await runtime.removeEndpoint(endpoint.id);
}
recordChatWebhookStage("runtime_initializing", record.endpoint.id);
const credentials = await resolveCredentials(record.endpoint);
await prepareDiscordRuntimeCommand(db, record.endpoint, context);
instance = record.endpoint.provider === "speko"
? await replaceSpekoChannelRuntime(runtime, record, persistence)
: await runtime.replaceEndpoint({
companyId: record.endpoint.companyId,
endpointId: record.endpoint.id,
providerConfig: runtimeConfig(
record.endpoint,
record.endpoint.botUsername ?? record.assignedAgentName,
credentials,
),
persistence,
concurrency: record.endpoint.concurrencyPolicy,
enableDiscordGateway: discordOwnership !== null,
callbacks: {
onMessage: (event) => handleSdkMessage(event, context),
...createPhotonRuntimeCallbacks(db, { endpointId: endpoint.id, connectionId: endpoint.connectionId, companyId: record.endpoint.companyId, assignedAgentId: record.endpoint.assignedAgentId, context, lineId: credentials.lineId, runtimeCallbackRecord, runtimeCallbackEndpoint, renewDiscordGatewayOwnershipForMessageAdmission }),
onPhotonEvent: (event) => handlePhotonEvent(endpoint.id, event, context),
onPhotonFailure: (error) => handlePhotonFailure(endpoint.id, error, context),
onDiscordRootMentionAdmission:
record.endpoint.provider === "discord"
? (event) =>
admitDiscordRootMention(record.endpoint, event, context)
: undefined,
onDiscordGatewayEvent:
record.endpoint.provider === "discord"
? (event) => handleDiscordGatewayEvent(event, context)
: undefined,
onAction:
record.endpoint.capabilities.actions === true
? (event) => handleAction(event, context)
: undefined,
onModalSubmit:
record.endpoint.capabilities.modals === true ||
record.endpoint.provider === "discord"
? (event) => handleModalSubmit(event, context)
: undefined,
onMessageDeleted: (event) => handleMessageDeleted(event, context),
onMessageUpdated: (event) => handleMessageUpdated(event, context),
onTeamsFileConsent:
record.endpoint.provider === "microsoft-teams"
? (event) => handleTeamsFileConsent(event, context)
: undefined,
onTelegramGenerationStopped:
record.endpoint.provider === "telegram"
? (event) => handleTelegramGenerationStopped(event, context)
: undefined,
onReaction:
record.endpoint.capabilities.reactions === true
? (event) => handleReaction(event, context)
: undefined,
// Dynamic options remain unregistered. Native question buttons and
// forms resolve only through issued durable rows. Reactions are an
// auditable social signal, never an approval or task instruction.
// Telegram exposes bot commands through Chat SDK's slash-command
// callback even though it does not support arbitrary registered slash
// commands. Paperclip still needs that callback for /new, /close, and
// /status session controls.
// Discord must always install the guarded private-response path:
// an unavailable registration still needs a terminal denial for a
// cached provider command, not the SDK's unhandled deferred reply.
onSlashCommand:
record.endpoint.provider === "discord" ||
record.endpoint.capabilities.slashCommands ||
record.endpoint.provider === "telegram"
? (event) => handleSlashCommand(event, context)
: undefined,
},
});
context.endpointRuntime = instance;
runtimeContexts.set(instance as object, context);
await instance.initialize();
const latest = await endpointRecord(endpoint.id);
const stillRegistered = runtime.get(endpoint.id) === instance;
const stillCurrent =
latest !== null &&
(latest.endpoint.status === record.endpoint.status || record.endpoint.provider === "imessage-photon" && photonRuntimeStatusMayAdvance(record.endpoint, latest.endpoint)) &&
runtimeContextForRecord(latest).version === context.version;
if (!stillRegistered || !stillCurrent) {
if (stillRegistered) {
await runtime.removeEndpoint(endpoint.id);
runtimeVersions.delete(endpoint.id);
}
throw conflict(
"Chat endpoint runtime changed while it was initializing",
{
code: "chat_endpoint_runtime_superseded",
},
);
}
if (
discordOwnership &&
(discordGatewayOwnerships.get(endpoint.id) !== discordOwnership ||
discordOwnership.stopping ||
discordOwnership.expiresAt.getTime() <= Date.now())
) {
throw discordGatewayNotOwnedError();
}
runtimeVersions.set(endpoint.id, context.version);
return instance;
} catch (error) {
if (instance && runtime.get(endpoint.id) === instance) {
await runtime.removeEndpoint(endpoint.id).catch(() => undefined);
runtimeVersions.delete(endpoint.id);
} else if (!runtime.get(endpoint.id)) {
runtimeVersions.delete(endpoint.id);
}
if (discordOwnership) {
await stopDiscordGatewayOwnership(discordOwnership, false);
}
throw error;
}
})();
initialization.promise = promise;
runtimeInitializations.set(endpoint.id, initialization);
try {
return await promise;
} finally {
if (runtimeInitializations.get(endpoint.id) === initialization) {
runtimeInitializations.delete(endpoint.id);
}
}
}
}
async function reconcileProviderRuntimes() {
if (shuttingDown) {
return { eligible: 0, local: 0, ownedElsewhere: 0, failed: 0 };
}
const eligibleRows = await db
.select({ endpointId: chatEndpoints.id })
.from(chatEndpoints)
.innerJoin(
toolConnections,
and(
eq(toolConnections.id, chatEndpoints.connectionId),
eq(toolConnections.companyId, chatEndpoints.companyId),
),
)
.where(
and(
inArray(chatEndpoints.provider, ["discord", "imessage-photon"]),
inArray(chatEndpoints.status, ["verifying", "active", "attention"]),
eq(toolConnections.status, "active"),
eq(toolConnections.enabled, true),
),
);
const eligibleIds = new Set(eligibleRows.map((row) => row.endpointId));
await Promise.all(
[...discordGatewayOwnerships.values()]
.filter((ownership) => !eligibleIds.has(ownership.endpointId))
.map((ownership) => stopDiscordGatewayOwnership(ownership)),
);
let local = 0;
let ownedElsewhere = 0;
let failed = 0;
for (const row of eligibleRows) {
const record = await endpointRecord(row.endpointId);
if (!record) continue;
try {
let instance = await runtimeFor(record.endpoint, {
requireDiscordOwnership: true,
});
await reconcileDiscordRuntime(row.endpointId, instance);
local += 1;
} catch (error) {
if (isDiscordGatewayNotOwnedError(error)) {
ownedElsewhere += 1;
continue;
}
failed += 1;
logger.warn(
{ endpointId: row.endpointId, error: redactError(error) },
"could not reconcile Discord Gateway runtime",
);
}
}
return {
eligible: eligibleRows.length,
local,
ownedElsewhere,
failed,
};
}
const {
retryCommentIds,
assertFailedRetryReceipt,
FailedChatRetryPublicationReadError,
failedChatRetrySource,
unregisterCommittedResponseAuthority,
prepareFailedChatRunRetry,
authorizeFailedChatRunRetry,
unregisterFailedRetryAuthority,
processFailedChatRunRetry,
processFailedChatRunRetries,
enqueueFailedChatRetryPublications,
pendingFailedRetries,
} = createChatFailedRunRetry(db, {
authorizeInboundWakeup,
isExternalActionAuthorizationChange,
inboundWakeScope,
lifecycleRuntimeFence,
isShuttingDown: () => shuttingDown,
wakeup: (...args) => options.heartbeat.wakeup(...args),
});
const { githubCurrentOmissionPreparedText, settleGitHubAutomaticPublication } = createGitHubPublicationPolicy(db, {
isExplicitOperatorPublication, getTaskBaseUrl, failedChatRetrySource, isExternalActionAuthorizationChange,
FailedChatRetryPublicationReadError, endpointRecord, runIdFromMilestonePublication, fetchImpl,
stageTerminalReceiptReactionRemovals, runtimeContextForRecord, scheduleMessageProcessing, processReceiptReaction,
});
const { stageTeamsBoardFileIntent, deriveTeamsFileTransferAuthority } = createTeamsFileAuthority({ runtimeGeneration, credentialFingerprint, runtime, runtimeContexts, localRuntimeEpoch, runtimeVersions, lockCurrentPrincipalAuthorization, lifecycleRuntimeFence, assertFailedRetryReceipt, inboundWakeScope, failedChatRetrySource, isExplicitOperatorPublication, retryCommentIds });
const { teamsTransferProtocol, handleTeamsFileConsent, tryProcessTeamsFilePublication, scheduleTeamsFileMaintenance } = createTeamsFileDelivery(db, {
endpointRecord, runtimeContextForRecord, runtimeFor, runtimeContexts, localRuntimeEpoch, runtime,
deriveTeamsFileTransferAuthority, publicationFiles, scheduleMessageProcessing, processPendingPublications,
withCredentialMutationLease, teamsFileRetryAfter, isShuttingDown: () => shuttingDown,
get teamsFileUploadRequest() { return options.teamsFileUploadRequest; },
});
const { enqueueInboundWakeupPublications, authorizeInboundWakePublication } = createInboundWakeNotices(db, { authorizeInboundWakeup, enqueueFailedChatRetryPublications, authorizeRetainedChatSourcePublication, isExternalActionAuthorizationChange, isShuttingDown: () => shuttingDown });
async function authorizeRetainedChatSourcePublication(
tx: DbOrTransaction,
publication: typeof chatPublications.$inferSelect,
): Promise<boolean> {
if (!(await canPublishIssueToChatAudience(tx, publication))) return false;
if (publication.idempotencyKey.startsWith("fast-response:")) {
const [request] = await tx.select().from(fastResponseRequests).where(and(eq(fastResponseRequests.companyId, publication.companyId), eq(fastResponseRequests.id, publication.idempotencyKey.slice("fast-response:".length))));
if (!request || request.commentId !== publication.commentId || request.endpointId !== publication.endpointId || request.conversationId !== publication.conversationId || !(await fastResponseSourceCurrent(tx as unknown as Db, request))) return false;
try { await fastResponseService(tx as unknown as Db).authorize(tx as unknown as Db, request); await authorizeFastResponse(tx as unknown as Db, request); return true; } catch { return false; }
}
const notice = parseInboundWakePublicationKey(publication.idempotencyKey);
let runId = runIdFromMilestonePublication(publication);
if (!runId && publication.commentId && !notice) {
const [comment] = await tx
.select({ runId: issueComments.createdByRunId })
.from(issueComments)
.where(
and(
eq(issueComments.id, publication.commentId),
eq(issueComments.companyId, publication.companyId),
),
);
runId = comment?.runId ?? null;
}
if (!runId && publication.payload.interactionId && !notice) {
const [interaction] = await tx
.select({ runId: issueThreadInteractions.sourceRunId })
.from(issueThreadInteractions)
.where(
and(
eq(
issueThreadInteractions.id,
String(publication.payload.interactionId),
),
eq(issueThreadInteractions.companyId, publication.companyId),
),
);
runId = interaction?.runId ?? null;
}
const [run] = runId
? await tx
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, runId),
eq(heartbeatRuns.companyId, publication.companyId),
),
)
: [];
try {
const committed = run?.resultJson?.nativeCommittedChatResponse;
if (committed !== undefined) {
if (
!run ||
!committed ||
typeof committed !== "object" ||
Array.isArray(committed)
)
return false;
const marker = committed as Record<string, unknown>;
if (
marker.schema !== "paperclip.native_committed_chat_response.v1" ||
typeof marker.resultId !== "string" ||
!isUuidLike(marker.resultId) ||
typeof marker.canonicalSha256 !== "string" ||
typeof marker.decisionId !== "string" ||
!isUuidLike(marker.decisionId)
)
return false;
const source = await failedChatRetrySource(tx, {
companyId: publication.companyId,
issueId: publication.issueId,
agentId: run.agentId,
failedRunId: run.id,
publication: true,
committedResponse: {
resultId: marker.resultId,
canonicalSha256: marker.canonicalSha256,
decisionId: marker.decisionId,
},
});
if (
source.endpointId !== publication.endpointId ||
source.conversationId !== publication.conversationId
)
return false;
}
const actionId = notice?.wakeId ?? run?.wakeupRequestId;
if (!actionId) return true;
const [action] = await tx
.select()
.from(chatActions)
.where(
and(
eq(chatActions.id, actionId),
eq(chatActions.companyId, publication.companyId),
),
);
if (action?.kind !== "failed_run_retry")
return !run?.contextSnapshot?.chatFailedRunRetry;
const source = await authorizeFailedChatRunRetry(tx, {
companyId: publication.companyId,
issueId: publication.issueId,
agentId: String(action.payload.agentId),
retryActionId: action.id,
...(run ? { runId: run.id } : {}),
publication: true,
});
if (
source.endpointId !== publication.endpointId ||
source.conversationId !== publication.conversationId
)
return false;
if (notice) {
const [receipt] = await tx
.select()
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, action.id))
.for("share", { noWait: true });
return (
publication.commentId === source.commentIds.at(-1) &&
!receipt?.runId &&
(notice.state === "queued"
? receipt?.status === "deferred_issue_execution"
: notice.state === "not_started" &&
["cancelled", "failed", "skipped"].includes(
receipt?.status ?? "",
))
);
}
return (
!!run &&
JSON.stringify(retryCommentIds(run.contextSnapshot ?? {})) ===
JSON.stringify(source.commentIds) &&
run.contextSnapshot?.source === `chat:${source.provider}`
);
} catch (error) {
if (
isExternalChatWaitAuthorizationContention(error) ||
error instanceof NativeChatReviewPresentationContentionError
)
throw new NativeChatReviewPresentationContentionError();
if (
isExternalActionAuthorizationChange(error) ||
(error &&
typeof error === "object" &&
"details" in error &&
error.details &&
typeof error.details === "object" &&
"code" in error.details &&
error.details.code === "chat_failed_run_retry_not_authorized")
)
return false;
throw error;
}
}
async function processMessage(
endpoint: EndpointRow,
thread: Thread,
message: Message,
trigger: ChatSdkMessageCallbackEvent["trigger"],
ingressOnly = false,
recoveredProviderUrl: string | null = null,
runtimeContext?: InboundRuntimeContext,
providerUpdateId?: number,
admittedDeliveryId: string | null = null,
receiptReactionSupported = true,
// Kept as a positional compatibility slot for already-normalized durable
// deliveries. Authorization snapshots are intentionally ignored: task
// mutation always rechecks current reach, identity link, and membership.
_preauthorizedUserId?: string | null,
deferDrainUntilFollowup = false,
suppressSetupDestinationActivation = false,
slackSlashControl = false,
) {
if (endpoint.provider === "telegram" && telegramIgnoresInboundMessage(message)) return;
if (
message.author.isMe ||
message.author.isBot === true ||
message.author.isSystem
)
return;
// One provider message can match both a mention handler and the catch-all
// new-message handler. The trigger is policy metadata, not provider event
// identity, so it must not defeat durable deduplication.
const { githubAutomatic, githubIssue, githubManual, providerEventId, surfaceKind, addressed, eventKind, endpointRuntime, nativeInboundAttachments, unavailableTeamsReferences, providerSentAt, providerUrl, normalized, redactedDestinationNormalized } = await prepareInboundEnvelope({ endpoint, thread, message, trigger, receiptReactionSupported, recoveredProviderUrl, runtimeContext, providerUpdateId, admittedDeliveryId, suppressSetupDestinationActivation, slackSlashControl });
const reorderWindow = INGRESS_REORDER_WINDOW_MS[endpoint.provider];
const scheduledAt =
ingressOnly && reorderWindow
? (scheduledConversationDrains.get(
conversationDrainKey(endpoint.id, thread.id),
) ?? Date.now() + reorderWindow)
: null;
const discordGatewayAdmissionState: {
lost?: DiscordGatewayOwnership;
renewed?: {
expiresAt: Date;
ownership: DiscordGatewayOwnership;
};
} = {};
const admission = await admitInboundMessage({ endpoint, thread, message, trigger, runtimeContext, admittedDeliveryId, deferDrainUntilFollowup, scheduledAt, envelope: { providerEventId, addressed, eventKind, normalized, redactedDestinationNormalized, githubAutomatic, githubIssue } }, discordGatewayAdmissionState);
// The database token is authoritative. Publish its new expiry to the
// process-local fast path only after the admission transaction commits.
// Conversely, synchronously fence an obsolete local owner after a failed
// token CAS, then stop it without awaiting the callback it may be serving.
const renewedDiscordOwnership = discordGatewayAdmissionState.renewed;
if (
renewedDiscordOwnership &&
discordGatewayOwnerships.get(endpoint.id) ===
renewedDiscordOwnership.ownership &&
!renewedDiscordOwnership.ownership.stopping
) {
renewedDiscordOwnership.ownership.expiresAt =
renewedDiscordOwnership.expiresAt;
}
const lostDiscordOwnership = discordGatewayAdmissionState.lost;
if (
lostDiscordOwnership &&
discordGatewayOwnerships.get(endpoint.id) === lostDiscordOwnership
) {
lostDiscordOwnership.stopping = true;
void stopDiscordGatewayOwnership(lostDiscordOwnership);
}
if (!admittedDeliveryId && admission?.candidate) {
recordChatWebhookReceipt(
endpoint.id,
admission.candidate.id,
"message_delivery",
providerSentAt,
);
}
if (admission?.accepting) {
// Persist the latest authenticated Teams reply route under a fresh
// runtime/credential fence. The helper holds the endpoint's NO KEY
// UPDATE lock through the SDK-state write, so reconnect, pause, and
// credential rotation cannot overtake it while the state table's
// foreign-key check remains deadlock-free. A failed write escapes so
// Teams retries the already-idempotent delivery.
await recordCurrentMicrosoftTeamsRoute(
endpoint,
runtimeContext,
thread.id,
message.raw,
);
}
let candidate = admission?.candidate ?? null;
if (!admission || !candidate) return;
if (!admission.accepting) {
liveInboundMessages.delete(candidate.id);
return;
}
if (["processed", "filtered", "failed"].includes(candidate.state)) return;
if (
!deferDrainUntilFollowup &&
endpoint.provider === "discord" &&
discordRootThreadPending(candidate)
) {
// The adapter invokes the ordinary message callback only after Discord
// has returned the created thread. Promote the pre-admission receipt
// atomically so the normal ordered drain can begin immediately.
const promoted = await promoteDiscordRootReceipt(candidate.id, scheduledAt);
if (promoted) candidate = promoted;
}
const now = new Date();
if (
!deferDrainUntilFollowup &&
candidate.state === "retry" &&
candidate.nextAttemptAt &&
candidate.nextAttemptAt > now
)
return;
const staleBefore = new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS);
if (candidate.state === "processing" && candidate.updatedAt > staleBefore)
return;
if (ingressOnly) {
// The verified provider request owns only the durable ingress write.
// Task mutation, attachment download, wakeup, and provider-visible
// acknowledgement continue under a durable conversation lease after the
// HTTP response. Keeping the original SDK objects process-locally lets
// the first attempt retain attachment download handles; another process
// can still reconstruct the delivery solely from the durable ledger.
liveInboundMessages.set(candidate.id, {
endpoint,
thread,
message,
trigger,
receiptReactionSupported,
providerUrl,
runtimeContext,
});
if (!deferDrainUntilFollowup) {
scheduleConversationDrain(
endpoint.id,
thread.id,
scheduledAt ?? Date.now(),
);
}
return;
}
const claimConditions = [
eq(chatDeliveries.id, candidate.id),
eq(chatDeliveries.state, candidate.state),
];
if (candidate.state === "processing")
claimConditions.push(lte(chatDeliveries.updatedAt, staleBefore));
const claimed = await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "processing",
attempts: candidate.attempts + 1,
nextAttemptAt: null,
redactedError: null,
updatedAt: now,
})
.where(and(...claimConditions))
.returning();
});
const activeDelivery = claimed[0];
if (!activeDelivery) return;
try {
if (admission.provisionalTeamsSetupReply) {
await deferTeamsSetupReply(db, activeDelivery, redactedDestinationNormalized, ORPHAN_FOLLOW_UP_MAX_ATTEMPTS, ORPHAN_FOLLOW_UP_GRACE_MS);
return;
}
// A committed inbound message link proves that a prior attempt completed
// its task mutation even if it crashed before returning to the provider.
const existingMessageLink = await db
.select({
id: chatMessageLinks.id,
conversationId: chatMessageLinks.conversationId,
commentId: chatMessageLinks.commentId,
})
.from(chatMessageLinks)
.where(
and(
eq(chatMessageLinks.endpointId, endpoint.id),
eq(chatMessageLinks.deliveryId, activeDelivery.id),
eq(chatMessageLinks.direction, "inbound"),
),
)
.then((rows) => rows[0] ?? null);
if (existingMessageLink) {
const inboundCommentId = existingMessageLink.commentId;
if (!inboundCommentId) {
throw new Error("Inbound message link is missing its task comment");
}
const rebound = await db
.select({
conversation: chatConversations,
issueId: chatConversations.issueId,
resourceId: chatConversations.resourceId,
assigneeAgentId: issues.assigneeAgentId,
issueStatus: issues.status,
issueIdentifier: issues.identifier,
authorUserId: issueComments.authorUserId,
})
.from(chatConversations)
.innerJoin(
issues,
and(
eq(issues.companyId, chatConversations.companyId),
eq(issues.id, chatConversations.issueId),
),
)
.leftJoin(issueComments, eq(issueComments.id, inboundCommentId))
.where(
and(
eq(chatConversations.id, existingMessageLink.conversationId),
eq(chatConversations.endpointId, endpoint.id),
),
)
.then((rows) => rows[0] ?? null);
if (!rebound) throw notFound("Bound task not found");
if (!activeDelivery.principalId)
throw new Error("chat_inbound_wakeup_principal_missing");
await stageInboundWakeup(db, {
endpoint,
deliveryId: activeDelivery.id,
conversation: rebound.conversation,
commentId: inboundCommentId,
principalId: activeDelivery.principalId,
actorUserId: rebound.authorUserId,
});
// Attachment storage follows the atomic task/comment/link mutation.
// If a later wakeup or provider subscription failed, retry from the
// committed delivery link and fill in only files that are still
// missing from this exact inbound comment.
const attachmentResult = await ingestAttachments({
endpoint,
endpointRuntime,
deliveryId: activeDelivery.id,
issueId: rebound.issueId,
issueCommentId: inboundCommentId,
attachments: nativeInboundAttachments,
attachmentLimitOmissions: githubAttachmentLimitOmissions(message),
unavailableReferenceCount: unavailableTeamsReferences,
actorUserId: rebound.authorUserId,
});
if (
await settleTelegramAttachmentOnlyFailure({
activeDelivery,
attachmentResult,
commentId: inboundCommentId,
conversationId: existingMessageLink.conversationId,
endpoint,
message,
principalId: activeDelivery.principalId,
resourceId: rebound.resourceId,
runtimeContext,
thread,
})
) {
return;
}
// Subscription is part of the durable acceptance boundary. If it
// fails, keep the delivery retryable; the committed message link makes
// the retry resume here without duplicating the task or comment.
if (addressed && !thread.isDM) await thread.subscribe();
await acceptInboundWakeup(activeDelivery.id, attachmentResult);
if (!(await processInboundWakeup(activeDelivery.id))) return;
const acceptedWake = await db
.select({ status: chatActions.status })
.from(chatActions)
.where(
and(
eq(chatActions.deliveryId, activeDelivery.id),
eq(chatActions.kind, "inbound_wakeup"),
),
)
.limit(1)
.then((rows) => rows[0] ?? null);
if (acceptedWake?.status !== "processed") return;
// Recover the visible receipt as well as the durable wake. The
// receipt action is idempotent and resolves the current runtime fence.
await Promise.allSettled([
receiptReactionSupported
? addReceiptReaction({
deliveryId: activeDelivery.id,
endpoint,
message,
runtimeContext,
thread,
})
: Promise.resolve(),
endpoint.provider === "slack"
? Promise.resolve()
: thread.startTyping("Working…"),
]);
return;
}
const principalResolution = await ensurePrincipal(
endpoint,
message.author,
message.raw,
);
const resourceAdmission = await admitMessageResource({ endpoint, thread, addressed, suppressSetupDestinationActivation, surfaceKind });
const { enabledResourceCount, resource } = resourceAdmission;
await db
.update(chatDeliveries)
.set({
principalId: principalResolution.principal.id,
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
let { conversation: existingConversation, issue: existingIssue, latestConversation, isLinear } =
await resolveInboundConversationBinding(db, {
endpoint, thread,
isLinear: () => endpoint.provider === "speko" ? SPEKO_CONVERSATION_IS_LINEAR : surfaceKind !== "native_thread",
lookupConversation: endpoint.provider === "microsoft-teams"
? () => conversationForThread(endpoint.id, thread.id)
: undefined,
conversationIsComplete: endpoint.provider === "imessage-photon"
? (conversationId) => photonConversationBindingIsComplete(conversationId, hasCommittedTaskControlCompletion)
: undefined,
});
const telegramControls = endpoint.provider === "telegram"
? telegramInboundControls(message.text, surfaceKind)
: null;
const controlCommand = telegramControls
? telegramControls.controlCommand
: isLinear ? linearControlCommand(message.text) : null;
const guidanceCommand = telegramControls?.guidanceCommand ?? null;
const bypassesControlChronology = telegramControls?.bypassesControlChronology ?? controlCommand === "status";
const endpointAllowed =
endpoint.status === "verifying" || endpoint.status === "active";
const destinationAllowed = thread.isDM
? endpoint.allowDirectMessages
: nonDirectDestinationAllowed(endpoint, resource);
const guestSponsorAllowed =
principalResolution.userId === null &&
!principalResolution.linkedDenied &&
endpoint.allowUnlinkedPeople
? await sponsorAllowsGuest(endpoint)
: false;
// This is a second authoritative check, not a provider-name exemption.
// Public voice authority requires an opted-in line, accepted signed call,
// and exact call-owned task with its low-trust containment still intact.
let publicVoiceTaskAllowed = false;
if (endpoint.provider === "speko" && principalResolution.userId === null && !principalResolution.linkedDenied) {
publicVoiceTaskAllowed = await authorizeSpekoPublicVoiceTask(db, endpoint, principalResolution.principal.externalId, voiceStore.authorizePrincipal);
}
const principalAllowed =
!principalResolution.linkedDenied &&
(principalResolution.userId !== null || guestSponsorAllowed || publicVoiceTaskAllowed);
const activationAllowed = addressed || existingConversation !== null;
const allowed =
endpointAllowed &&
destinationAllowed &&
principalAllowed &&
activationAllowed;
const isPlausibleOrphanFollowUp =
endpoint.provider === "github" || endpoint.provider === "discord" ||
(endpoint.provider === "slack" && slackOrphanFollowUp(thread.id, message.id)) ||
(endpoint.provider === "microsoft-teams" && teamsOrphanFollowUp(thread.id, message.id));
const setupDestinationCanBeEnabledByEarlierMention =
(endpoint.provider === "github" ||
endpoint.provider === "discord" ||
(endpoint.provider === "microsoft-teams" &&
teamsRootMentionCanEnableSetupDestination(resource))) &&
!thread.isDM &&
endpoint.status === "verifying" &&
enabledResourceCount === 0 &&
resource.availability === "available";
if (
!allowed &&
isPlausibleOrphanFollowUp &&
!addressed &&
existingConversation === null &&
activeDelivery.attempts <= ORPHAN_FOLLOW_UP_MAX_ATTEMPTS &&
endpointAllowed &&
principalAllowed &&
(destinationAllowed || setupDestinationCanBeEnabledByEarlierMention)
) {
// GitHub, Slack, and Teams can deliver a thread reply before the older
// root callback that creates its Paperclip task. Keep this exact
// delivery for a bounded minute, without admitting or waking it, so
// the durable thread drain can sort again if a delayed root arrives.
// A standalone unaddressed message is filtered after the bounded
// retention window.
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "retry",
nextAttemptAt: new Date(Date.now() + ORPHAN_FOLLOW_UP_GRACE_MS),
redactedError: "Waiting briefly for an earlier root mention",
updatedAt: new Date(),
})
.where(
and(
eq(chatDeliveries.id, activeDelivery.id),
eq(chatDeliveries.state, "processing"),
),
);
});
return;
}
if (!allowed) {
const filteredReason = !endpointAllowed
? "Connection is not active"
: !destinationAllowed
? "Destination is not enabled in Paperclip"
: principalResolution.linkedDenied
? "Linked Paperclip account is not currently permitted"
: !principalAllowed
? endpoint.allowUnlinkedPeople
? "Endpoint sponsor can no longer authorize external guests"
: "External identity must be linked to a Paperclip account"
: "Message did not address the agent or an active task thread";
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "filtered",
normalizedEvent: redactedDestinationNormalized,
principalId: null,
nextAttemptAt: null,
redactedError: filteredReason,
processedAt: new Date(),
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
});
return;
}
const filterPreControlSource = async (database: DbOrTransaction) => {
if (
bypassesControlChronology ||
(await readChatControlChronology(
database,
endpoint,
thread,
activeDelivery,
)) !== "before_or_unproven"
)
return false;
const filteredAt = new Date();
await database
.update(chatDeliveries)
.set({
state: "filtered",
nextAttemptAt: null,
processedAt: filteredAt,
updatedAt: filteredAt,
redactedError:
"Message predates or cannot be ordered after a completed chat close/new. Send a new request to start work.",
})
.where(
and(
eq(chatDeliveries.id, activeDelivery.id),
eq(chatDeliveries.companyId, endpoint.companyId),
eq(chatDeliveries.state, "processing"),
),
);
return true;
};
if (await filterPreControlSource(db)) return;
const photonQuote =
endpoint.provider === "imessage-photon"
? photonReplyReference(message.raw)
: null;
if (controlCommand && photonQuote) {
if (!(await photonControlTargetsConversation(db, endpoint, photonQuote.guid, existingConversation?.id))) {
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "filtered",
processedAt: new Date(),
nextAttemptAt: null,
redactedError:
"Quoted control does not belong to the current task generation",
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
});
return;
}
}
if (guidanceCommand) {
const assignedAgentName =
guidanceCommand === "start"
? await db
.select({ name: agents.name })
.from(agents)
.where(
and(
eq(agents.companyId, endpoint.companyId),
eq(agents.id, endpoint.assignedAgentId),
),
)
.then((rows) => rows[0]?.name ?? "this agent")
: null;
const responseText = telegramGuidanceResponse(guidanceCommand, endpoint.botUsername, assignedAgentName);
const publicationBinding =
existingConversation && existingIssue
? { conversation: existingConversation, issue: existingIssue }
: null;
let effect: typeof chatActions.$inferSelect | null = null;
if (publicationBinding) {
await db.transaction(async (tx) => {
await tx
.update(chatEndpoints)
.set({ lastEventAt: new Date(), updatedAt: new Date() })
.where(eq(chatEndpoints.id, endpoint.id));
await notifyChatDeliveryWork(tx);
await tx
.update(chatDeliveries)
.set({
conversationId: publicationBinding.conversation.id,
state: "processed",
processedAt: new Date(),
redactedError: null,
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
await stageAuthorizedTaskControlPublication(tx, {
companyId: endpoint.companyId,
endpointId: endpoint.id,
conversationId: publicationBinding.conversation.id,
issueId: publicationBinding.issue.id,
idempotencyKey: `control:guidance:${activeDelivery.id}`,
payload: projectSafeChatPublication({
classification: "external",
source: "task_control",
text: responseText,
}),
principalId: principalResolution.principal.id,
});
});
} else {
const effectContext =
runtimeContext ??
runtimeContextForRecord(
(await endpointRecord(endpoint.id)) ??
(() => {
throw new Error("Chat endpoint is unavailable");
})(),
);
effect = await db.transaction((tx) =>
stageProviderEffect(tx, {
endpoint,
deliveryId: activeDelivery.id,
principalId: principalResolution.principal.id,
providerActionId: `provider_effect:delivery:${activeDelivery.id}`,
payload: {
version: 1,
effect: "thread_message",
threadId: thread.id,
text: responseText,
settleDelivery: true,
resourceId: resource.id,
},
runtimeContext: effectContext,
}),
);
if (!effect) throw new Error("Provider effect was not persisted");
if ((await processProviderEffect(effect.id, thread)) !== "processed")
return;
}
// Telegram's command reply is the acknowledgement. A control-only
// request starts no run that could later retire a processing receipt.
await Promise.allSettled([
receiptReactionSupported && endpoint.provider !== "telegram"
? addReceiptReaction({
deliveryId: activeDelivery.id,
endpoint,
message,
runtimeContext,
thread,
})
: Promise.resolve(),
addressed && !thread.isDM ? thread.subscribe() : Promise.resolve(),
]);
return;
}
if (controlCommand) {
const isTelegramForumTopic = telegramControls?.isForumTopic === true;
const taskLabel = existingIssue
? `${existingIssue.identifier}: ${existingIssue.title}`
: "No task is active in this conversation.";
const responseText =
controlCommand === "status"
? existingIssue
? `${taskLabel} — ${existingIssue.status}`
: taskLabel
: controlCommand === "new"
? isTelegramForumTopic
? telegramForumControlResponse("new", taskLabel, Boolean(existingIssue), Boolean(existingConversation))
: "Send your request to start a new Paperclip task."
: existingConversation
? isTelegramForumTopic
? telegramForumControlResponse("close", taskLabel, Boolean(existingIssue), true)
: "This chat conversation is closed. Send another message to start a new task."
: "No task is active. Send a message to start one.";
const publicationBinding =
existingConversation && existingIssue
? { conversation: existingConversation, issue: existingIssue }
: null;
if (publicationBinding) {
const publicationControl =
controlCommand === "new" && isTelegramForumTopic
? "new_guidance"
: controlCommand;
const staged = await db.transaction(async (tx) => {
await tx
.update(chatEndpoints)
.set({ lastEventAt: new Date(), updatedAt: new Date() })
.where(eq(chatEndpoints.id, endpoint.id));
if (await filterPreControlSource(tx)) return false;
await notifyChatDeliveryWork(tx);
await tx
.update(chatDeliveries)
.set({
conversationId: publicationBinding.conversation.id,
state: "processed",
processedAt: new Date(),
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
await stageAuthorizedTaskControlPublication(tx, {
companyId: endpoint.companyId,
endpointId: endpoint.id,
conversationId: publicationBinding.conversation.id,
issueId: publicationBinding.issue.id,
idempotencyKey: `control:${publicationControl}:${activeDelivery.id}`,
payload: projectSafeChatPublication({
classification: "external",
source: "task_control",
text: responseText,
}),
principalId: principalResolution.principal.id,
});
return true;
});
if (!staged) return;
} else {
const effectContext =
runtimeContext ??
runtimeContextForRecord(
(await endpointRecord(endpoint.id)) ??
(() => {
throw new Error("Chat endpoint is unavailable");
})(),
);
const effect = await db.transaction(async (tx) => {
return stageProviderEffect(tx, {
endpoint,
deliveryId: activeDelivery.id,
principalId: principalResolution.principal.id,
providerActionId: `provider_effect:delivery:${activeDelivery.id}`,
payload: {
version: 1,
effect: "thread_message",
threadId: thread.id,
text: responseText,
settleDelivery: true,
resourceId: resource.id,
},
runtimeContext: effectContext,
});
});
if (!effect) throw new Error("Provider effect was not persisted");
if ((await processProviderEffect(effect.id, thread)) !== "processed")
return;
}
// This command stages a reply, not new agent work to acknowledge.
await Promise.allSettled([
receiptReactionSupported && endpoint.provider !== "telegram"
? addReceiptReaction({
deliveryId: activeDelivery.id,
endpoint,
message,
runtimeContext,
thread,
})
: Promise.resolve(),
addressed && !thread.isDM ? thread.subscribe() : Promise.resolve(),
]);
return;
}
const inboundActivityPublications: ActivityPublication[] = [];
const persistTaskMutation = async (
taskTx: DbOrTransaction,
taskEndpoint: EndpointRow,
taskUserId: string | null,
) => {
let conversation = existingConversation;
if (!conversation) {
const [taskResource] = await taskTx.select({ executionDefaults: chatEndpointResources.executionDefaults })
.from(chatEndpointResources).where(and(eq(chatEndpointResources.id, resource.id), eq(chatEndpointResources.endpointId, taskEndpoint.id)));
const executionDefaults = resolveChatExecutionDefaults(taskEndpoint.executionDefaults, taskResource?.executionDefaults);
const responsibleUserId = taskUserId ?? taskEndpoint.sponsorUserId;
await assertChatTaskCreationAccess(db, {
companyId: taskEndpoint.companyId,
assigneeAgentId: taskEndpoint.assignedAgentId,
actor: responsibleUserId ? { type: "board", userId: responsibleUserId, source: "session", ignoreInstanceAdmin: true } : { type: "none" },
defaults: executionDefaults,
}, taskTx);
await assertChatTaskCreationAccess(db, {
companyId: taskEndpoint.companyId,
assigneeAgentId: taskEndpoint.assignedAgentId,
actor: { type: "agent", agentId: taskEndpoint.assignedAgentId, companyId: taskEndpoint.companyId, onBehalfOfUserId: responsibleUserId, source: "agent_key" },
defaults: executionDefaults,
}, taskTx);
const sessionGeneration = isLinear
? (latestConversation?.sessionGeneration ?? 0) + 1
: 1;
const issue = await issuesSvc.create(
taskEndpoint.companyId,
{
title: safeTitle(
githubInboundTaskTitle(message.text, githubAutomatic, githubIssue),
`${PROVIDER_LABELS[taskEndpoint.provider]} conversation`,
),
description: `Started from ${PROVIDER_LABELS[taskEndpoint.provider]}: ${resource.label}`,
status: "todo",
priority: "medium",
assigneeAgentId: taskEndpoint.assignedAgentId,
createdByUserId: taskUserId ?? taskEndpoint.sponsorUserId,
responsibleUserId: taskUserId ?? taskEndpoint.sponsorUserId,
originKind: "chat_channel",
originId: `${taskEndpoint.id}:${thread.id}:${sessionGeneration}`,
idempotencyKey: `chat:${taskEndpoint.id}:${thread.id}:${sessionGeneration}`,
...(executionDefaults.projectId !== undefined ? { projectId: executionDefaults.projectId } : {}),
...(executionDefaults.workspace ? { workspaceSelection: executionDefaults.workspace, workspaceSelectionSource: "channel" as const } : {}),
},
taskTx,
);
await taskTx
.insert(chatConversations)
.values({
companyId: endpoint.companyId,
endpointId: endpoint.id,
resourceId: resource.id,
issueId: issue.id,
externalConversationId: thread.channelId,
externalThreadId: thread.id,
sessionGeneration,
externalLabel: resource.label,
providerUrl,
isDirectMessage: thread.isDM,
communicationGuidance: buildChatCommunicationGuidance({
provider: taskEndpoint.provider,
isDirectMessage: thread.isDM,
communicationInstructions: taskEndpoint.communicationInstructions,
}),
state: "active",
lastActivityAt: new Date(),
})
.onConflictDoNothing();
conversation = await taskTx
.select()
.from(chatConversations)
.where(
and(
eq(chatConversations.endpointId, endpoint.id),
eq(chatConversations.externalConversationId, thread.channelId),
eq(chatConversations.externalThreadId, thread.id),
eq(chatConversations.sessionGeneration, sessionGeneration),
),
)
.then((rows) => rows[0] ?? null);
}
if (!conversation)
throw conflict(
"Could not bind external conversation to a Paperclip task",
);
const issue =
existingIssue?.id === conversation.issueId
? existingIssue
: await taskTx
.select()
.from(issues)
.where(eq(issues.id, conversation.issueId))
.then((rows) => rows[0] ?? null);
if (!issue) throw notFound("Bound task not found");
const githubAgentRequiresLowTrust = taskEndpoint.provider === "github" && await githubAssignedAgentRequiresLowTrust(taskTx, taskEndpoint);
if (!taskUserId || ((githubAutomatic || githubIssue) && !principalResolution.userId) || githubAgentRequiresLowTrust) {
await applyChatTaskReviewContainment(taskTx, endpoint, issue);
}
if (issue.status === "done" || issue.status === "cancelled") {
await issuesSvc.update(
issue.id,
{
status: "todo",
actorUserId: taskUserId ?? taskEndpoint.sponsorUserId,
},
taskTx,
);
}
const body =
githubInboundCommentText({ manual: githubManual, repository: resource.providerResourceId, thread: thread.id, sender: { id: principalResolution.principal.externalId, login: principalResolution.principal.handle }, message: message.text }) ||
(message.attachments.length > 0
? taskEndpoint.provider === "microsoft-teams" &&
!thread.isDM &&
nativeInboundAttachments.length === 0
? teamsAttachmentReferenceText(message.attachments.length, providerUrl)
: `Shared ${message.attachments.length} file${message.attachments.length === 1 ? "" : "s"}.`
: taskEndpoint.provider === "slack" && (trigger === "mention" || message.isMention === true)
? "Started a conversation by mentioning the agent."
: "Sent an empty message.");
let comment!: Awaited<ReturnType<typeof issuesSvc.addComment>>;
await taskTx
.update(chatEndpoints)
.set({
status: taskEndpoint.status,
setup: taskEndpoint.setup,
healthMessage:
taskEndpoint.status === "verifying"
? "Test conversation received"
: "Connected",
lastEventAt: new Date(),
activatedAt:
taskEndpoint.status === "active"
? taskEndpoint.activatedAt
: null,
updatedAt: new Date(),
})
.where(eq(chatEndpoints.id, taskEndpoint.id));
comment = await issuesSvc.addComment(
conversation!.issueId,
body.slice(0, MAX_INBOUND_TEXT),
taskUserId ? { userId: taskUserId } : {},
{
authorType: taskUserId ? "user" : "system",
metadata: {
version: 1,
...(endpoint.provider === "imessage-photon"
? { sourceChannel: "imessage-photon" as const }
: {}),
sections: [
{
title: `${PROVIDER_LABELS[endpoint.provider]} sender`,
rows: [
...(endpoint.provider === "imessage-photon" ? photonReplyMetadata(message.raw) : []),
{
type: "key_value",
label: "Name",
value: message.author.fullName || message.author.userName,
},
{
type: "key_value",
label: "Provider ID",
value: stableExternalPrincipalId(
endpoint.provider,
message.author,
message.raw,
),
},
{
type: "key_value",
label: "Authority",
value: taskUserId
? "Linked Paperclip user"
: "Sponsored external guest (restricted)",
},
],
},
],
},
sourceTrust: taskUserId
? null
: {
preset: LOW_TRUST_REVIEW_PRESET,
disposition: "quarantined",
sourceIssueId: conversation!.issueId,
},
},
taskTx,
);
await taskTx
.update(chatDeliveries)
.set({
conversationId: conversation!.id,
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, activeDelivery.id));
await taskTx
.insert(chatMessageLinks)
.values({
companyId: endpoint.companyId,
endpointId: endpoint.id,
conversationId: conversation!.id,
deliveryId: activeDelivery.id,
commentId: comment.id,
providerMessageId: message.id,
direction: "inbound",
})
.onConflictDoNothing();
if (taskEndpoint.provider === "slack" || taskEndpoint.provider === "speko") {
await resumeExternalConversation(taskTx as unknown as Db, endpoint.companyId, conversation.issueId);
}
await taskTx
.update(chatConversations)
.set({
state: "active",
lastActivityAt: new Date(),
...(providerUrl ? { providerUrl } : {}),
updatedAt: new Date(),
})
.where(eq(chatConversations.id, conversation!.id));
await taskTx
.update(toolConnections)
.set({
status: "active",
enabled: true,
healthStatus: "healthy",
healthMessage: "Connected",
lastHealthAt: new Date(),
updatedAt: new Date(),
})
.where(eq(toolConnections.id, taskEndpoint.connectionId));
if (endpoint.provider === "github") {
await retainGitHubDeliveryAuthority(taskTx, taskEndpoint, activeDelivery.id, principalResolution.userId, taskUserId);
}
if (githubAutomatic) {
await retainGitHubReviewReceipt(taskTx, endpoint, issue.id, activeDelivery.id, githubAutomatic);
}
await stageInboundWakeup(taskTx, {
endpoint: taskEndpoint,
deliveryId: activeDelivery.id,
conversation,
commentId: comment.id,
principalId: principalResolution.principal.id,
actorUserId: taskUserId,
});
if (taskEndpoint.provider === "imessage-photon") {
await recordPhotonInboundActivity(taskTx, taskEndpoint, issue, comment.id, taskUserId, inboundActivityPublications);
}
return { actorUserId: taskUserId, comment, conversation, issue };
};
const taskMutation = await db.transaction(async (tx) => {
// Admission and durable task mutation are separate so the provider
// webhook can return inside its response budget. Take the endpoint and
// destination locks again at the authoritative mutation boundary for
// every provider: either a reach revocation commits first and this
// event is redacted, or this task/comment commits first and the
// revocation waits. There is no stale-snapshot middle.
await options.reachAuthorizationBarrier?.();
const currentEndpoint = await tx
.select()
.from(chatEndpoints)
.where(eq(chatEndpoints.id, endpoint.id))
.for("update")
.then((rows) => rows[0] ?? null);
if (!currentEndpoint) throw notFound("Chat endpoint not found");
const currentResource = thread.isDM
? resource
: await tx
.select()
.from(chatEndpointResources)
.where(
and(
eq(chatEndpointResources.id, resource.id),
eq(chatEndpointResources.endpointId, endpoint.id),
),
)
.for("update")
.then((rows) => rows[0] ?? null);
const currentPrincipalAuthorization =
await lockCurrentPrincipalAuthorization(
tx,
currentEndpoint,
principalResolution.principal.id,
);
const automaticContext = githubAutomatic?.context ?? githubIssue?.context;
const automaticAdmission = automaticContext ? await githubAutomaticAdmission(tx, currentEndpoint, automaticContext) : null;
if (automaticContext && !automaticAdmission?.allowed) currentPrincipalAuthorization.allowed = false;
if (automaticAdmission?.allowed) currentPrincipalAuthorization.userId = automaticAdmission.responsibleUserId ?? null;
const endpointStillAllowed =
currentEndpoint.status === "verifying" ||
currentEndpoint.status === "active";
const destinationStillAllowed = thread.isDM
? currentEndpoint.allowDirectMessages
: nonDirectDestinationAllowed(currentEndpoint, currentResource);
const mentionStillAllowed =
currentEndpoint.provider !== "slack" ||
slackMessageMeetsMentionRequirement(currentEndpoint, thread, trigger, message);
if (
!endpointStillAllowed ||
!mentionStillAllowed ||
!destinationStillAllowed ||
!currentPrincipalAuthorization.allowed
) {
const filteredAt = new Date();
await notifyChatDeliveryWork(tx);
await tx
.update(chatDeliveries)
.set({
state: "filtered",
normalizedEvent: redactedDestinationNormalized,
principalId: null,
nextAttemptAt: null,
processedAt: filteredAt,
redactedError: !mentionStillAllowed
? "Message did not @mention the bot"
: endpointStillAllowed
? destinationStillAllowed
? currentPrincipalAuthorization.linkedDenied
? "Linked Paperclip account is not currently permitted"
: currentEndpoint.allowUnlinkedPeople
? "Endpoint sponsor can no longer authorize external guests"
: "External identity must be linked to a Paperclip account"
: "Destination is not enabled in Paperclip"
: "Connection is not active",
updatedAt: filteredAt,
})
.where(
and(
eq(chatDeliveries.id, activeDelivery.id),
eq(chatDeliveries.state, "processing"),
),
);
// A principal first observed only by this now-revoked event must not
// survive as a side-channel. Preserve established or linked
// identities referenced by any other durable record.
await tx
.delete(chatExternalPrincipals)
.where(
and(
eq(chatExternalPrincipals.id, principalResolution.principal.id),
notExists(
tx
.select({ id: chatDeliveries.id })
.from(chatDeliveries)
.where(
eq(
chatDeliveries.principalId,
principalResolution.principal.id,
),
),
),
notExists(
tx
.select({ id: chatIdentityLinks.id })
.from(chatIdentityLinks)
.where(
eq(
chatIdentityLinks.principalId,
principalResolution.principal.id,
),
),
),
notExists(
tx
.select({ id: chatActions.id })
.from(chatActions)
.where(
eq(
chatActions.principalId,
principalResolution.principal.id,
),
),
),
),
);
return null;
}
// Close completion takes this endpoint lock in its receipt transaction.
// Recheck after the lock: an old input held between precheck and task
// mutation must not create work after that control commits.
if (await filterPreControlSource(tx)) return null;
return persistTaskMutation(
tx,
currentPrincipalAuthorization.sponsorUserId ? { ...currentEndpoint, sponsorUserId: currentPrincipalAuthorization.sponsorUserId } : currentEndpoint,
currentPrincipalAuthorization.userId,
);
});
if (!taskMutation) return;
// The open task can fetch the comment immediately, before attachments
// finish preparing or the agent starts. Never publish an uncommitted row.
for (const publication of inboundActivityPublications) {
publishActivity(publication);
}
const { actorUserId, comment, conversation, issue } = taskMutation;
const attachmentResult = await ingestAttachments({
endpoint,
endpointRuntime,
deliveryId: activeDelivery.id,
issueId: conversation.issueId,
issueCommentId: comment.id,
attachments: nativeInboundAttachments,
attachmentLimitOmissions: githubAttachmentLimitOmissions(message),
unavailableReferenceCount: unavailableTeamsReferences,
actorUserId,
});
if (
await settleTelegramAttachmentOnlyFailure({
activeDelivery,
attachmentResult,
commentId: comment.id,
conversationId: conversation.id,
endpoint,
message,
principalId: principalResolution.principal.id,
resourceId: resource.id,
runtimeContext,
thread,
})
) {
return;
}
// Do not discard a subscription failure after marking the delivery
// processed. A retry reuses the committed message link above and tries
// this idempotent subscription again before completing the delivery.
if (addressed && !thread.isDM) await thread.subscribe();
await acceptInboundWakeup(activeDelivery.id, attachmentResult);
if (!(await processInboundWakeup(activeDelivery.id))) return;
const acceptedWake = await db
.select({ status: chatActions.status })
.from(chatActions)
.where(
and(
eq(chatActions.deliveryId, activeDelivery.id),
eq(chatActions.kind, "inbound_wakeup"),
),
)
.limit(1)
.then((rows) => rows[0] ?? null);
if (acceptedWake?.status !== "processed") return;
// Provider-visible acknowledgement begins only after the task, external
// comment, durable wakeup request, delivery state, and message link commit.
await Promise.allSettled([
receiptReactionSupported
? addReceiptReaction({
deliveryId: activeDelivery.id,
endpoint,
message,
runtimeContext,
thread,
})
: Promise.resolve(),
// Slack implements this through assistant.threads.setStatus, which
// requires assistant:write. The least-privilege Paperclip manifest
// deliberately does not request that scope; the coalesced lifecycle
// reply below is the visible working state instead.
endpoint.provider === "slack"
? Promise.resolve()
: thread.startTyping("Working…"),
]);
} catch (error) {
const providerEffectAmbiguous =
error !== null &&
typeof error === "object" &&
"code" in error &&
error.code === "CHAT_PROVIDER_EFFECT_AMBIGUOUS";
const terminal =
providerEffectAmbiguous ||
isExternalActionAuthorizationChange(error) ||
activeDelivery.attempts >= 5;
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: terminal ? "failed" : "retry",
nextAttemptAt: terminal
? null
: new Date(
Date.now() +
Math.min(60_000, 1000 * 2 ** activeDelivery.attempts),
),
redactedError: redactError(error),
updatedAt: new Date(),
})
.where(
and(
eq(chatDeliveries.id, activeDelivery.id),
eq(chatDeliveries.state, "processing"),
),
);
});
if (terminal) await settleRejectedInboundWakeups(activeDelivery.id);
throw error;
}
}
async function drainConversationDeliveries(...args: Parameters<ReturnType<typeof createChatConversationDeliveryQueue>["drainConversationDeliveries"]>) {
return conversationDeliveryQueue.drainConversationDeliveries(...args);
}
async function handleSdkMessage(
event: ChatSdkMessageCallbackEvent,
runtimeContext?: RuntimeContext,
messageOptions: {
receiptReactionSupported?: boolean;
slackSlashControl?: boolean;
} = {},
) {
if (event.provider === "discord" && !(await admitDiscordSdkMessage(
event.endpointId,
runtimeContext,
{
ensureRuntimeIsCurrent: ensureDiscordGatewayRuntimeIsCurrent,
messageAdmissionBarrier: () => options.discordGatewayMessageAdmissionBarrier?.(),
},
))) return;
return receiveSdkMessages(event, runtimeContext, messageOptions);
}
async function handleMessageUpdated(...args: Parameters<ReturnType<typeof createChatMessageLifecycle>["handleMessageUpdated"]>) {
return messageLifecycle.handleMessageUpdated(...args);
}
async function handleMessageDeleted(...args: Parameters<ReturnType<typeof createChatMessageLifecycle>["handleMessageDeleted"]>) {
return messageLifecycle.handleMessageDeleted(...args);
}
async function handleSlashCommand(
event: ChatSdkCallbackEvent<SlashCommandEvent>,
runtimeContext: RuntimeContext,
) {
if (event.provider === "discord")
return handleDiscordNativeCommand(event, runtimeContext);
const record = await runtimeCallbackRecord(
event.endpointId,
runtimeContext,
["verifying", "active"],
);
if (!record) return;
const command = event.event.command.toLowerCase().split("@")[0];
const text = event.event.text.trim();
if (event.provider === "telegram") return handleTelegramSlashCommand(event, runtimeContext, record, command, text);
return handleSlackSlashCommand(event, runtimeContext, record, command, text);
}
async function applyProviderLifecycleEffect(
endpoint: EndpointRow,
effect: ChatProviderLifecycleEffect,
runtimeContext?: RuntimeContext,
): Promise<boolean> {
const providerEventId = `lifecycle:${effect.providerEventId}`;
const deduplicationKey = createHash("sha256")
.update(`${effect.provider}:${providerEventId}`)
.digest("hex");
const eventKind: ChatEventKind =
effect.availability === "available" ? "installation" : "uninstallation";
const [inserted] = await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.insert(chatDeliveries)
.values({
companyId: endpoint.companyId,
endpointId: endpoint.id,
providerEventId,
deduplicationKey,
eventKind,
normalizedEvent: {
providerEventId,
kind: eventKind,
lifecycle: effect,
...(runtimeContext
? {
runtimeContext: {
credentialFingerprint: runtimeContext.credentialFingerprint,
generation: runtimeContext.generation,
},
}
: {}),
},
state: "received",
})
.onConflictDoNothing()
.returning();
});
const candidate =
inserted ??
(await db
.select()
.from(chatDeliveries)
.where(
and(
eq(chatDeliveries.endpointId, endpoint.id),
eq(chatDeliveries.providerEventId, providerEventId),
),
)
.then((rows) => rows[0] ?? null));
if (!candidate || candidate.state === "processed") return false;
const staleBefore = new Date(Date.now() - DELIVERY_PROCESSING_STALE_MS);
const [claimed] = await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "processing",
attempts: candidate.attempts + 1,
redactedError: null,
nextAttemptAt: null,
updatedAt: new Date(),
})
.where(
and(
eq(chatDeliveries.id, candidate.id),
or(
inArray(chatDeliveries.state, ["received", "retry"]),
and(
eq(chatDeliveries.state, "processing"),
lte(chatDeliveries.updatedAt, staleBefore),
),
),
),
)
.returning({ id: chatDeliveries.id });
});
if (!claimed) return false;
let refreshRuntimeAfterLifecycle = false;
let stopRuntimeAfterFailure = false;
let stopRuntimeAfterLifecycle = false;
let lifecycleFailurePersisted = false;
try {
await withCredentialMutationLease(endpoint, async (credentialLease) => {
const currentRecord = await endpointRecord(endpoint.id);
if (!currentRecord) throw notFound("Chat endpoint not found");
const currentEndpoint = currentRecord.endpoint;
const admittedRuntimeContext = lifecycleRuntimeFence(candidate);
const currentRuntimeContext = runtimeContextForRecord(currentRecord);
if (
!admittedRuntimeContext ||
admittedRuntimeContext.generation !==
currentRuntimeContext.generation ||
admittedRuntimeContext.credentialFingerprint !==
currentRuntimeContext.credentialFingerprint
) {
const filteredAt = new Date();
await db.transaction(async (tx) => {
await credentialLease.assertOwned(tx);
await notifyChatDeliveryWork(tx);
await tx
.update(chatDeliveries)
.set({
state: "filtered",
processedAt: filteredAt,
nextAttemptAt: null,
redactedError:
"Provider lifecycle callback belonged to a superseded runtime",
updatedAt: filteredAt,
})
.where(
and(
eq(chatDeliveries.id, claimed.id),
eq(chatDeliveries.state, "processing"),
eq(chatDeliveries.attempts, candidate.attempts + 1),
),
);
await credentialLease.assertOwned(tx);
});
return;
}
if (currentEndpoint.provider === "github") {
const outcome = await applyGitHubInstallationLifecycle({
currentEndpoint, effect, credentialLease, admittedRuntimeContext, claimed, candidate,
onFailurePersisted: () => {
stopRuntimeAfterFailure = true;
lifecycleFailurePersisted = true;
},
});
refreshRuntimeAfterLifecycle = outcome.refreshRuntimeAfterLifecycle;
stopRuntimeAfterLifecycle = outcome.stopRuntimeAfterLifecycle;
return;
}
const staleResourceEffect =
effect.kind === "resource" &&
(await hasNewerProcessedProviderLifecycleEffect(
currentEndpoint,
effect,
));
const endpointEffectStopsRuntime =
effect.kind === "endpoint" &&
(effect.availability === "attention" ||
effect.availability === "revoked");
await applyProviderAvailability({ currentEndpoint, effect, credentialLease, staleResourceEffect, claimed, candidate });
stopRuntimeAfterLifecycle = endpointEffectStopsRuntime;
});
} catch (error) {
if (stopRuntimeAfterFailure) {
await invalidateRuntime(endpoint.id).catch((runtimeError) => {
logger.warn(
{ endpointId: endpoint.id, error: redactError(runtimeError) },
"failed to stop GitHub endpoint after lifecycle revalidation failed",
);
});
}
if (!lifecycleFailurePersisted) {
const terminal = candidate.attempts + 1 >= 5;
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: terminal ? "failed" : "retry",
redactedError: redactError(error),
nextAttemptAt: terminal ? null : new Date(),
updatedAt: new Date(),
})
.where(
and(
eq(chatDeliveries.id, claimed.id),
eq(chatDeliveries.state, "processing"),
eq(chatDeliveries.attempts, candidate.attempts + 1),
),
)
.catch(() => undefined);
});
}
throw error;
}
if (refreshRuntimeAfterLifecycle) {
// The verified reinstall may have assigned a new installation id. Drop
// the runtime that authenticated the lifecycle webhook with the prior
// id; the next send recreates it from the newly persisted credentials.
await invalidateRuntime(endpoint.id).catch((error) => {
logger.warn(
{ endpointId: endpoint.id, error: redactError(error) },
"failed to refresh recovered GitHub chat endpoint runtime",
);
});
}
if (stopRuntimeAfterLifecycle) {
await invalidateRuntime(endpoint.id).catch((error) => {
logger.warn(
{ endpointId: endpoint.id, error: redactError(error) },
"failed to stop unavailable chat endpoint runtime",
);
});
}
return true;
}
async function applyProviderLifecycleEffects(
endpoint: EndpointRow,
effects: ChatProviderLifecycleEffect[],
runtimeContext: RuntimeContext,
) {
for (const effect of effects) {
if (effect.provider !== endpoint.provider) continue;
const applied = await applyProviderLifecycleEffect(
endpoint,
effect,
runtimeContext,
);
if (applied && endpoint.provider === "slack") {
await scheduleSlackLifecycleResourceHydration(db, endpoint, effect, { scheduleMessageProcessing, hydrateSlackResourceLabel, redactError });
}
}
}
async function handleWebhook(
publicId: string,
provider: ChatSdkProvider,
request: Request,
internalContext?: {
githubIngressActionId: string;
githubIngressFence: LifecycleRuntimeFence;
superseded?: boolean;
},
) {
const serviceReceivedAtMs = Date.now();
const githubScope = createGitHubWebhookScope(provider, internalContext, options.githubWebhookResponseBudgetMs, GITHUB_WEBHOOK_RESPONSE_BUDGET_MS);
const { replayingDurableGitHubIngress, ignoreSupersededIngress, matchesGitHubIngressFence, githubResponseDeadlineAt } = githubScope;
const endpoint = await db
.select()
.from(chatEndpoints)
.where(
and(
eq(chatEndpoints.publicId, publicId),
eq(chatEndpoints.provider, provider),
),
)
.then((rows) => rows[0] ?? null);
const recoveringRevokedGitHubInstallation =
githubWebhookCanRecoverRevokedInstallation(endpoint?.status, provider, request);
if (
!endpoint ||
endpoint.status === "archived" ||
(endpoint.status === "revoked" && !recoveringRevokedGitHubInstallation)
) {
if (replayingDurableGitHubIngress) return ignoreSupersededIngress();
throw notFound("Chat endpoint not found");
}
recordChatWebhookStage("endpoint_resolved", endpoint.id);
// A pause is a durable ingress fence. Providers generally retry non-2xx
// webhooks, so acknowledge late callbacks without recreating a runtime or
// admitting any Paperclip mutation.
if (endpoint.status === "paused") {
if (replayingDurableGitHubIngress) return ignoreSupersededIngress();
return new Response("ignored", { status: 200 });
}
if (provider === "slack") {
const verification = await verifySlackRequestUrl(endpoint, request);
if (verification) return verification;
}
if (provider === "github") {
const response = await authenticateGitHubWebhook(endpoint, request, githubScope);
if (response) return response;
}
if (provider === "slack") {
const response = await inspectSlackWebhookScope(endpoint, request);
if (response) return response;
}
const lifecycleInspection = request.clone();
const slackCallbackInspection =
provider === "slack" ? request.clone() : null;
const githubInspection = provider === "github" ? request.clone() : null;
recordChatWebhookStage("runtime_requested");
const runtimePromise = runtimeFor(endpoint);
const initialized = await initializeGitHubWebhookRuntime(endpoint, runtimePromise, githubResponseDeadlineAt, redactError);
if (initialized.response) return initialized.response;
const endpointRuntime = initialized.runtime;
recordChatWebhookStage("runtime_ready");
const runtimeContext = runtimeContexts.get(endpointRuntime as object);
if (!runtimeContext) {
throw conflict("Chat endpoint runtime is not current", {
code: "chat_endpoint_runtime_superseded",
});
}
// runtimeFor can return a newer instance than the leased preflight saw.
// Keep the exact staged epoch; callbacks on a matching instance retain
// their normal transactional fences if a pause wins after this check.
if (!matchesGitHubIngressFence(runtimeContext))
return ignoreSupersededIngress();
if (provider === "github") {
const response = await handleGitHubWebhookEvent(endpoint, request, endpointRuntime, runtimeContext);
if (response) return response;
}
const response = await endpointRuntime.handleWebhook(
request,
undefined,
githubResponseDeadlineAt ?? undefined,
provider === "telegram" ? serviceReceivedAtMs : undefined,
);
let lifecyclePayload: unknown = null;
if (response.ok) {
try {
const contentType =
lifecycleInspection.headers.get("content-type") ?? "";
if (contentType.includes("application/json"))
lifecyclePayload = await lifecycleInspection.json();
} catch {
// The native adapter remains authoritative for non-JSON callback
// envelopes such as Slack slash commands.
}
}
if (
response.ok &&
lifecyclePayload &&
endpointRuntime.acceptsProviderScope(lifecyclePayload)
) {
if (endpoint.provider === "slack") {
await handleSlackSessionStopPayload(endpoint, lifecyclePayload, runtimeContext);
}
const effects = parseChatProviderLifecycle({
provider: endpoint.provider,
headers: lifecycleInspection.headers,
payload: lifecyclePayload,
botExternalId: endpoint.botExternalId,
});
// Provider success is returned only after every recognized lifecycle
// transition is durable. A persistence failure therefore becomes a
// non-2xx webhook response and asks the provider to retry.
await applyProviderLifecycleEffects(endpoint, effects, runtimeContext);
}
if (slackCallbackInspection && response.ok) {
await recordSlackWebhookCallback(endpoint, request, slackCallbackInspection, runtimeContext);
}
if (response.ok) {
const lifecycle = githubInspection
? await githubSupplementalLifecycleFromRequest(githubInspection)
: provider === "telegram" && lifecyclePayload
? telegramLifecycleEventFromPayload(lifecyclePayload)
: provider === "microsoft-teams"
? teamsWebhookLifecycleEventFromPayload(lifecyclePayload, endpointRuntime)
: null;
if (lifecycle) {
await recordLifecycleDelivery(
{
endpointId: endpoint.id,
...lifecycle,
},
runtimeContext,
);
}
}
return response;
}
/**
* Reconcile a verified normalized delivery whose original request was
* interrupted. Closed, credential-free provider descriptors reconstruct
* authenticated attachment downloads through the endpoint runtime; unsafe
* or no-longer-available files are omitted without losing the text turn.
*/
async function processPendingChatMaintenance(limit = 25) {
return Promise.allSettled([
processPendingTelegramMaintenance(limit),
]);
}
async function processPendingDeliveries(limit = 25, onlyDeliveryId?: string, maintenance = true) {
await settleRejectedInboundWakeups(onlyDeliveryId);
// Provider-visible effects that are not backed by a task publication use
// chat_actions as their outbox. Reconcile them before inbound deliveries
// so a crashed processing claim is quarantined before the delivery worker
// could otherwise replay it.
const actionRecovery = onlyDeliveryId || !maintenance ? null : Promise.allSettled([
processPendingChatMaintenance(limit), processFailedChatRunRetries(limit),
processPendingGitHubWebhookIngress(limit),
processPendingProviderEffects(limit), processPendingSlackBoardMessages(limit),
processPendingSlackTaskStarts(limit), processPendingReceiptReactions(limit),
processPendingSlackSessionStops(limit), slackRegistration.processPendingVerificationMessages(limit),
]);
const reactionRecovery = processPendingReactionDeliveries(
limit,
onlyDeliveryId,
).catch((error) => {
logger.warn(
{ error: redactError(error) },
"chat reaction-link recovery selection failed",
);
return 0;
});
let ordinaryFailed = false;
let ordinaryError: unknown = null;
let rows: DeliveryRow[] = [];
try {
const now = new Date();
const staleBefore = new Date(
now.getTime() - DELIVERY_PROCESSING_STALE_MS,
);
rows = await db
.select()
.from(chatDeliveries)
.where(
and(
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.provider = 'agentmail')`,
onlyDeliveryId ? eq(chatDeliveries.id, onlyDeliveryId) : undefined,
notInArray(chatDeliveries.eventKind, [
"reaction_added",
"reaction_removed",
]),
or(
and(
eq(chatDeliveries.state, "processed"),
pendingInboundWakeupCondition(true),
),
and(
eq(chatDeliveries.state, "received"),
or(
isNull(chatDeliveries.nextAttemptAt),
lte(chatDeliveries.nextAttemptAt, now),
),
),
and(
eq(chatDeliveries.state, "retry"),
or(
isNull(chatDeliveries.nextAttemptAt),
lte(chatDeliveries.nextAttemptAt, now),
),
),
and(
eq(chatDeliveries.state, "processing"),
lte(chatDeliveries.updatedAt, staleBefore),
),
),
),
)
.orderBy(asc(chatDeliveries.receivedAt))
.limit(limit);
const conversations = new Set<string>();
for (const delivery of rows) {
const lifecycleEffect = normalizedLifecycleEffect(delivery);
if (lifecycleEffect) {
const record = await endpointRecord(delivery.endpointId);
if (!record || record.endpoint.status === "archived") {
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "failed",
redactedError: "Chat endpoint is no longer available",
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, delivery.id));
});
continue;
}
try {
await applyProviderLifecycleEffect(
record.endpoint,
lifecycleEffect,
);
} catch (error) {
logger.warn(
{
endpointId: delivery.endpointId,
deliveryId: delivery.id,
error: redactError(error),
},
"chat provider lifecycle retry failed",
);
}
continue;
}
const externalThreadId = normalizedDeliveryThreadId(delivery);
if (!externalThreadId) {
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
return tx
.update(chatDeliveries)
.set({
state: "failed",
redactedError: "Normalized delivery is incomplete",
updatedAt: new Date(),
})
.where(eq(chatDeliveries.id, delivery.id));
});
continue;
}
const key = conversationDrainKey(delivery.endpointId, externalThreadId);
if (conversations.has(key)) continue;
conversations.add(key);
await drainConversationDeliveries(
delivery.endpointId,
externalThreadId,
);
}
} catch (error) {
ordinaryFailed = true;
ordinaryError = error;
}
// Provider recovery runs independently so a slow Slack Web API request
// cannot hold unrelated verified inbound messages behind it. Callers that
// explicitly replay one delivery skip these global sweeps entirely.
if (actionRecovery) await actionRecovery;
const reactionCount = await reactionRecovery;
if (ordinaryFailed) throw ordinaryError;
return reactionCount + rows.length;
}
async function replayDelivery(endpointId: string, deliveryId: string) {
const delivery = await db
.select()
.from(chatDeliveries)
.where(
and(
eq(chatDeliveries.endpointId, endpointId),
eq(chatDeliveries.id, deliveryId),
),
)
.then((rows) => rows[0] ?? null);
if (!delivery) throw notFound("Delivery not found");
if (!delivery.conversationId)
throw unprocessable(
"Only a delivery already bound to a task can be replayed safely",
);
const ambiguousProviderEffect = await db
.select({ id: chatActions.id })
.from(chatActions)
.where(
and(
eq(chatActions.endpointId, endpointId),
eq(chatActions.deliveryId, delivery.id),
eq(chatActions.kind, "provider_effect"),
eq(chatActions.status, "delivery_unknown"),
),
)
.limit(1)
.then((rows) => rows[0] ?? null);
if (ambiguousProviderEffect) {
throw conflict(
"Resolve the unconfirmed provider reply before replaying this delivery",
{
code: "chat_delivery_provider_effect_resolution_required",
},
);
}
if (delivery.state !== "failed") {
throw conflict("Only a failed delivery can be replayed", {
code: "chat_delivery_not_failed",
});
}
await db.transaction(async (tx) => {
await notifyChatDeliveryWork(tx);
const claimed = await tx
.update(chatDeliveries)
.set({
state: "retry",
nextAttemptAt: new Date(),
redactedError: null,
updatedAt: new Date(),
})
.where(
and(
eq(chatDeliveries.id, delivery.id),
eq(chatDeliveries.state, "failed"),
eq(chatDeliveries.attempts, delivery.attempts),
),
)
.returning({ id: chatDeliveries.id });
if (!claimed.length) {
throw conflict("This delivery is already being replayed", {
code: "chat_delivery_replay_conflict",
});
}
// The operator requested a fresh attempt at an input that never reached
// scheduling. Preserve its immutable scope; acceptance rechecks the
// original external actor and current reach before any admission.
await tx
.update(chatActions)
.set({ status: "preparing", result: null, updatedAt: new Date() })
.where(
and(
eq(chatActions.endpointId, endpointId),
eq(chatActions.deliveryId, delivery.id),
eq(chatActions.kind, "inbound_wakeup"),
eq(chatActions.status, "failed"),
sql`not exists (select 1 from ${agentWakeupRequests} where ${agentWakeupRequests.id} = ${chatActions.id})`,
),
);
});
await processPendingDeliveries(1, delivery.id);
const replayed = await db
.select({ state: chatDeliveries.state })
.from(chatDeliveries)
.where(eq(chatDeliveries.id, delivery.id))
.then((rows) => rows[0] ?? null);
if (replayed?.state !== "processed") {
throw conflict("The delivery replay did not complete", {
code: "chat_delivery_replay_incomplete",
state: replayed?.state ?? "missing",
});
}
}
async function resolveAction(
endpointId: string,
actionId: string,
resolution: "mark_delivered" | "retry_anyway" | "cancel",
userId: string,
) {
const initialRecord = await endpointRecord(endpointId);
if (!initialRecord) throw notFound("Chat endpoint not found");
const initialAction = await db
.select({ kind: chatActions.kind })
.from(chatActions)
.where(
and(
eq(chatActions.endpointId, endpointId),
eq(chatActions.id, actionId),
),
)
.then((rows) => rows[0] ?? null);
if (!initialAction) throw notFound("Provider action not found");
if (initialAction.kind === "provider_effect") {
await resolveProviderEffect(endpointId, actionId, resolution, userId);
return;
}
return resolveSlackTaskStart(endpointId, actionId, resolution, userId, initialRecord);
}
async function recordExactOutboundPublicationLink(...args: Parameters<ReturnType<typeof createChatPublicationTransport>["recordExactOutboundPublicationLink"]>) {
return publicationTransport.recordExactOutboundPublicationLink(...args);
}
async function stageTerminalReceiptReactionRemovals(...args: Parameters<ReturnType<typeof createChatPublicationWorker>["stageTerminalReceiptReactionRemovals"]>) {
return publicationWorker.stageTerminalReceiptReactionRemovals(...args);
}
async function processPendingPublications(...args: Parameters<ReturnType<typeof createChatPublicationQueue>["processPendingPublications"]>) {
return publicationQueue.processPendingPublications(...args);
}
const githubReviews = githubChatReviewService(db, fetchImpl);
const { scheduleGitHubMaintenance } = createGitHubMaintenance(db, fetchImpl, {
scheduleMessageProcessing, isShuttingDown: () => shuttingDown,
});
async function processPublicationMaintenance(limit = 25) {
scheduleTeamsFileMaintenance();
scheduleGitHubMaintenance();
await enqueueInboundWakeupPublications();
await reconcileTerminalConfirmationActions(limit);
await processPendingInteractionWakeups(limit);
}
// The app refills free endpoint slots on commits and durable deadlines.
// Work remains tracked by this service and is joined before runtime shutdown.
async function schedulePendingPublications(limit = 25) {
await processPublicationMaintenance(limit);
return processPendingPublications(limit, { waitForCompletion: false, maintenance: false });
}
let wakePublications = () => {};
async function nextPublicationAt(): Promise<number | null> {
const cooldowns = Object.fromEntries([...teamsFileRetryAfter].map(([id, at]) => [id, new Date(at).toISOString()]));
const [queued, streaming] = await Promise.all([
db.select({ at: sql<string | null>`min(greatest(coalesce(${chatPublications.nextAttemptAt}, now()),
(${JSON.stringify(cooldowns)}::jsonb->>${chatPublications.id}::text)::timestamptz))` })
.from(chatPublications).where(publicationEligibility({ busyEndpointIds: [...publicationEndpointTasks.keys()] })),
db.select({ at: sql<string | null>`min(greatest(${chatPublications.updatedAt} + interval '60 seconds',
(select max(l.expires_at) from chat_endpoint_leases l
where l.company_id = chat_publications.company_id and l.endpoint_id = chat_publications.endpoint_id
and l.lease_key = 'publication:' || chat_publications.id::text || ':' || chat_publications.attempts::text)))` })
.from(chatPublications).where(and(
eq(chatPublications.state, "streaming"),
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`,
sql`not exists (select 1 from ${chatTeamsFileTransfers} f where f.company_id = ${chatPublications.companyId} and f.publication_id = ${chatPublications.id})`,
)),
]);
const deadlines = [queued[0]?.at, streaming[0]?.at].filter(at => at != null).map(at => new Date(at!).getTime());
return deadlines.length ? Math.min(...deadlines) : null;
}
async function getIssueBinding(
issueId: string,
): Promise<ExternalChannelBindingSummary | null> {
const companyId = await db
.select({ companyId: issues.companyId })
.from(issues)
.where(eq(issues.id, issueId))
.then((rows) => rows[0]?.companyId ?? null);
return companyId
? getExternalChannelBindingSummary(db, companyId, issueId)
: null;
}
const { configure, configureWithCredentialLease, runtimeConfig } = createChatConfiguration(db, {
endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime,
nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict,
legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials,
persistCredentials, clearCredentials, resolveCredentials, prepareProviderInventory,
reconcileProviderResourceRows, reconcileDiscordCommands, runtimeFor, runtimeContextForRecord,
stageTelegramMaintenance, processTelegramMaintenance, recordTelegramStopSubscription,
fetchImpl, redactError, runtimeGeneration, leasedChatProvider, PROVIDER_LABELS, SUPPLIED_CREDENTIAL_KEYS,
PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, isSupportedTelegramWebhookBaseUrl,
get slackRegistration() { return slackRegistration; },
get githubWizard() { return githubWizard; },
get nativeBotIdentityClaimBarrier() { return options.nativeBotIdentityClaimBarrier; },
get allowLocalVoiceBoard() { return options.allowLocalVoiceBoard; },
});
const { slackRegistration, verifySlackRequestUrl, sendSlackTestMessage } = createSlackSetupServices(db, {
endpointRecord, ensurePrincipal, withCredentialMutationLease, resolveCredentials,
resolveCredentialRefs, configureWithCredentialLease, createLinkIntent,
getPublicBaseUrl, getWebhookPublicBaseUrl, fetchImpl, scheduleMessageProcessing,
renderSlackAvatar: options.renderSlackAvatar, slackManaged: options.slackManaged,
});
const { storeGitHubApp, githubRegistration, refreshGitHubRepositories, githubWizard, unregisterGitHubCloudIngress, saveGitHubSetupProgress } = createGitHubAppSetup(db, {
endpointRecord, withCredentialMutationLease, normalizedCredentials, verifyCredentials, nativeBotIdentityMatches,
assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict,
persistCredentials, resolveCredentials, fetchImpl, get, getPublicBaseUrl, getWebhookPublicBaseUrl,
prepareProviderInventory, reconcileProviderResourceRows, listResources, replaceResources, update, configure, test, handleWebhook,
get githubWizardOrigin() { return options.githubWizardOrigin; },
});
const { prepareInboundEnvelope } = createInboundEnvelope(db, { runtimeFor, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages });
const { admitInboundMessage } = createInboundAdmission(db, { runtimeGeneration, credentialFingerprint, leasedChatProvider, renewDiscordGatewayOwnershipForMessageAdmission, ensureResource, DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS });
const { admitMessageResource } = createMessageResourceAdmission(db, { ensureResource });
const messageLifecycle = createChatMessageLifecycle(db, {
DELIVERY_PROCESSING_STALE_MS, ORPHAN_FOLLOW_UP_GRACE_MS, ORPHAN_FOLLOW_UP_MAX_ATTEMPTS,
INGRESS_REORDER_WINDOW_MS, issuesSvc, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed,
redactError, runtimeCallbackEndpoint, sameTeamsThreadIdentity, durableExternalThreadIdentity,
recordCurrentMicrosoftTeamsRoute, conversationDrainKey, scheduleConversationDrain, scheduledConversationDrains,
drainConversationDeliveries, ensureDiscordGatewayRuntimeIsCurrent,
get deferWebhookProcessing() { return options.deferWebhookProcessing; },
});
const { recordLifecycleDelivery, processLifecycleDelivery, conversationForThread } = messageLifecycle;
const conversationDeliveryQueue = createChatConversationDeliveryQueue(db, {
DELIVERY_DRAIN_LIMIT, DELIVERY_LEASE_TTL_MS, discordRootThreadPending, reconcileDiscordRootThread,
durableExternalThreadIdentity, liveInboundMessages, processInboundWakeup, processLifecycleDelivery, processMessage,
redactError, runtimeContexts, runtimeFor, settleRejectedInboundWakeups, sanitizeFilename,
githubManualMessages, githubAutomaticMessages, githubIssueMessages, conversationDrainKey,
get conversationLeaseRenewalIntervalMs() { return options.conversationLeaseRenewalIntervalMs; },
get renewConversationDeliveryLease() { return options.renewConversationDeliveryLease; },
});
const { nextInboundDeliveryAt } = conversationDeliveryQueue;
const { currentTaskControlPayload, hasCommittedTaskControlCompletion, commitTaskControlCompletion } = createChatPublicationTaskControl(db);
const publicationTransport = createChatPublicationTransport(db, {
endpointRecord, fetchImpl, getTaskBaseUrl, publicationFiles, receiptReactionCompletionRunId,
runIdFromMilestonePublication, runtime, runtimeFor, runtimeGeneration, CAPABILITIES,
voice, voiceStore, scheduleMessageProcessing,
});
const { postSafePublication } = publicationTransport;
const { claimPublicationTransportAuthorization } = createChatPublicationAuthorization(db, {
authorizeInboundWakePublication, authorizeRetainedChatSourcePublication, fetchImpl,
githubCurrentOmissionPreparedText, hasCommittedTaskControlCompletion, lockCurrentPrincipalAuthorization,
nonDirectDestinationAllowed, resolveCredentials, runtimeCallbackEndpoint, FailedChatRetryPublicationReadError,
});
const { finalizePublicationFailure } = createChatPublicationFailures(db, {
FailedChatRetryPublicationReadError, MAX_ERROR_TEXT, getTaskBaseUrl, invalidateRuntime, runtimeCallbackEndpoint,
});
const { ensureDurablePublicationTransport } = createChatPublicationPreparation(db, {
runPublicationToReplace,
get publicationTransportPreparationBarrier() { return options.publicationTransportPreparationBarrier; },
});
const publicationWorker = createChatPublicationWorker(db, {
CAPABILITIES, authorizeRetainedChatSourcePublication, claimPublicationTransportAuthorization,
closeProgressToReplace, commitTaskControlCompletion, currentTaskControlPayload, endpointRecord,
ensureDurablePublicationTransport, finalizePublicationFailure, hasCommittedTaskControlCompletion,
hasTelegramStopSubscription, inboundWakePublicationToReplace, interactionPromptPublicationToReplace,
interactionResolutionPublicationToReplace, nonDirectDestinationAllowed, postSafePublication,
processReceiptReaction, progressSupersededByFastResponse, recordSlackFileUploadReceipt, redactError,
reserveTelegramPublicationDraft, runOwnershipMilestoneSupersessionReason, runProgressSupersededByPublishedInteraction,
runPublicationToReplace, runtimeContextForRecord, scheduleMessageProcessing, settleGitHubAutomaticPublication,
stageSlackFilePublicationSuccess, taskStatusPublicationToReplace, tryProcessTeamsFilePublication,
withCredentialMutationLease, receiptReactionCompletionRunId,
});
const { processSelectedPublication } = publicationWorker;
const { processPendingInteractionWakeups } = createChatInteractionContinuations(db, {
redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS, get heartbeat() { return options.heartbeat; },
});
const { handleAction, handleModalSubmit, reconcileTerminalConfirmationActions } = createChatInteractionCallbacks(db, {
CAPABILITIES, MAX_ERROR_TEXT, actionThreadMatchesConversation, nonDirectDestinationAllowed, sameTeamsThreadIdentity,
ensurePrincipal, endpointRecord, runtimeCallbackRecord, runtimeCallbackEndpoint,
requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange,
recordCurrentMicrosoftTeamsRoute, lockTelegramCallbackNoticeAuthority, stageProviderEffect,
scheduleProviderEffect, invalidateRuntime, redactError, persistence, questionResponses,
scheduleMessageProcessing, processPendingPublications,
get confirmationResolutionPersistBarrier() { return options.confirmationResolutionPersistBarrier; },
get questionFormOpenAuthorizationBarrier() { return options.questionFormOpenAuthorizationBarrier; },
get questionResolutionPersistBarrier() { return options.questionResolutionPersistBarrier; },
});
const publicationQueue = createChatPublicationQueue(db, {
PUBLICATION_ENDPOINT_CONCURRENCY, enqueueInboundWakeupPublications, processPendingInteractionWakeups,
reconcileTerminalConfirmationActions, processSelectedPublication, publicationEndpointTasks,
teamsFileRetryAfter, redactError, isShuttingDown: () => shuttingDown, wakePublications: () => wakePublications(),
});
const { publicationEligibility } = publicationQueue;
const { unregisterSlackTaskAuthority } = createSlackTaskAuthorityRegistration(db, { authorizeInboundWakeup, resolveCredentials });
const { replayPublication, resolvePublication, resolveProviderEffect } = createChatPublicationResolution(db, { endpointRecord, withCredentialMutationLease, teamsTransferProtocol, commitTaskControlCompletion, processPendingPublications, processProviderEffect });
const { prepareTeamsBoardSend } = createTeamsBoardSendPreparation(db, { endpointRecord, runtimeFor });
const { getPublicationBatchStatus, publishComment, publishBoardMessage } = createChatPublicationRequests(db, { endpointRecord, stageTeamsBoardFileIntent, processPendingSlackBoardMessages, processPendingPublications, prepareTeamsBoardSend });
return {
voice,
authorizeFastResponse,
slackRegistration,
sendSlackTestMessage,
saveGitHubSetupProgress,
githubWizard,
startGitHubRegistration: githubRegistration.start,
completeGitHubRegistration: githubRegistration.complete,
storeGitHubApp,
refreshGitHubRepositories,
runtime,
list,
get,
create,
update,
generateSetupSecret,
configure,
inspectPhoton,
test,
finishSlackSetup: (endpointId: string, userId: string) => test(endpointId, { optionalSlackTestForUser: userId }),
setupTestStatus,
handleWebhook,
listResources,
listGitHubRepositories,
toggleAllGitHubRepositories,
replaceResources,
listPrincipals,
createLinkIntent,
previewIdentityLink,
requestIdentityAccess,
confirmIdentityLink,
revokeLink,
listConversations,
listActivity,
listActivityPage,
replayDelivery,
replayPublication,
resolveAction,
resolvePublication,
publishComment,
getPublicationBatchStatus,
publishBoardMessage,
reconcileProviderRuntimes,
processPendingPublications,
enqueueInboundWakeupPublications,
schedulePendingPublications,
processPendingDeliveries,
processQueuedDeliveries: () => processPendingDeliveries(25, undefined, false),
scheduleQueuedPublications: () => processPendingPublications(25, { waitForCompletion: false, maintenance: false }),
processPendingChatMaintenance,
processPendingTelegramActions,
processTelegramSubscriptionDiscovery,
scheduleTelegramSubscriptionDiscovery,
hasPendingTelegramSubscriptionDiscovery,
processPendingSlackBoardMessages,
processPendingSlackTaskStarts,
processPendingVerificationMessages: slackRegistration.processPendingVerificationMessages,
nextVerificationMessageAt: slackRegistration.nextVerificationMessageAt,
nextChatActionAt: (kind: ScheduledChatAction) => nextChatActionAt(db, kind),
processPublicationMaintenance,
nextInboundDeliveryAt,
nextPublicationAt,
processPendingGitHubReviewPublications: githubReviews.processPending,
nextGitHubReviewPublicationAt: githubReviews.nextPendingAt,
nextSlackReceiptAt,
onPublicationsSettled: (wake: () => void) => { wakePublications = wake; },
prepareFailedChatRunRetry,
authorizeFailedChatRunRetry,
processFailedChatRunRetry,
processFailedChatRunRetries,
processPendingGitHubWebhookIngress,
processFailedGitHubWebhookDeliveries,
processPendingProviderEffects,
processPendingReceiptReactions,
processPendingSlackFileUploadReceipts,
processPendingSlackSessionStops,
processPendingSlackSessionSyncs,
getIssueBinding,
shutdown: async () => {
shuttingDown = true;
await Promise.allSettled(pendingFailedRetries());
unregisterFailedRetryAuthority();
unregisterSlackTaskAuthority();
voice.unregisterAgentTools();
unregisterGitHubCloudIngress();
unregisterCommittedResponseAuthority();
await Promise.allSettled([...publicationEndpointTasks.values()]);
await Promise.allSettled([...backgroundMessageTasks]);
scheduledConversationDrains.clear();
liveInboundMessages.clear();
const ownedDiscordGateways = [...discordGatewayOwnerships.values()];
for (const ownership of ownedDiscordGateways) {
ownership.stopping = true;
if (ownership.renewTimer) {
clearInterval(ownership.renewTimer);
ownership.renewTimer = null;
}
}
discordGatewayOwnerships.clear();
await Promise.allSettled(
ownedDiscordGateways
.map((ownership) => ownership.renewal)
.filter((renewal): renewal is Promise<void> => renewal !== null),
);
await Promise.allSettled([...discordGatewayStopTasks]);
// Stop every provider runtime before releasing leader rows. This gives a
// standby an immediate takeover path without overlapping Gateway sockets.
await runtime.shutdown();
await slackRegistration.close();
await Promise.all(
ownedDiscordGateways.map((ownership) =>
releaseDiscordGatewayLeaseRow(ownership),
),
);
},
};
}
export type ChatChannelService = ReturnType<typeof chatChannelService>;