mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
refactor(server): split chat channel workflows into provider and shared modules (#16080)
Move provider operations into channel folders and shared chat work into named interaction, setup, admission, delivery and publication workflows. Consolidate reviewed PRs #16085, #16089, #16096 and #16105. Preserve upstream execution-default and recovery behavior. The main service is 7,917 lines. All 55 current-head CI checks pass, with two optional Storybook skips. Greptile is 5/5 with no findings. All 1,294 current chat integration cases, repository-wide typecheck and build pass locally. Broad local tool-route timing/socket failures are recorded in the PR. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
60bc0d5fee
commit
eb06c9fedd
43 files changed
+13705
-11475
No files matched your search
@@ -0,0 +1,159 @@
|
||||
# Chat channel workflow refactor
|
||||
|
||||
The user authorized six hours of sustained work starting at 2026-10-13 02:14 UTC,
|
||||
with a target of fewer than 8,000 lines in `server/src/services/chat-channels.ts`.
|
||||
The starting commit is `7eaa9cfa9ce04398a5eb0211a86aacaaa01959dd`: 19,153 lines,
|
||||
762,442 bytes. At kickoff, the preceding ten PRs were reviewed and green, but not yet merged.
|
||||
The user authorizes publishing and merging reviewed changes to paperclipai/paperclip.
|
||||
|
||||
The previous plan prioritized too many small helper cuts. This plan prioritizes
|
||||
complete workflows, provider ownership, and meaningful reductions per PR. Shared
|
||||
transactions and admission policy belong in shared workflow modules. Provider
|
||||
transport, envelope checks, setup, and native interactions belong in provider
|
||||
folders. Do not put shared code in one provider's folder or use a mutable context
|
||||
bag to move the entire service unchanged.
|
||||
|
||||
| Priority | Responsibility and destination | Estimated net reduction | Contract and state | Risk and verification |
|
||||
| --- | --- | ---: | --- | --- |
|
||||
| 1 | Native interaction callbacks and continuation: `interactions/`, `telegram/confirmation-actions.ts`, `discord/question-forms.ts` | 2,900 | Current actor/runtime/destination readers; durable token and interaction transactions; deferred wakeups. | Preserve token replay, audit order, provider acknowledgement, post-commit recovery, and live fault barriers. Existing public service, HTTP and real-database interaction tests before and after. |
|
||||
| 2 | Endpoint configuration and provider setup: provider `setup.ts`, shared credential persistence/configuration | 1,600–2,200 | Credential lease and current endpoint; provider registration outcome; stable native identity. | Preserve fail-closed persistence, webhook/receiver ownership, partial setup cleanup, and credential rotation races. Existing setup and runtime tests. |
|
||||
| 3 | Message admission and provider policy: shared intake workflow, GitHub message policy, provider envelope projections | 2,400–3,000 | Delivery claim and conversation/task admission; explicit provider policy results; per-service message provenance. | Preserve row locks, deduplication, author authority, task generations, attachments, and GitHub automatic/explicit admission. Existing real-database ingress and replay tests. |
|
||||
| 4 | Durable delivery and lifecycle: shared queue, provider lifecycle inputs | 1,700–2,000 | Conversation leases, replay decode, ordered drains, mutation reconciliation; shared live receipt state. | Preserve stale lease exclusion, event order, deletion/edit races, deferred startup and shutdown. Existing concurrency and lifecycle tests. |
|
||||
| 5 | Outbound publication: provider delivery modules and shared publication worker | 2,700–3,400 | Current-source authorization; exact transport parts; attempts, unknown outcomes, completion receipts and cleanup. | Preserve no duplicate sends, ambiguous acceptance, credential fencing, file recovery, concurrency and current task-control completion. Existing assembled publication and file tests. |
|
||||
| 6, if needed | Operator recovery and endpoint projections | 800–1,200 | Explicit replay/resolution and company-scoped board calls. | Preserve retry audit, current policy and authorization. Existing operator-resolution integration tests. |
|
||||
|
||||
The estimates are not a PR quota. Inspect every full execution path before moving
|
||||
it. Increase size only where a cohesive responsibility has a narrow contract.
|
||||
After every reviewed slice, remap the remaining file and report the measured size.
|
||||
|
||||
For each slice: run a behavioral baseline before editing production; retain
|
||||
existing bodies and SQL; compare normalized AST and statement order; run server
|
||||
typecheck, module boundaries and public-path tests. Review all intentional callback
|
||||
and dispatch substitutions. Preserve live option reads with getters or callbacks.
|
||||
Keep per-service state per service and shared cache/lease ownership shared.
|
||||
|
||||
Open one PR per cohesive extraction with the full repository template. Verify its
|
||||
current-head review and all CI gates before opening the next extraction. Build each
|
||||
on its immediate predecessor. Run repository-wide typecheck, tests and build once
|
||||
at the final handoff; investigate and disclose any failures or unreached coverage.
|
||||
The earlier batch recorded local timing/host cleanup failures; green CI and passing
|
||||
focused retries do not erase that evidence. No timeout or assertion weakening.
|
||||
|
||||
Public application fixtures prove assembled service/HTTP/database behavior. They
|
||||
do not prove live third-party execution. Use only owned fixture data and processes.
|
||||
|
||||
## First extraction
|
||||
|
||||
The native interaction workflow reduces the service from 19,153 to 16,117 lines
|
||||
and from 762,442 to 656,146 bytes. Telegram owns confirmation resolution and
|
||||
settlement. Discord owns correction draft retention and accepted-answer cleanup.
|
||||
Shared callbacks retain token/source/actor checks and atomic question resolution.
|
||||
Shared continuation work retains durable wakeup recovery. SDK state persistence
|
||||
has a direct module import and retains the public compatibility export.
|
||||
|
||||
The same 62 public-path interaction cases pass before and after. Seven supporting
|
||||
suites pass all 70 cases. Server typecheck and module boundaries pass. AST checks
|
||||
cover nine moved declarations, three provider bodies, and retained service order.
|
||||
Fault barriers remain live getters. No assertion or timeout changed.
|
||||
|
||||
## Second extraction
|
||||
|
||||
Endpoint configuration and provider setup reduce the service to 14,241 lines and
|
||||
580,227 bytes. Shared modules own endpoint records, credential identity/persistence,
|
||||
and the credential-leased configuration state machine. Channel folders own Telegram
|
||||
registration, GitHub repair, Speko tool installation, Photon configuration and
|
||||
inspection, and Slack setup projection. Registration dependencies remain live.
|
||||
|
||||
All 1,278 chat cases passed on the first PR before this move. The final setup wiring
|
||||
passes 268 selected integration cases, server typecheck, and module boundaries.
|
||||
AST checks cover 25 declarations and nine provider bodies. One final test attempt
|
||||
could not start PostgreSQL because this Mac exhausted IPC slots; after reclaiming
|
||||
one unused zero-attachment segment with a dead creator, the same suite passed.
|
||||
|
||||
## Third extraction
|
||||
|
||||
The publication workflow reduces the service to 11,198 lines and 462,108 bytes.
|
||||
Authorization, durable preparation, transport, failures, attempt ownership, task
|
||||
control completion, and bounded endpoint scheduling now have named modules.
|
||||
Photon owns prompt and reply delivery. Discord and Telegram own durable Markdown
|
||||
attachment preparation. Shared native splitting remains shared across providers.
|
||||
|
||||
The same 301 public-path publication cases pass before and after. Eight supporting
|
||||
suites pass all 185 cases. Server typecheck, module boundaries, and AST checks pass.
|
||||
The AST audit covers 26 declarations, three provider bodies, and retained service
|
||||
order. Shared endpoint lanes and Teams cooldowns retain their original maps;
|
||||
shutdown and publication wake listeners stay live. Early callback registration
|
||||
uses typed forwarding functions instead of reading factories before initialization.
|
||||
|
||||
## Fourth extraction
|
||||
|
||||
Ordered delivery and lifecycle reconciliation reduce the service to 9,287 lines
|
||||
and 388,734 bytes. The delivery ledger owns ordering and runtime-fence projections.
|
||||
The conversation queue owns exact leases, reconstruction and ordered replay.
|
||||
The lifecycle workflow owns edit/delete admission and reconciliation. Teams owns
|
||||
authenticated route recording. GitHub owns restoration of durable message policy
|
||||
into the same per-service provenance maps.
|
||||
|
||||
The same 225 lifecycle, ordering, lease, replay and recovery cases pass before and
|
||||
after. Typecheck and module boundaries pass. AST checks cover 26 declarations, the
|
||||
GitHub restoration body, and retained service order. Live maps and lease hooks retain
|
||||
their original lifetime. The publication predecessor's packaging retry passed unchanged. This fourth
|
||||
slice is published as PR #16096.
|
||||
|
||||
|
||||
## Fifth extraction
|
||||
|
||||
Inbound authority, envelope preparation, durable admission and resource admission
|
||||
reduce the service to 7,899 lines and 335,607 bytes. The shared admission module
|
||||
keeps endpoint locking, runtime and credential fences, gateway lease renewal,
|
||||
deduplication and delivery insertion in one transaction. Teams, Telegram and
|
||||
Photon own their destination rules and use that same transaction. The main
|
||||
message path now calls preparation, durable admission and resource admission
|
||||
before the retained task workflow.
|
||||
|
||||
The same 278 selected public-path cases pass before and after the first move.
|
||||
All 1,278 existing chat integration cases pass on the final provider dispatch
|
||||
in 309.47 seconds. Server typecheck and module boundaries pass. One full-suite attempt
|
||||
could not allocate PostgreSQL shared memory before any test ran. Only a verified
|
||||
unused 56-byte segment with a dead creator and zero attachments was reclaimed
|
||||
before retrying. No active database was stopped.
|
||||
|
||||
AST checks cover 21 moved declarations, envelope and resource phase bodies,
|
||||
three provider destination bodies, and the complete admission transaction after
|
||||
restoring the explicit provider calls. The service still owns the per-call
|
||||
gateway ownership handoff. The envelope retains its per-call recovery map and
|
||||
the original authenticated runtime. No assertion or timeout changed.
|
||||
|
||||
## Stack landing
|
||||
|
||||
The preceding ten PRs reached master through #16058. The five workflow PRs are
|
||||
#16080, #16085, #16089, #16096 and #16105. Each extraction has individual behavioral
|
||||
evidence and a 5/5 review. The four child PRs merged into their predecessors after
|
||||
each combined revision passed its own checks and review. Every consolidation
|
||||
preserved the tested tip tree exactly.
|
||||
|
||||
The final master integration leaves the service at 7,917 lines and 337,195 bytes.
|
||||
The 18-line increase over the extraction tip preserves upstream execution-default
|
||||
selection and authorization. Endpoint serialization and credential-leased updates
|
||||
retain that policy in the extracted endpoint module. Task creation and failed-run
|
||||
recovery retain their upstream behavior. All 107 extracted declarations match
|
||||
current master after reversing the documented binding substitutions. The endpoint,
|
||||
task-creation and recovery functions also pass exact normalized AST comparisons.
|
||||
|
||||
Repository-wide typecheck and build pass on this integration. The first full local
|
||||
test command passed 18,391 server tests and skipped 105, but one unchanged tool-access
|
||||
test hit its 15-second timeout. That case passed alone. The UI and CLI groups passed
|
||||
8,706 tests. Shared and adapter groups passed 4,751 tests and skipped 20. A database
|
||||
group could not initialize one PostgreSQL fixture. This Mac has a 32-segment shared
|
||||
memory limit and concurrent fixture databases. The unchanged database cases are
|
||||
being rerun one file at a time before the remaining chat and serialized route lanes.
|
||||
Keep the original failed runs in the verification record. No assertion or timeout
|
||||
was changed. The root PR remains open for fresh review and CI against master.
|
||||
|
||||
The service now has 67 direct functions. The largest retained paths are message
|
||||
task handling (1,449 lines), webhook dispatch (685), operator action resolution
|
||||
(488), and publication resolution (343). Provider setup, transport preparation,
|
||||
native interactions and destination admission live in their channel folders.
|
||||
Shared admission, delivery, lifecycle and publication transactions have named
|
||||
workflow modules. The retained message path calls those phases and keeps task
|
||||
mutation and per-call ownership together.
|
||||
+209
-11473
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,968 @@
|
||||
import { preparePhotonConfiguration, createPhotonConfiguration } from "./imessage-photon/setup.js";
|
||||
import { slackRuntimeConfig } from "./slack/setup.js";
|
||||
import { createTelegramSetup } from "./telegram/setup.js";
|
||||
import { createGitHubSetup } from "./github/setup.js";
|
||||
import { createSpekoSetup } from "./speko/setup.js";
|
||||
import { createGitHubAppSetup } from "./github/app-setup.js";
|
||||
|
||||
import type { VerifiedProviderIdentity } from "./runtime-types.js";
|
||||
|
||||
import { isProvisionedSlackSetup } from "@paperclipai/shared";
|
||||
|
||||
import { slackCallbackMatchesPublicUrl } from "./slack/callback-verification.js";
|
||||
|
||||
import { createDiscordCommands } from "./discord/commands.js";
|
||||
|
||||
import { createTelegramMaintenance } from "./telegram/maintenance.js";
|
||||
import type { EndpointRow, RuntimeContext, InternalSetupState } from "./runtime-types.js";
|
||||
|
||||
import { createChatResourceReconciliation } from "./resource-reconciliation.js";
|
||||
|
||||
import { notifyChatDeliveryWork, notifyChatEndpointWork } from "../chat-work-notifications.js";
|
||||
|
||||
import type { ChatEndpointRuntime } from "../voice/voice-runtime.js";
|
||||
|
||||
import { chatCredentialMutationLease, type CredentialMutationLeaseGuard } from "../chat-credential-mutation-lease.js";
|
||||
|
||||
import { slackChatRegistrationService } from "../chat-slack-registration.js";
|
||||
|
||||
import {
|
||||
createHash,
|
||||
} from "node:crypto";
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
inArray,
|
||||
isNull,
|
||||
ne,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatConversations,
|
||||
chatDeliveries,
|
||||
chatEndpoints,
|
||||
toolApplications,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatEndpoint,
|
||||
ChatEndpointSetupState,
|
||||
ChatProvider,
|
||||
ConfigureChatEndpointInput,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
MAX_ATTACHMENT_BYTES,
|
||||
} from "../../attachment-types.js";
|
||||
|
||||
import {
|
||||
badRequest,
|
||||
conflict,
|
||||
notFound,
|
||||
unprocessable,
|
||||
} from "../../errors.js";
|
||||
|
||||
import {
|
||||
type ResolvedChatSdkProviderConfig,
|
||||
} from "../chat-sdk-runtime.js";
|
||||
import {
|
||||
logActivity,
|
||||
} from "../activity-log.js";
|
||||
|
||||
import type { EndpointRecord } from "./runtime-types.js";
|
||||
import type { createChatCredentials } from "./credentials.js";
|
||||
type ChatConfigurationOptions = Pick<ReturnType<typeof createChatCredentials>, "normalizedCredentials" | "verifyCredentials" | "nativeBotIdentityMatches" | "legacyGitHubLabelsMatchVerifiedCredentials" | "assertNativeBotIdentityAvailable" | "isNativeBotIdentityUniqueViolation" | "nativeBotIdentityConflict" | "persistCredentials" | "clearCredentials" | "resolveCredentials" | "prepareProviderInventory"> & {
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
get: (endpointId: string) => Promise<ChatEndpoint>;
|
||||
withCredentialMutationLease: ReturnType<typeof chatCredentialMutationLease>;
|
||||
getWebhookPublicBaseUrl: () => string | null;
|
||||
invalidateRuntime: (endpointId: string) => Promise<boolean>;
|
||||
reconcileProviderResourceRows: ReturnType<typeof createChatResourceReconciliation>["reconcileProviderResourceRows"];
|
||||
reconcileDiscordCommands: ReturnType<typeof createDiscordCommands>["reconcileDiscordCommands"];
|
||||
runtimeFor: (endpoint: EndpointRow, input?: { requireDiscordOwnership?: boolean; waitForDiscordOwnership?: boolean }) => Promise<ChatEndpointRuntime>;
|
||||
runtimeContextForRecord: (record: EndpointRecord) => RuntimeContext;
|
||||
stageTelegramMaintenance: ReturnType<typeof createTelegramMaintenance>["stageTelegramMaintenance"];
|
||||
processTelegramMaintenance: ReturnType<typeof createTelegramMaintenance>["processTelegramMaintenance"];
|
||||
recordTelegramStopSubscription: ReturnType<typeof createTelegramMaintenance>["recordTelegramStopSubscription"];
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
redactError: (error: unknown) => string;
|
||||
runtimeGeneration: (setup: ChatEndpointSetupState) => number;
|
||||
leasedChatProvider: (provider: string) => boolean;
|
||||
PROVIDER_LABELS: Record<ChatProvider, string>;
|
||||
SUPPLIED_CREDENTIAL_KEYS: Record<ChatProvider, readonly string[]>;
|
||||
PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number;
|
||||
isSupportedTelegramWebhookBaseUrl: (url: string | null) => boolean;
|
||||
slackRegistration: Pick<ReturnType<typeof slackChatRegistrationService>, "cleanup">;
|
||||
githubWizard: Pick<ReturnType<typeof createGitHubAppSetup>["githubWizard"], "repairCloud">;
|
||||
nativeBotIdentityClaimBarrier?: () => Promise<void>;
|
||||
allowLocalVoiceBoard?: boolean;
|
||||
};
|
||||
export function createChatConfiguration(db: Db, options: ChatConfigurationOptions) {
|
||||
const { endpointRecord, get, withCredentialMutationLease, getWebhookPublicBaseUrl, invalidateRuntime, nativeBotIdentityMatches, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, legacyGitHubLabelsMatchVerifiedCredentials, normalizedCredentials, verifyCredentials, persistCredentials, clearCredentials, resolveCredentials, prepareProviderInventory, reconcileProviderResourceRows, reconcileDiscordCommands, runtimeFor, runtimeContextForRecord, stageTelegramMaintenance, processTelegramMaintenance, recordTelegramStopSubscription, fetchImpl, redactError, runtimeGeneration, leasedChatProvider, PROVIDER_LABELS, SUPPLIED_CREDENTIAL_KEYS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, isSupportedTelegramWebhookBaseUrl } = options;
|
||||
const { persistPhotonConfiguration } = createPhotonConfiguration(db, { invalidateRuntime });
|
||||
const { registerTelegramWebhook } = createTelegramSetup(db, { fetchImpl, getWebhookPublicBaseUrl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, runtimeContextForRecord, recordTelegramStopSubscription, stageTelegramMaintenance, processTelegramMaintenance });
|
||||
const { repairGitHubCloudSetup, repairGitHubWebhookSetup } = createGitHubSetup(db, { fetchImpl, getWebhookPublicBaseUrl, get githubWizard() { return options.githubWizard; } });
|
||||
const { installSpekoVoiceTools, configureSpekoTools } = createSpekoSetup(db, { fetchImpl, getWebhookPublicBaseUrl, get allowLocalVoiceBoard() { return options.allowLocalVoiceBoard; } });
|
||||
|
||||
|
||||
|
||||
async function configure(
|
||||
endpointId: string,
|
||||
input: ConfigureChatEndpointInput,
|
||||
actorUserId?: string | null,
|
||||
) {
|
||||
const record = await endpointRecord(endpointId);
|
||||
if (!record) throw notFound("Chat endpoint not found");
|
||||
if (record.endpoint.provider === "agentmail") throw badRequest("Use the email inbox API for AgentMail");
|
||||
if (input.photon && record.endpoint.provider !== "imessage-photon") throw badRequest("Photon configuration is only valid for iMessage Photon");
|
||||
const suppliedCredentialKeys = Object.keys(input.credentials ?? {});
|
||||
if (suppliedCredentialKeys.length > 0) {
|
||||
const credentialAction =
|
||||
input.action === "configure" || input.action === "reconnect";
|
||||
const allowedKeys = new Set(
|
||||
credentialAction
|
||||
? SUPPLIED_CREDENTIAL_KEYS[record.endpoint.provider]
|
||||
: [],
|
||||
);
|
||||
const unsupportedKeys = suppliedCredentialKeys.filter(
|
||||
(key) => !allowedKeys.has(key),
|
||||
);
|
||||
if (unsupportedKeys.length > 0) {
|
||||
throw unprocessable(
|
||||
`Unsupported ${PROVIDER_LABELS[record.endpoint.provider]} credential fields for ${input.action}: ${unsupportedKeys.join(", ")}`,
|
||||
{
|
||||
code: "chat_endpoint_credentials_invalid",
|
||||
provider: record.endpoint.provider,
|
||||
action: input.action,
|
||||
unsupportedKeys,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
return withCredentialMutationLease(record.endpoint, (credentialLease) =>
|
||||
configureWithCredentialLease(
|
||||
endpointId,
|
||||
input,
|
||||
credentialLease,
|
||||
actorUserId,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async function configureWithCredentialLease(
|
||||
endpointId: string,
|
||||
input: ConfigureChatEndpointInput,
|
||||
credentialLease: CredentialMutationLeaseGuard,
|
||||
actorUserId?: string | null,
|
||||
) {
|
||||
const record = await endpointRecord(endpointId);
|
||||
if (!record) throw notFound("Chat endpoint not found");
|
||||
const endpoint = record.endpoint;
|
||||
if (input.action === "pause") {
|
||||
if (endpoint.status !== "active") {
|
||||
throw conflict("Only an active chat connection can be paused", {
|
||||
code: "chat_endpoint_not_active",
|
||||
});
|
||||
}
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const pausedAt = new Date();
|
||||
const current = await tx
|
||||
.select({ setup: chatEndpoints.setup, status: chatEndpoints.status })
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current || current.status !== "active") {
|
||||
throw conflict("Only an active chat connection can be paused", {
|
||||
code: "chat_endpoint_not_active",
|
||||
});
|
||||
}
|
||||
await notifyChatEndpointWork(tx);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "paused",
|
||||
setup: {
|
||||
...current.setup,
|
||||
runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1),
|
||||
} as InternalSetupState,
|
||||
updatedAt: pausedAt,
|
||||
})
|
||||
.where(eq(chatEndpoints.id, endpoint.id));
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({ status: "disabled", enabled: false, updatedAt: new Date() })
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await notifyChatDeliveryWork(tx);
|
||||
await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
nextAttemptAt: null,
|
||||
redactedError: "Connection was paused before processing",
|
||||
processedAt: pausedAt,
|
||||
updatedAt: pausedAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
endpoint.provider === "imessage-photon" ? sql`false` : undefined,
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
inArray(chatDeliveries.state, ["received", "retry"]),
|
||||
),
|
||||
);
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
await invalidateRuntime(endpoint.id);
|
||||
await logActivity(db, {
|
||||
companyId: endpoint.companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action: "chat_endpoint.paused",
|
||||
entityType: "tool_connection",
|
||||
entityId: endpoint.connectionId,
|
||||
details: { endpointId: endpoint.id },
|
||||
});
|
||||
return get(endpoint.id);
|
||||
}
|
||||
if (input.action === "resume") {
|
||||
if (endpoint.status !== "paused" || endpoint.setup.step !== "complete") {
|
||||
throw conflict(
|
||||
"Only a previously active chat connection can be resumed",
|
||||
{
|
||||
code: "chat_endpoint_not_resumable",
|
||||
},
|
||||
);
|
||||
}
|
||||
let credentials = await resolveCredentials(endpoint).catch(() => {
|
||||
throw conflict("Reconnect the provider credentials before resuming", {
|
||||
code: "chat_endpoint_credentials_missing",
|
||||
});
|
||||
});
|
||||
const identity = await verifyCredentials(endpoint.provider, credentials);
|
||||
const upgradingLegacyGitHubIdentity =
|
||||
endpoint.provider === "github" &&
|
||||
!endpoint.botExternalId &&
|
||||
legacyGitHubLabelsMatchVerifiedCredentials(endpoint, identity);
|
||||
if (
|
||||
!upgradingLegacyGitHubIdentity &&
|
||||
!nativeBotIdentityMatches(endpoint.provider, endpoint, identity)
|
||||
) {
|
||||
throw conflict(
|
||||
"The provider credentials now identify a different bot; reconnect this connection instead",
|
||||
{
|
||||
code: "chat_bot_identity_changed",
|
||||
},
|
||||
);
|
||||
}
|
||||
await assertNativeBotIdentityAvailable(endpoint, identity);
|
||||
const prepared = await prepareProviderInventory(endpoint, credentials);
|
||||
if (
|
||||
endpoint.provider === "github" &&
|
||||
prepared.credentials.installationId !== credentials.installationId
|
||||
) {
|
||||
await persistCredentials(
|
||||
endpoint,
|
||||
prepared.credentials,
|
||||
credentialLease,
|
||||
actorUserId,
|
||||
);
|
||||
}
|
||||
credentials = prepared.credentials;
|
||||
if (prepared.inventory)
|
||||
await reconcileProviderResourceRows(
|
||||
endpoint,
|
||||
prepared.inventory,
|
||||
credentialLease,
|
||||
);
|
||||
let activatedEndpoint!: EndpointRow;
|
||||
try {
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const resumedAt = new Date();
|
||||
const current = await tx
|
||||
.select()
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current || current.status !== "paused") {
|
||||
throw conflict(
|
||||
"Only a previously active chat connection can be resumed",
|
||||
{
|
||||
code: "chat_endpoint_not_resumable",
|
||||
},
|
||||
);
|
||||
}
|
||||
// Fail closed for any legacy or racing delivery that remained open
|
||||
// while this endpoint was paused. Resume must never execute traffic
|
||||
// that Paperclip acknowledged during the inactive interval.
|
||||
await notifyChatDeliveryWork(tx);
|
||||
await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
nextAttemptAt: null,
|
||||
redactedError: "Connection was paused when this event arrived",
|
||||
processedAt: resumedAt,
|
||||
updatedAt: resumedAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
endpoint.provider === "imessage-photon" ? sql`false` : undefined,
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
inArray(chatDeliveries.state, ["received", "retry"]),
|
||||
),
|
||||
);
|
||||
await notifyChatEndpointWork(tx);
|
||||
[activatedEndpoint] = await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "active",
|
||||
...(upgradingLegacyGitHubIdentity
|
||||
? { botExternalId: identity.botExternalId }
|
||||
: {}),
|
||||
healthMessage: "Connected",
|
||||
lastError: null,
|
||||
setup: {
|
||||
...current.setup,
|
||||
...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: resumedAt.toISOString() } : {}),
|
||||
runtimeGeneration: runtimeGeneration(current.setup) + (endpoint.provider === "imessage-photon" ? 0 : 1),
|
||||
} as InternalSetupState,
|
||||
updatedAt: resumedAt,
|
||||
})
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.returning();
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({
|
||||
status: "active",
|
||||
enabled: true,
|
||||
healthStatus: "healthy",
|
||||
healthMessage: "Connected",
|
||||
lastError: null,
|
||||
healthCheckedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
await invalidateRuntime(endpoint.id);
|
||||
if (activatedEndpoint.provider === "discord")
|
||||
await reconcileDiscordCommands(endpoint.id, credentialLease, true);
|
||||
await runtimeFor(activatedEndpoint, {
|
||||
requireDiscordOwnership: leasedChatProvider(activatedEndpoint.provider),
|
||||
waitForDiscordOwnership: leasedChatProvider(activatedEndpoint.provider),
|
||||
});
|
||||
} catch (error) {
|
||||
await invalidateRuntime(endpoint.id).catch(() => undefined);
|
||||
if (activatedEndpoint) {
|
||||
const failure = redactError(error);
|
||||
const failedAt = new Date();
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
await notifyChatEndpointWork(tx);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "attention",
|
||||
healthMessage: "Provider resume needs attention",
|
||||
lastError: failure,
|
||||
updatedAt: failedAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, endpoint.id),
|
||||
eq(chatEndpoints.status, "active"),
|
||||
sql`coalesce((${chatEndpoints.setup}->>'runtimeGeneration')::integer, 0) = ${runtimeGeneration(activatedEndpoint.setup)}`,
|
||||
),
|
||||
);
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({
|
||||
status: "disabled",
|
||||
enabled: false,
|
||||
healthStatus: "degraded",
|
||||
healthMessage: "Provider resume failed",
|
||||
lastError: failure,
|
||||
healthCheckedAt: failedAt,
|
||||
updatedAt: failedAt,
|
||||
})
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await notifyChatDeliveryWork(tx);
|
||||
await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
nextAttemptAt: null,
|
||||
redactedError: "Connection activation failed",
|
||||
processedAt: failedAt,
|
||||
updatedAt: failedAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
inArray(chatDeliveries.state, ["received", "retry"]),
|
||||
),
|
||||
);
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
await logActivity(db, {
|
||||
companyId: endpoint.companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action: "chat_endpoint.resumed",
|
||||
entityType: "tool_connection",
|
||||
entityId: endpoint.connectionId,
|
||||
details: { endpointId: endpoint.id },
|
||||
});
|
||||
return get(endpoint.id);
|
||||
}
|
||||
if (input.action === "remove") {
|
||||
if (endpoint.status === "archived") {
|
||||
if (endpoint.provider === "slack") await options.slackRegistration.cleanup(endpoint.id, credentialLease);
|
||||
// Archival is the durable ingress fence and intentionally commits
|
||||
// before secret-store cleanup. If that cleanup failed, a repeated
|
||||
// remove is the recovery operation; once refs are empty, retain the
|
||||
// normal already-removed conflict contract.
|
||||
if (record.credentialSecretRefs.length > 0) {
|
||||
await invalidateRuntime(endpoint.id).catch(() => undefined);
|
||||
if (endpoint.provider === "telegram") {
|
||||
await credentialLease.assertOwned();
|
||||
const action = await stageTelegramMaintenance(
|
||||
endpoint,
|
||||
"remove_endpoint",
|
||||
);
|
||||
if (action)
|
||||
await processTelegramMaintenance(
|
||||
action.id,
|
||||
true,
|
||||
credentialLease,
|
||||
);
|
||||
} else {
|
||||
await clearCredentials(endpoint, credentialLease);
|
||||
}
|
||||
return get(endpoint.id);
|
||||
}
|
||||
throw conflict("This chat connection has already been removed", {
|
||||
code: "chat_endpoint_already_removed",
|
||||
});
|
||||
}
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const current = await tx
|
||||
.select({ setup: chatEndpoints.setup, status: chatEndpoints.status })
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current || current.status === "archived") {
|
||||
throw conflict("This chat connection has already been removed", {
|
||||
code: "chat_endpoint_already_removed",
|
||||
});
|
||||
}
|
||||
await notifyChatEndpointWork(tx);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "archived",
|
||||
setup: {
|
||||
...current.setup,
|
||||
runtimeGeneration: runtimeGeneration(current.setup) + 1,
|
||||
} as InternalSetupState,
|
||||
archivedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatEndpoints.id, endpoint.id));
|
||||
await tx
|
||||
.update(chatConversations)
|
||||
.set({ state: "endpoint_removed", updatedAt: new Date() })
|
||||
.where(eq(chatConversations.endpointId, endpoint.id));
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({ status: "archived", enabled: false, updatedAt: new Date() })
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
const otherConnection = await tx
|
||||
.select({ id: toolConnections.id })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.applicationId, record.applicationId),
|
||||
ne(toolConnections.id, endpoint.connectionId),
|
||||
ne(toolConnections.status, "archived"),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!otherConnection) {
|
||||
await tx
|
||||
.update(toolApplications)
|
||||
.set({
|
||||
status: "archived",
|
||||
archivedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(toolApplications.id, record.applicationId));
|
||||
}
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
if (endpoint.provider === "slack") await options.slackRegistration.cleanup(endpoint.id, credentialLease);
|
||||
await invalidateRuntime(endpoint.id).catch(() => undefined);
|
||||
if (endpoint.provider === "telegram") {
|
||||
await credentialLease.assertOwned();
|
||||
const action = await stageTelegramMaintenance(
|
||||
endpoint,
|
||||
"remove_endpoint",
|
||||
);
|
||||
if (action) {
|
||||
await processTelegramMaintenance(action.id, true, credentialLease);
|
||||
}
|
||||
} else {
|
||||
await clearCredentials(endpoint, credentialLease);
|
||||
}
|
||||
return get(endpoint.id);
|
||||
}
|
||||
if (
|
||||
input.action !== "configure" &&
|
||||
input.action !== "reconnect" &&
|
||||
input.action !== "verify"
|
||||
) {
|
||||
throw unprocessable("Unsupported chat endpoint setup action");
|
||||
}
|
||||
if (!getWebhookPublicBaseUrl() && endpoint.provider !== "discord" && endpoint.provider !== "imessage-photon" && !(endpoint.provider === "github" && endpoint.setup.github?.cloudRegistrationId)) {
|
||||
throw unprocessable(
|
||||
`A public HTTPS Paperclip URL is required before connecting ${PROVIDER_LABELS[endpoint.provider]}`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
endpoint.provider === "telegram" &&
|
||||
(input.action === "configure" || input.action === "reconnect") &&
|
||||
!isSupportedTelegramWebhookBaseUrl(getWebhookPublicBaseUrl())
|
||||
) {
|
||||
throw unprocessable(
|
||||
"Telegram webhooks require PAPERCLIP_CHAT_WEBHOOK_PUBLIC_URL to use HTTPS on port 443, 80, 88, or 8443",
|
||||
{
|
||||
code: "chat_telegram_webhook_url_unsupported",
|
||||
provider: "telegram",
|
||||
supportedPorts: [443, 80, 88, 8443],
|
||||
},
|
||||
);
|
||||
}
|
||||
if (input.action === "verify") {
|
||||
if (
|
||||
endpoint.provider !== "slack" ||
|
||||
endpoint.status !== "verifying" ||
|
||||
endpoint.setup.step !== "provider_setup"
|
||||
) {
|
||||
throw conflict(
|
||||
"Provider verification is not available at this setup step",
|
||||
{
|
||||
code: "chat_endpoint_invalid_setup_step",
|
||||
},
|
||||
);
|
||||
}
|
||||
if (!endpoint.setup.webhookVerifiedAt) {
|
||||
throw conflict("Paperclip has not received a verified Slack callback yet", {
|
||||
code: "chat_webhook_not_verified",
|
||||
});
|
||||
}
|
||||
} else if (
|
||||
input.action === "configure" &&
|
||||
!["draft", "attention", "revoked"].includes(endpoint.status) &&
|
||||
// GitHub installation discovery can advance the status before the
|
||||
// wizard has configured its runtime. Finish that exact setup stage;
|
||||
// configured connections still require the explicit reconnect flow.
|
||||
!(
|
||||
endpoint.provider === "github" &&
|
||||
endpoint.status === "verifying" &&
|
||||
endpoint.setup.step === "provider_setup"
|
||||
)
|
||||
) {
|
||||
throw conflict(
|
||||
"This connection is already configured; use its reconnect flow if credentials need repair",
|
||||
{
|
||||
code: "chat_endpoint_already_configured",
|
||||
},
|
||||
);
|
||||
} else if (
|
||||
input.action === "reconnect" &&
|
||||
!["verifying", "active", "attention", "revoked", "paused"].includes(
|
||||
endpoint.status,
|
||||
)
|
||||
) {
|
||||
throw conflict(
|
||||
"This chat connection does not currently need reconnecting",
|
||||
{
|
||||
code: "chat_endpoint_not_reconnectable",
|
||||
},
|
||||
);
|
||||
}
|
||||
if (
|
||||
endpoint.provider === "github" &&
|
||||
(input.action === "configure" || input.action === "reconnect") &&
|
||||
!endpoint.setup.webhookVerifiedAt
|
||||
) {
|
||||
throw conflict(
|
||||
"GitHub has not delivered a signed webhook ping for this secret yet",
|
||||
{
|
||||
code: "chat_webhook_not_verified",
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
let verifiedCurrentIdentity: VerifiedProviderIdentity = endpoint;
|
||||
if (
|
||||
input.action === "reconnect" &&
|
||||
endpoint.provider === "github" &&
|
||||
!endpoint.botExternalId
|
||||
) {
|
||||
const storedCredentials = await resolveCredentials(endpoint).catch(() => {
|
||||
throw conflict(
|
||||
"Reconnect the existing GitHub App credentials before changing them",
|
||||
{
|
||||
code: "chat_bot_identity_unverifiable",
|
||||
},
|
||||
);
|
||||
});
|
||||
const storedIdentity = await verifyCredentials(
|
||||
"github",
|
||||
storedCredentials,
|
||||
);
|
||||
if (
|
||||
!legacyGitHubLabelsMatchVerifiedCredentials(endpoint, storedIdentity)
|
||||
) {
|
||||
throw conflict(
|
||||
"The stored credentials no longer verify the GitHub App that owns this connection",
|
||||
{
|
||||
code: "chat_bot_identity_unverifiable",
|
||||
},
|
||||
);
|
||||
}
|
||||
verifiedCurrentIdentity = storedIdentity;
|
||||
}
|
||||
|
||||
const previousSpekoSigningSecret = endpoint.provider === "speko"
|
||||
? (await resolveCredentials(endpoint).catch(() => ({} as Record<string, string>))).signingSecret : undefined;
|
||||
let credentials =
|
||||
input.credentials && Object.keys(input.credentials).length > 0
|
||||
? await normalizedCredentials(endpoint, input.credentials)
|
||||
: await resolveCredentials(endpoint).catch(() => {
|
||||
throw unprocessable(
|
||||
`${PROVIDER_LABELS[endpoint.provider]} credentials are required`,
|
||||
);
|
||||
});
|
||||
if (endpoint.provider === "imessage-photon") {
|
||||
credentials = preparePhotonConfiguration(endpoint, input, credentials);
|
||||
}
|
||||
const identity = await verifyCredentials(endpoint.provider, credentials);
|
||||
// Once setup has claimed a provider bot identity, every credential repair
|
||||
// must prove that same identity before secrets can be replaced. A process
|
||||
// interruption can leave the endpoint in `attention` after the claim but
|
||||
// before credential persistence; a repeated first-setup request must not
|
||||
// turn that recovery state into an identity-swap escape hatch.
|
||||
if (input.action === "reconnect" || Boolean(endpoint.botExternalId)) {
|
||||
if (
|
||||
!nativeBotIdentityMatches(
|
||||
endpoint.provider,
|
||||
verifiedCurrentIdentity,
|
||||
identity,
|
||||
)
|
||||
) {
|
||||
throw conflict(
|
||||
"The provider credentials identify a different bot; create a new connection for that bot instead",
|
||||
{ code: "chat_bot_identity_changed" },
|
||||
);
|
||||
}
|
||||
}
|
||||
await assertNativeBotIdentityAvailable(endpoint, identity);
|
||||
const prepared = await prepareProviderInventory(endpoint, credentials);
|
||||
const credentialsChangedByDiscovery =
|
||||
endpoint.provider === "github" &&
|
||||
prepared.credentials.installationId !== credentials.installationId;
|
||||
credentials = prepared.credentials;
|
||||
await options.nativeBotIdentityClaimBarrier?.();
|
||||
// Claim a fresh provider identity before any newly supplied secrets are
|
||||
// persisted. The provider-global unique indexes are the multi-process
|
||||
// arbitration boundary when two setup requests both pass the optimistic
|
||||
// preflight. A process that stops after this small claim leaves a
|
||||
// fail-closed attention endpoint that the same operator can reconnect;
|
||||
// it never leaves credentials bound to the losing endpoint.
|
||||
if (!endpoint.botExternalId) {
|
||||
try {
|
||||
const claimed = await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
await notifyChatEndpointWork(tx);
|
||||
const rows = await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "attention",
|
||||
providerAccountId: identity.providerAccountId ?? null,
|
||||
providerAccountLabel: identity.providerAccountLabel ?? null,
|
||||
botExternalId: identity.botExternalId ?? null,
|
||||
botUsername: identity.botUsername ?? null,
|
||||
botDisplayName:
|
||||
identity.botLabel ??
|
||||
endpoint.botDisplayName ??
|
||||
record.assignedAgentName,
|
||||
healthMessage: "Provider setup must be completed",
|
||||
lastError: null,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, endpoint.id),
|
||||
eq(chatEndpoints.status, endpoint.status),
|
||||
isNull(chatEndpoints.botExternalId),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatEndpoints.id });
|
||||
await credentialLease.assertOwned(tx);
|
||||
return rows;
|
||||
});
|
||||
if (claimed.length !== 1) {
|
||||
throw conflict("This chat connection changed during setup", {
|
||||
code: "chat_endpoint_setup_changed",
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
if (isNativeBotIdentityUniqueViolation(error)) {
|
||||
throw nativeBotIdentityConflict(endpoint.provider);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
if (endpoint.provider === "imessage-photon") {
|
||||
await persistPhotonConfiguration(endpoint, credentials, credentialLease);
|
||||
}
|
||||
if (
|
||||
(input.credentials && Object.keys(input.credentials).length > 0) ||
|
||||
credentialsChangedByDiscovery
|
||||
)
|
||||
await persistCredentials(
|
||||
endpoint,
|
||||
credentials,
|
||||
credentialLease,
|
||||
actorUserId,
|
||||
);
|
||||
if (prepared.inventory)
|
||||
await reconcileProviderResourceRows(
|
||||
endpoint,
|
||||
prepared.inventory,
|
||||
credentialLease,
|
||||
);
|
||||
const updatedAt = new Date();
|
||||
const waitingForSlackConfiguration =
|
||||
endpoint.provider === "slack" &&
|
||||
(input.action === "configure" || input.action === "reconnect");
|
||||
try {
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const current = await tx
|
||||
.select({ setup: chatEndpoints.setup })
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) throw notFound("Chat endpoint not found");
|
||||
if (endpoint.provider === "speko") {
|
||||
await installSpekoVoiceTools(tx, endpoint, actorUserId);
|
||||
}
|
||||
const observedSlackUrl = (current.setup as InternalSetupState).slackCallbackSurfaces?.events?.url;
|
||||
const preserveSlackVerification = isProvisionedSlackSetup(current.setup.slackSetupMethod)
|
||||
&& (current.setup as InternalSetupState).slackVerificationSigningFingerprint === createHash("sha256").update(credentials.signingSecret ?? "").digest("hex")
|
||||
&& Boolean(observedSlackUrl && slackCallbackMatchesPublicUrl(observedSlackUrl, `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/slack`));
|
||||
await notifyChatEndpointWork(tx);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "verifying",
|
||||
providerAccountId: identity.providerAccountId ?? null,
|
||||
providerAccountLabel: identity.providerAccountLabel ?? null,
|
||||
botExternalId: identity.botExternalId ?? null,
|
||||
botUsername: identity.botUsername ?? null,
|
||||
botDisplayName:
|
||||
identity.botLabel ??
|
||||
endpoint.botDisplayName ??
|
||||
record.assignedAgentName,
|
||||
healthMessage: waitingForSlackConfiguration
|
||||
? "Finish provider webhook configuration"
|
||||
: "Waiting for a test conversation",
|
||||
lastError: null,
|
||||
lastEventAt: null,
|
||||
setup: {
|
||||
...current.setup,
|
||||
...(endpoint.provider === "github" ? { github: { ...current.setup.github, stage: current.setup.github?.stage ?? "connect", appOwnerType: identity.githubOwnerType, appOwnerLogin: identity.githubOwnerLogin } } : {}),
|
||||
step: waitingForSlackConfiguration ? "provider_setup" : "test",
|
||||
...(endpoint.provider === "imessage-photon" ? { photonIntakeAfter: (current.setup as InternalSetupState).photonIntakeAfter ?? updatedAt.toISOString() } : {}),
|
||||
testStartedAt: waitingForSlackConfiguration
|
||||
? null
|
||||
: updatedAt.toISOString(),
|
||||
webhookVerifiedAt: waitingForSlackConfiguration
|
||||
? preserveSlackVerification ? (current.setup.webhookVerifiedAt ?? null) : null
|
||||
: (current.setup.webhookVerifiedAt ?? null),
|
||||
runtimeGeneration: runtimeGeneration(current.setup) + 1,
|
||||
} as InternalSetupState,
|
||||
updatedAt,
|
||||
})
|
||||
.where(eq(chatEndpoints.id, endpoint.id));
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({
|
||||
status: "active",
|
||||
enabled: true,
|
||||
healthStatus: "healthy",
|
||||
healthMessage: "Provider credentials verified",
|
||||
healthCheckedAt: updatedAt,
|
||||
updatedAt,
|
||||
})
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await tx
|
||||
.update(toolApplications)
|
||||
.set({ status: "active", archivedAt: null, updatedAt })
|
||||
.where(eq(toolApplications.id, record.applicationId));
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
} catch (error) {
|
||||
if (isNativeBotIdentityUniqueViolation(error)) {
|
||||
throw nativeBotIdentityConflict(endpoint.provider);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
try {
|
||||
const next = await endpointRecord(endpoint.id);
|
||||
if (!next) throw notFound("Chat endpoint not found");
|
||||
await invalidateRuntime(endpoint.id);
|
||||
if (endpoint.provider === "github" && input.action === "reconnect" && endpoint.setup.github?.cloudRegistrationId) {
|
||||
await repairGitHubCloudSetup(endpoint, credentialLease, actorUserId);
|
||||
}
|
||||
if (endpoint.provider === "github" && input.action === "reconnect" && !endpoint.setup.github?.cloudRegistrationId) {
|
||||
await repairGitHubWebhookSetup(endpoint, credentials, credentialLease, actorUserId);
|
||||
}
|
||||
if (next.endpoint.provider === "speko") {
|
||||
await configureSpekoTools(endpoint, credentials, credentialLease, previousSpekoSigningSecret);
|
||||
}
|
||||
if (next.endpoint.provider === "discord")
|
||||
await reconcileDiscordCommands(endpoint.id, credentialLease, true);
|
||||
await runtimeFor(next.endpoint, {
|
||||
requireDiscordOwnership: next.endpoint.provider === "discord",
|
||||
waitForDiscordOwnership: next.endpoint.provider === "discord",
|
||||
});
|
||||
|
||||
if (endpoint.provider === "telegram" && getWebhookPublicBaseUrl()) {
|
||||
await registerTelegramWebhook(endpoint, input, credentials, credentialLease, next);
|
||||
}
|
||||
} catch (error) {
|
||||
const failure = redactError(error);
|
||||
await invalidateRuntime(endpoint.id).catch(() => undefined);
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
await notifyChatEndpointWork(tx);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "attention",
|
||||
healthMessage: "Provider setup needs attention",
|
||||
lastError: failure,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatEndpoints.id, endpoint.id));
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({
|
||||
healthStatus: "degraded",
|
||||
healthMessage: "Provider setup failed",
|
||||
lastError: failure,
|
||||
healthCheckedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
throw unprocessable(failure, { code: "chat_provider_setup_failed" });
|
||||
}
|
||||
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
await logActivity(tx as unknown as Db, {
|
||||
companyId: endpoint.companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action:
|
||||
input.action === "reconnect"
|
||||
? "chat_endpoint.reconnected"
|
||||
: "chat_endpoint.credentials_verified",
|
||||
entityType: "tool_connection",
|
||||
entityId: endpoint.connectionId,
|
||||
details: { endpointId: endpoint.id, provider: endpoint.provider },
|
||||
});
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
return get(endpoint.id);
|
||||
}
|
||||
|
||||
function runtimeConfig(
|
||||
endpoint: EndpointRow,
|
||||
userName: string,
|
||||
credentials: Record<string, string>,
|
||||
): ResolvedChatSdkProviderConfig {
|
||||
if (endpoint.provider === "speko") throw conflict("Speko uses the voice runtime");
|
||||
if (endpoint.provider === "imessage-photon") return {
|
||||
provider: "imessage-photon", userName,
|
||||
intakeAfter: Date.parse(String((endpoint.setup as InternalSetupState).photonIntakeAfter ?? endpoint.setup.testStartedAt ?? endpoint.createdAt.toISOString())),
|
||||
credentials: { allocation: credentials.allocation === "shared" ? "shared" : "dedicated", projectId: credentials.projectId, lineId: credentials.lineId, projectSecret: credentials.projectSecret, phoneNumber: endpoint.botExternalId! },
|
||||
};
|
||||
if (endpoint.provider === "slack")
|
||||
return slackRuntimeConfig(endpoint, userName, credentials);
|
||||
if (endpoint.provider === "telegram")
|
||||
return {
|
||||
provider: "telegram",
|
||||
userName,
|
||||
maxDownloadBytes: MAX_ATTACHMENT_BYTES,
|
||||
credentials: {
|
||||
botToken: credentials.botToken,
|
||||
secretToken: credentials.webhookSecret,
|
||||
},
|
||||
};
|
||||
if (endpoint.provider === "discord")
|
||||
return {
|
||||
provider: "discord",
|
||||
userName,
|
||||
credentials: {
|
||||
applicationId: credentials.applicationId,
|
||||
botToken: credentials.botToken,
|
||||
guildId: credentials.guildId,
|
||||
},
|
||||
};
|
||||
if (endpoint.provider === "microsoft-teams")
|
||||
return {
|
||||
provider: "microsoft-teams",
|
||||
userName,
|
||||
credentials: {
|
||||
appId: credentials.clientId,
|
||||
appPassword: credentials.clientSecret,
|
||||
appTenantId: credentials.tenantId,
|
||||
appType: "SingleTenant",
|
||||
},
|
||||
};
|
||||
return {
|
||||
provider: "github",
|
||||
userName,
|
||||
credentials: {
|
||||
appId: credentials.appId,
|
||||
privateKey: credentials.privateKey,
|
||||
installationId: credentials.installationId
|
||||
? Number(credentials.installationId)
|
||||
: undefined,
|
||||
webhookSecret: credentials.webhookSecret,
|
||||
},
|
||||
};
|
||||
}
|
||||
return { configure, configureWithCredentialLease, runtimeConfig };
|
||||
}
|
||||
@@ -0,0 +1,507 @@
|
||||
import { prepareSlackInventory } from "./slack/inventory.js";
|
||||
import { prepareGitHubInventory } from "./github/inventory.js";
|
||||
import { prepareDiscordInventory } from "./discord/inventory.js";
|
||||
import type { VerifiedProviderIdentity } from "./runtime-types.js";
|
||||
import { verifySlackCredentials } from "./slack/credentials.js";
|
||||
import { verifyDiscordCredentials } from "./discord/credentials.js";
|
||||
import { verifyTelegramCredentials } from "./telegram/credentials.js";
|
||||
import { verifyGitHubCredentials } from "./github/credentials.js";
|
||||
import { verifySpekoCredentials } from "./speko/credentials.js";
|
||||
import { verifyTeamsCredentials } from "./microsoft-teams/credentials.js";
|
||||
import { createPhotonCredentialChecks } from "./imessage-photon/credentials.js";
|
||||
|
||||
import type { EndpointRow, DbOrTransaction } from "./runtime-types.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../chat-credential-mutation-lease.js";
|
||||
|
||||
import { photonSharedScope } from "../photon/cloud.js";
|
||||
|
||||
import { photonChannelConfigurationSchema } from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
randomBytes,
|
||||
randomUUID,
|
||||
} from "node:crypto";
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
inArray,
|
||||
ne,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatEndpoints,
|
||||
companySecretBindings,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatProvider,
|
||||
ToolCredentialSecretRef,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import { isUniqueViolation } from "../../db-errors.js";
|
||||
|
||||
import {
|
||||
conflict,
|
||||
notFound,
|
||||
unprocessable,
|
||||
} from "../../errors.js";
|
||||
import { logger } from "../../middleware/logger.js";
|
||||
|
||||
import { type ChatProviderInventoryResult } from "../chat-provider-inventory.js";
|
||||
|
||||
import {
|
||||
normalizeMicrosoftTeamsCredentialIds,
|
||||
} from "../chat-teams-credentials.js";
|
||||
|
||||
import { secretService } from "../secrets.js";
|
||||
|
||||
type ChatCredentialsOptions = {
|
||||
secrets: ReturnType<typeof secretService>;
|
||||
PROVIDER_LABELS: Record<ChatProvider, string>;
|
||||
REQUIRED_CREDENTIALS: Record<Exclude<ChatProvider, "github">, readonly string[]>;
|
||||
PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number;
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
redactError: (error: unknown) => string;
|
||||
credentialFingerprint: (refs: ToolCredentialSecretRef[]) => string;
|
||||
verifyPhotonCredentials: ReturnType<typeof createPhotonCredentialChecks>["verifyPhotonCredentials"];
|
||||
};
|
||||
export function createChatCredentials(db: Db, options: ChatCredentialsOptions) {
|
||||
const { secrets, PROVIDER_LABELS, REQUIRED_CREDENTIALS, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, fetchImpl, redactError, credentialFingerprint, verifyPhotonCredentials } = options;
|
||||
|
||||
|
||||
async function normalizedCredentials(
|
||||
endpoint: EndpointRow,
|
||||
supplied: Record<string, string> | undefined,
|
||||
) {
|
||||
const values = await resolveCredentials(endpoint).catch(
|
||||
() => ({}) as Record<string, string>,
|
||||
);
|
||||
for (const [key, rawValue] of Object.entries(supplied ?? {})) {
|
||||
if (typeof rawValue === "string" && rawValue.trim())
|
||||
values[key] = key === "privateKey" ? rawValue : rawValue.trim();
|
||||
}
|
||||
if (endpoint.provider === "telegram" && !values.webhookSecret) {
|
||||
values.webhookSecret = randomBytes(32).toString("hex");
|
||||
}
|
||||
const required =
|
||||
endpoint.provider === "github"
|
||||
? ["appId", "privateKey", "webhookSecret"]
|
||||
: REQUIRED_CREDENTIALS[endpoint.provider];
|
||||
const missing = required.filter((key) => !values[key]);
|
||||
if (missing.length > 0) {
|
||||
throw unprocessable(
|
||||
`Missing required ${PROVIDER_LABELS[endpoint.provider]} credentials: ${missing.join(", ")}`,
|
||||
);
|
||||
}
|
||||
return endpoint.provider === "microsoft-teams"
|
||||
? normalizeMicrosoftTeamsCredentialIds(values)
|
||||
: values;
|
||||
}
|
||||
|
||||
async function verifyCredentials(
|
||||
provider: ChatProvider,
|
||||
credentials: Record<string, string>,
|
||||
): Promise<VerifiedProviderIdentity> {
|
||||
if (provider === "imessage-photon") return verifyPhotonCredentials(credentials);
|
||||
if (provider === "slack") return verifySlackCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS);
|
||||
if (provider === "discord") return verifyDiscordCredentials(credentials, fetchImpl, redactError);
|
||||
if (provider === "telegram") return verifyTelegramCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS);
|
||||
if (provider === "github") return verifyGitHubCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS);
|
||||
if (provider === "speko") return verifySpekoCredentials(credentials, fetchImpl);
|
||||
return verifyTeamsCredentials(credentials, fetchImpl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS);
|
||||
}
|
||||
|
||||
function nativeBotIdentityKey(
|
||||
provider: ChatProvider,
|
||||
identity: VerifiedProviderIdentity,
|
||||
): string | null {
|
||||
const account = identity.providerAccountId?.trim().toLowerCase();
|
||||
const bot = identity.botExternalId?.trim().toLowerCase();
|
||||
if (provider === "github" || provider === "discord") {
|
||||
if (bot) return `${provider}:app:${bot}`;
|
||||
return null;
|
||||
}
|
||||
if (!account || !bot) return null;
|
||||
return `${provider}:${account}:${bot}`;
|
||||
}
|
||||
|
||||
function providerUsesGlobalApplicationIdentity(
|
||||
provider: ChatProvider,
|
||||
): boolean {
|
||||
return (
|
||||
provider === "github" ||
|
||||
provider === "discord" ||
|
||||
provider === "microsoft-teams" || provider === "imessage-photon"
|
||||
);
|
||||
}
|
||||
|
||||
function nativeBotIdentityMatches(
|
||||
provider: ChatProvider,
|
||||
current: VerifiedProviderIdentity,
|
||||
incoming: VerifiedProviderIdentity,
|
||||
): boolean {
|
||||
const currentKey = nativeBotIdentityKey(provider, current);
|
||||
return (
|
||||
currentKey !== null &&
|
||||
currentKey === nativeBotIdentityKey(provider, incoming)
|
||||
);
|
||||
}
|
||||
|
||||
function legacyGitHubLabelsMatchVerifiedCredentials(
|
||||
current: VerifiedProviderIdentity,
|
||||
verified: VerifiedProviderIdentity,
|
||||
): boolean {
|
||||
return (
|
||||
!current.botExternalId &&
|
||||
Boolean(verified.botExternalId) &&
|
||||
current.providerAccountId?.trim().toLowerCase() ===
|
||||
verified.providerAccountId?.trim().toLowerCase() &&
|
||||
current.botUsername?.trim().toLowerCase() ===
|
||||
verified.botUsername?.trim().toLowerCase()
|
||||
);
|
||||
}
|
||||
|
||||
async function assertNativeBotIdentityAvailable(
|
||||
endpoint: EndpointRow,
|
||||
identity: VerifiedProviderIdentity,
|
||||
): Promise<void> {
|
||||
const key = nativeBotIdentityKey(endpoint.provider, identity);
|
||||
if (!key) {
|
||||
throw unprocessable(
|
||||
`${PROVIDER_LABELS[endpoint.provider]} did not return a stable native bot identity`,
|
||||
);
|
||||
}
|
||||
const candidates = await db
|
||||
.select({
|
||||
id: chatEndpoints.id,
|
||||
companyId: chatEndpoints.companyId,
|
||||
assignedAgentId: chatEndpoints.assignedAgentId,
|
||||
providerAccountId: chatEndpoints.providerAccountId,
|
||||
botExternalId: chatEndpoints.botExternalId,
|
||||
botUsername: chatEndpoints.botUsername,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.where(
|
||||
and(
|
||||
providerUsesGlobalApplicationIdentity(endpoint.provider)
|
||||
? undefined
|
||||
: eq(chatEndpoints.companyId, endpoint.companyId),
|
||||
eq(chatEndpoints.provider, endpoint.provider),
|
||||
ne(chatEndpoints.id, endpoint.id),
|
||||
endpoint.provider === "imessage-photon" ? ne(chatEndpoints.status, "archived") : inArray(chatEndpoints.status, [
|
||||
"verifying",
|
||||
"active",
|
||||
"paused",
|
||||
"attention",
|
||||
]),
|
||||
),
|
||||
);
|
||||
const conflictEndpoint = candidates.find((candidate) => {
|
||||
if (!providerUsesGlobalApplicationIdentity(endpoint.provider)) {
|
||||
return nativeBotIdentityKey(endpoint.provider, candidate) === key;
|
||||
}
|
||||
const candidateAppId = candidate.botExternalId?.trim().toLowerCase();
|
||||
const incomingAppId = identity.botExternalId?.trim().toLowerCase();
|
||||
if (candidateAppId && incomingAppId)
|
||||
return candidateAppId === incomingAppId;
|
||||
return (
|
||||
candidate.providerAccountId?.trim().toLowerCase() ===
|
||||
identity.providerAccountId?.trim().toLowerCase() &&
|
||||
candidate.botUsername?.trim().toLowerCase() ===
|
||||
identity.botUsername?.trim().toLowerCase()
|
||||
);
|
||||
});
|
||||
if (conflictEndpoint) {
|
||||
throw conflict(
|
||||
`This ${PROVIDER_LABELS[endpoint.provider]} bot already represents another Paperclip agent connection`,
|
||||
conflictEndpoint.companyId === endpoint.companyId
|
||||
? {
|
||||
code: "chat_bot_identity_in_use",
|
||||
endpointId: conflictEndpoint.id,
|
||||
assignedAgentId: conflictEndpoint.assignedAgentId,
|
||||
}
|
||||
: { code: "chat_bot_identity_in_use" },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function isNativeBotIdentityUniqueViolation(error: unknown): boolean {
|
||||
return (
|
||||
isUniqueViolation(error, "chat_endpoints_photon_number_uq") ||
|
||||
isUniqueViolation(error, "chat_endpoints_live_bot_external_uq") ||
|
||||
isUniqueViolation(error, "chat_endpoints_live_discord_bot_external_uq") ||
|
||||
isUniqueViolation(
|
||||
error,
|
||||
"chat_endpoints_live_global_app_bot_external_uq",
|
||||
) ||
|
||||
isUniqueViolation(error, "chat_endpoints_live_bot_username_uq")
|
||||
);
|
||||
}
|
||||
|
||||
function nativeBotIdentityConflict(provider: ChatProvider) {
|
||||
return conflict(
|
||||
`This ${PROVIDER_LABELS[provider]} bot already represents another Paperclip agent connection`,
|
||||
{ code: "chat_bot_identity_in_use" },
|
||||
);
|
||||
}
|
||||
|
||||
async function persistCredentials(
|
||||
endpoint: EndpointRow,
|
||||
credentials: Record<string, string>,
|
||||
credentialLease: CredentialMutationLeaseGuard,
|
||||
actorUserId?: string | null,
|
||||
) {
|
||||
await credentialLease.assertOwned();
|
||||
const previousRefs = await db
|
||||
.select({ refs: toolConnections.credentialSecretRefs })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.id, endpoint.connectionId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]?.refs ?? []);
|
||||
const refs: ToolCredentialSecretRef[] = [];
|
||||
try {
|
||||
for (const [key, value] of Object.entries(credentials)) {
|
||||
if (endpoint.provider === "imessage-photon" && key !== "projectSecret") continue;
|
||||
await credentialLease.assertOwned();
|
||||
const suffix = randomUUID();
|
||||
const secret = await secrets.create(
|
||||
endpoint.companyId,
|
||||
{
|
||||
name: `chat.${endpoint.provider}.${endpoint.id.slice(0, 8)}.${key}.${suffix.slice(0, 8)}`,
|
||||
key: `CHAT_${endpoint.provider.replaceAll("-", "_")}_${endpoint.id.replaceAll("-", "_")}_${key}_${suffix}`.toUpperCase(),
|
||||
provider: "local_encrypted",
|
||||
managedMode: "paperclip_managed",
|
||||
value,
|
||||
description: `${PROVIDER_LABELS[endpoint.provider]} channel credential for endpoint ${endpoint.id}`,
|
||||
},
|
||||
{ userId: actorUserId ?? undefined },
|
||||
);
|
||||
refs.push({
|
||||
secretId: secret.id,
|
||||
versionSelector: "latest",
|
||||
configPath: `credentials.${key}`,
|
||||
required: true,
|
||||
label: key,
|
||||
projectionClass: "unclassified",
|
||||
});
|
||||
await credentialLease.assertOwned();
|
||||
}
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const current = await tx
|
||||
.select({ refs: toolConnections.credentialSecretRefs })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.id, endpoint.connectionId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) throw notFound("Chat connection not found");
|
||||
if (
|
||||
credentialFingerprint(current.refs) !==
|
||||
credentialFingerprint(previousRefs)
|
||||
) {
|
||||
throw conflict("Chat credentials changed during rotation", {
|
||||
code: "chat_endpoint_credentials_changed",
|
||||
});
|
||||
}
|
||||
await tx
|
||||
.delete(companySecretBindings)
|
||||
.where(
|
||||
and(
|
||||
eq(companySecretBindings.companyId, endpoint.companyId),
|
||||
eq(companySecretBindings.targetType, "tool_connection"),
|
||||
eq(companySecretBindings.targetId, endpoint.connectionId),
|
||||
),
|
||||
);
|
||||
if (refs.length > 0) {
|
||||
await tx.insert(companySecretBindings).values(
|
||||
refs.map((ref) => ({
|
||||
companyId: endpoint.companyId,
|
||||
secretId: ref.secretId,
|
||||
targetType: "tool_connection",
|
||||
targetId: endpoint.connectionId,
|
||||
configPath: ref.configPath,
|
||||
versionSelector: String(ref.versionSelector ?? "latest"),
|
||||
required: ref.required,
|
||||
label: ref.label ?? null,
|
||||
projectionClass: ref.projectionClass ?? "unclassified",
|
||||
projectionAllowlistKey: ref.projectionAllowlistKey ?? null,
|
||||
})),
|
||||
);
|
||||
}
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({ credentialSecretRefs: refs, updatedAt: new Date() })
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
} catch (error) {
|
||||
// A lost database acknowledgement can be ambiguous even when the
|
||||
// transaction committed. Delete only secrets proven unbound after the
|
||||
// failure; when that proof cannot be read, leak safely for later cleanup
|
||||
// instead of deleting a possibly-authoritative credential.
|
||||
const boundSecretIds =
|
||||
refs.length === 0
|
||||
? new Set<string>()
|
||||
: await db
|
||||
.select({ secretId: companySecretBindings.secretId })
|
||||
.from(companySecretBindings)
|
||||
.where(
|
||||
and(
|
||||
eq(companySecretBindings.companyId, endpoint.companyId),
|
||||
inArray(
|
||||
companySecretBindings.secretId,
|
||||
refs.map((ref) => ref.secretId),
|
||||
),
|
||||
),
|
||||
)
|
||||
.then((rows) => new Set(rows.map((row) => row.secretId)))
|
||||
.catch(() => null);
|
||||
if (boundSecretIds) {
|
||||
await Promise.allSettled(
|
||||
refs
|
||||
.filter((ref) => !boundSecretIds.has(ref.secretId))
|
||||
.map((ref) => secrets.remove(ref.secretId)),
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const currentIds = new Set(refs.map((ref) => ref.secretId));
|
||||
const retired = previousRefs.filter((ref) => !currentIds.has(ref.secretId));
|
||||
const results = await Promise.allSettled(
|
||||
retired.map((ref) => secrets.remove(ref.secretId)),
|
||||
);
|
||||
if (results.some((result) => result.status === "rejected")) {
|
||||
logger.warn(
|
||||
{ endpointId: endpoint.id },
|
||||
"chat credential rotation left an unbound retired secret for later cleanup",
|
||||
);
|
||||
}
|
||||
return refs;
|
||||
}
|
||||
|
||||
async function clearCredentials(
|
||||
endpoint: EndpointRow,
|
||||
credentialLease: CredentialMutationLeaseGuard,
|
||||
) {
|
||||
let refs: ToolCredentialSecretRef[] = [];
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
const current = await tx
|
||||
.select({ refs: toolConnections.credentialSecretRefs })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.id, endpoint.connectionId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) throw notFound("Chat connection not found");
|
||||
refs = current.refs;
|
||||
await tx
|
||||
.delete(companySecretBindings)
|
||||
.where(
|
||||
and(
|
||||
eq(companySecretBindings.companyId, endpoint.companyId),
|
||||
eq(companySecretBindings.targetType, "tool_connection"),
|
||||
eq(companySecretBindings.targetId, endpoint.connectionId),
|
||||
),
|
||||
);
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({ credentialSecretRefs: [], updatedAt: new Date() })
|
||||
.where(eq(toolConnections.id, endpoint.connectionId));
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
const results = await Promise.allSettled(
|
||||
refs.map((ref) => secrets.remove(ref.secretId)),
|
||||
);
|
||||
if (results.some((result) => result.status === "rejected")) {
|
||||
logger.warn(
|
||||
{ endpointId: endpoint.id },
|
||||
"chat endpoint removal left an unbound retired secret for later cleanup",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveCredentialRefs(
|
||||
endpoint: EndpointRow,
|
||||
refs: ToolCredentialSecretRef[],
|
||||
database: DbOrTransaction = db,
|
||||
): Promise<Record<string, string>> {
|
||||
const values: Record<string, string> = {};
|
||||
for (const ref of refs) {
|
||||
const key = ref.configPath.replace(/^credentials\./, "");
|
||||
values[key] = await (database === db ? secrets : secretService(database as Db)).resolveSecretValue(
|
||||
endpoint.companyId,
|
||||
ref.secretId,
|
||||
ref.versionSelector ?? "latest",
|
||||
{
|
||||
consumerType: "tool_connection",
|
||||
consumerId: endpoint.connectionId,
|
||||
configPath: ref.configPath,
|
||||
actorType: "system",
|
||||
actorId: null,
|
||||
},
|
||||
);
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
async function resolveCredentials(
|
||||
endpoint: EndpointRow,
|
||||
database: DbOrTransaction = db,
|
||||
): Promise<Record<string, string>> {
|
||||
const connection = await database
|
||||
.select({ refs: toolConnections.credentialSecretRefs, config: toolConnections.config })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.id, endpoint.connectionId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!connection) throw notFound("Chat connection not found");
|
||||
const values = await resolveCredentialRefs(endpoint, connection.refs, database);
|
||||
if (endpoint.provider === "imessage-photon") {
|
||||
const configuration = photonChannelConfigurationSchema.parse(connection.config.photon);
|
||||
return { ...values, ...configuration, lineId: configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId };
|
||||
}
|
||||
return values;
|
||||
}
|
||||
|
||||
async function prepareProviderInventory(
|
||||
endpoint: EndpointRow,
|
||||
credentials: Record<string, string>,
|
||||
): Promise<{
|
||||
credentials: Record<string, string>;
|
||||
inventory: ChatProviderInventoryResult | null;
|
||||
managementUrl?: string;
|
||||
}> {
|
||||
try {
|
||||
if (endpoint.provider === "slack") return await prepareSlackInventory(credentials, fetchImpl);
|
||||
if (endpoint.provider === "github") return await prepareGitHubInventory(credentials, fetchImpl);
|
||||
if (endpoint.provider === "discord") return await prepareDiscordInventory(credentials, fetchImpl);
|
||||
return { credentials, inventory: null };
|
||||
} catch (error) {
|
||||
throw unprocessable(redactError(error), {
|
||||
code: "chat_provider_inventory_failed",
|
||||
});
|
||||
}
|
||||
}
|
||||
return { normalizedCredentials, verifyCredentials, nativeBotIdentityMatches, legacyGitHubLabelsMatchVerifiedCredentials, assertNativeBotIdentityAvailable, isNativeBotIdentityUniqueViolation, nativeBotIdentityConflict, persistCredentials, clearCredentials, resolveCredentialRefs, resolveCredentials, prepareProviderInventory };
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
import { createChatPublicationReplacement } from "../publication-replacement.js";
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
import {
|
||||
eq,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
chatPublications,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
import {
|
||||
discordMarkdownRequiresAttachment,
|
||||
} from "../../chat-discord.js";
|
||||
|
||||
type PublicationPreparationOptions = {
|
||||
runPublicationToReplace: ReturnType<typeof createChatPublicationReplacement>["runPublicationToReplace"];
|
||||
};
|
||||
export function createDiscordPublicationPreparation(db: Db, options: PublicationPreparationOptions) {
|
||||
const { runPublicationToReplace } = options;
|
||||
async function prepareDiscordPublication(publication: typeof chatPublications.$inferSelect, persisted: SafeChatPublicationPayload): Promise<typeof chatPublications.$inferSelect> {
|
||||
if (
|
||||
persisted.transportPart ||
|
||||
persisted.card ||
|
||||
persisted.interactionId ||
|
||||
!discordMarkdownRequiresAttachment(persisted.text)
|
||||
) {
|
||||
return publication;
|
||||
}
|
||||
|
||||
// A normal response can replace its run's working placeholder. Discord's
|
||||
// adapter cannot add a file while editing, so first turn that placeholder
|
||||
// into a durable hand-off and then publish the complete Markdown as its
|
||||
// own ordered attachment row. Publications without a replacement lane
|
||||
// remain one provider call: a short caption and the lossless file.
|
||||
const replacesProgressMessage = Boolean(
|
||||
await runPublicationToReplace(publication, persisted),
|
||||
);
|
||||
return db.transaction(async (tx) => {
|
||||
const current = await tx
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(eq(chatPublications.id, publication.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) return publication;
|
||||
const currentPayload = current.payload as SafeChatPublicationPayload;
|
||||
if (
|
||||
currentPayload.transportPart ||
|
||||
currentPayload.card ||
|
||||
currentPayload.interactionId ||
|
||||
!["pending", "retry"].includes(current.state) ||
|
||||
!discordMarkdownRequiresAttachment(currentPayload.text)
|
||||
) {
|
||||
return current;
|
||||
}
|
||||
|
||||
const batchId = current.id;
|
||||
const updatedAt = new Date();
|
||||
const transportPart = (
|
||||
index: number,
|
||||
count: number,
|
||||
mode: "inline" | "discord_markdown_attachment",
|
||||
) => ({
|
||||
batchId,
|
||||
count,
|
||||
index,
|
||||
mode,
|
||||
orderKey: `${batchId}:${String(index).padStart(4, "0")}`,
|
||||
});
|
||||
if (!replacesProgressMessage) {
|
||||
const attachmentPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
transportPart: transportPart(0, 1, "discord_markdown_attachment"),
|
||||
};
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: attachmentPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
return { ...current, payload: attachmentPayload, updatedAt };
|
||||
}
|
||||
|
||||
const handoffPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
text: "This response needs a separate attachment because it exceeds the message limit.",
|
||||
transportPart: transportPart(0, 2, "inline"),
|
||||
};
|
||||
delete handoffPayload.attachmentIds;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: handoffPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
|
||||
const attachmentPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
transportPart: transportPart(1, 2, "discord_markdown_attachment"),
|
||||
};
|
||||
delete attachmentPayload.progressState;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx.insert(chatPublications).values({
|
||||
companyId: current.companyId,
|
||||
endpointId: current.endpointId,
|
||||
conversationId: current.conversationId,
|
||||
issueId: current.issueId,
|
||||
commentId: current.commentId,
|
||||
idempotencyKey: `discord-markdown-attachment:${current.id}`,
|
||||
payload: attachmentPayload,
|
||||
state: "pending",
|
||||
createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`,
|
||||
updatedAt,
|
||||
});
|
||||
return { ...current, payload: handoffPayload, updatedAt };
|
||||
});
|
||||
}
|
||||
return { prepareDiscordPublication };
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
import { createChatSdkStatePersistence } from "../sdk-state.js";
|
||||
|
||||
import type { DbTransaction, RuntimeContext, LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
} from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
chatConversations,
|
||||
chatExternalPrincipals,
|
||||
chatPublications,
|
||||
issueThreadInteractions,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
AskUserQuestionsInteraction,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
forbidden,
|
||||
} from "../../../errors.js";
|
||||
import { logger } from "../../../middleware/logger.js";
|
||||
import {
|
||||
type ChatSdkCallbackEvent,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
import type {
|
||||
ChatSdkStatePersistence,
|
||||
} from "../../chat-sdk-state.js";
|
||||
|
||||
import {
|
||||
deleteDiscordQuestionFormCorrection,
|
||||
retainDiscordQuestionFormCorrection,
|
||||
} from "../../chat-discord-question-forms.js";
|
||||
|
||||
import { issueThreadInteractionService } from "../../issue-thread-interactions.js";
|
||||
|
||||
import {
|
||||
isChatQuestionFormOpenActionId,
|
||||
loadChatQuestionFormSubmissionToken,
|
||||
resolveChatQuestionFormOpen,
|
||||
} from "../../chat-question-forms.js";
|
||||
import type {
|
||||
ModalResponse,
|
||||
ModalSubmitEvent,
|
||||
} from "chat";
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
|
||||
import type { ChatQuestionFormValidationResult } from "../../chat-question-forms.js";
|
||||
type LoadedForm = NonNullable<Awaited<ReturnType<typeof loadChatQuestionFormSubmissionToken>>>;
|
||||
type DiscordFormActor = { principal: typeof chatExternalPrincipals.$inferSelect; userId: string };
|
||||
type DiscordFormCorrectionInput = {
|
||||
record: EndpointRecord; event: ChatSdkCallbackEvent<ModalSubmitEvent>; runtimeContext: RuntimeContext;
|
||||
conversation: typeof chatConversations.$inferSelect; principal: DiscordFormActor; loaded: LoadedForm;
|
||||
originalPublication: typeof chatPublications.$inferSelect; providerMessageId: string;
|
||||
interaction: AskUserQuestionsInteraction;
|
||||
payload: SafeChatPublicationPayload; validation: Exclude<ChatQuestionFormValidationResult, { ok: true }>;
|
||||
deny: (code: string) => Promise<ModalResponse>;
|
||||
};
|
||||
type DiscordQuestionFormsOptions = {
|
||||
requireCurrentExternalActionAuthorization: (tx: DbTransaction, input: { conversationId: string; endpointId: string; expectedUserId: string; principalId: string; runtimeContext: LifecycleRuntimeFence }) => Promise<void>;
|
||||
isExternalActionAuthorizationChange: (error: unknown) => boolean;
|
||||
persistence: ChatSdkStatePersistence;
|
||||
};
|
||||
export function createDiscordQuestionForms(db: Db, options: DiscordQuestionFormsOptions) {
|
||||
const { requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, persistence } = options;
|
||||
const denialError = () => forbidden("This chat form is not a current Paperclip question");
|
||||
async function handleDiscordFormCorrection(input: DiscordFormCorrectionInput) {
|
||||
const { record, event, runtimeContext, conversation, principal, loaded, originalPublication, providerMessageId, interaction, payload, validation, deny } = input;
|
||||
|
||||
const openAction = payload.card?.actions?.find(
|
||||
(action) =>
|
||||
action.type === "callback" &&
|
||||
isChatQuestionFormOpenActionId(action.actionId),
|
||||
);
|
||||
const openActionId =
|
||||
openAction?.type === "callback" ? openAction.actionId : null;
|
||||
if (!openActionId) return deny("chat_modal_open_token_not_current");
|
||||
try {
|
||||
return await db.transaction(async (tx) => {
|
||||
await requireCurrentExternalActionAuthorization(tx, {
|
||||
conversationId: conversation.id,
|
||||
endpointId: record.endpoint.id,
|
||||
expectedUserId: principal.userId!,
|
||||
principalId: principal.principal.id,
|
||||
runtimeContext,
|
||||
});
|
||||
const pending = await tx
|
||||
.select({ id: issueThreadInteractions.id })
|
||||
.from(issueThreadInteractions)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
issueThreadInteractions.companyId,
|
||||
record.endpoint.companyId,
|
||||
),
|
||||
eq(issueThreadInteractions.issueId, conversation.issueId),
|
||||
eq(issueThreadInteractions.id, interaction.id),
|
||||
eq(issueThreadInteractions.status, "pending"),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0]);
|
||||
const resolved = pending
|
||||
? await resolveChatQuestionFormOpen(tx, {
|
||||
companyId: record.endpoint.companyId,
|
||||
endpointId: record.endpoint.id,
|
||||
conversationId: conversation.id,
|
||||
interaction,
|
||||
openActionId,
|
||||
})
|
||||
: null;
|
||||
if (
|
||||
!resolved ||
|
||||
resolved.submitActionId !== loaded.payload.formActionId ||
|
||||
resolved.publicationId !== originalPublication.id
|
||||
)
|
||||
throw denialError();
|
||||
return retainDiscordQuestionFormCorrection(
|
||||
createChatSdkStatePersistence(tx as unknown as Db),
|
||||
{
|
||||
companyId: record.endpoint.companyId,
|
||||
endpointId: record.endpoint.id,
|
||||
},
|
||||
{
|
||||
principalId: principal.principal.id,
|
||||
userId: principal.userId!,
|
||||
externalUserId: event.event.user.userId,
|
||||
conversationId: conversation.id,
|
||||
publicationId: originalPublication.id,
|
||||
providerMessageId,
|
||||
threadId: conversation.externalThreadId,
|
||||
interactionId: interaction.id,
|
||||
openActionId,
|
||||
submitActionId: resolved.submitActionId,
|
||||
parentExpiresAt: loaded.payload.expiresAt,
|
||||
modal: resolved.modal,
|
||||
values: event.event.values,
|
||||
fieldErrors: validation.fieldErrors,
|
||||
},
|
||||
);
|
||||
});
|
||||
} catch (error) {
|
||||
if (
|
||||
isExternalActionAuthorizationChange(error) ||
|
||||
(error &&
|
||||
typeof error === "object" &&
|
||||
"status" in error &&
|
||||
error.status === 403)
|
||||
)
|
||||
return deny("chat_modal_correction_not_authorized");
|
||||
throw error;
|
||||
}
|
||||
|
||||
}
|
||||
async function acceptDiscordQuestionForm(input: { record: EndpointRecord; event: ChatSdkCallbackEvent<ModalSubmitEvent>; principal: DiscordFormActor; loaded: LoadedForm }): Promise<ModalResponse> {
|
||||
const { record, event, principal, loaded } = input;
|
||||
|
||||
try {
|
||||
await deleteDiscordQuestionFormCorrection(
|
||||
persistence,
|
||||
{
|
||||
companyId: record.endpoint.companyId,
|
||||
endpointId: record.endpoint.id,
|
||||
},
|
||||
{
|
||||
principalId: principal.principal.id,
|
||||
userId: principal.userId!,
|
||||
externalUserId: event.event.user.userId,
|
||||
},
|
||||
loaded.payload.formActionId,
|
||||
);
|
||||
} catch {
|
||||
// The answer receipt is already committed. Cleanup is retryable on
|
||||
// exact replay/expiry access and cannot relabel an accepted answer.
|
||||
logger.warn(
|
||||
{
|
||||
endpointId: record.endpoint.id,
|
||||
code: "discord_question_correction_cleanup_failed",
|
||||
},
|
||||
"Discord correction draft cleanup will be retried",
|
||||
);
|
||||
}
|
||||
|
||||
return { action: "clear" };
|
||||
}
|
||||
return { handleDiscordFormCorrection, acceptDiscordQuestionForm };
|
||||
}
|
||||
@@ -0,0 +1,476 @@
|
||||
import { slackSetupState } from "./slack/setup.js";
|
||||
import { assertChatExecutionDefaultsAccess } from "../chat-execution-defaults.js";
|
||||
|
||||
import type { EndpointRow } from "./runtime-types.js";
|
||||
|
||||
import { chatCredentialMutationLease } from "../chat-credential-mutation-lease.js";
|
||||
|
||||
import { slackChatRegistrationService, slackRegistrationProjection } from "../chat-slack-registration.js";
|
||||
import { chatSlackRegistrations } from "@paperclipai/db";
|
||||
|
||||
import { defaultSlackAppConfiguration } from "@paperclipai/shared";
|
||||
import {
|
||||
randomBytes,
|
||||
randomUUID,
|
||||
} from "node:crypto";
|
||||
import {
|
||||
and,
|
||||
desc,
|
||||
eq,
|
||||
ne,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
agents,
|
||||
chatEndpoints,
|
||||
toolApplications,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatAdapterCapabilities,
|
||||
ChatEndpoint,
|
||||
ChatProvider,
|
||||
CreateChatEndpointInput,
|
||||
UpdateChatEndpointInput,
|
||||
} from "@paperclipai/shared";
|
||||
import {
|
||||
isAgentStatusInvokable,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
badRequest,
|
||||
conflict,
|
||||
notFound,
|
||||
unprocessable,
|
||||
} from "../../errors.js";
|
||||
|
||||
import {
|
||||
logActivity,
|
||||
} from "../activity-log.js";
|
||||
|
||||
import type { EndpointRecord } from "./runtime-types.js";
|
||||
type ChatEndpointsOptions = {
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
CAPABILITIES: Record<ChatProvider, ChatAdapterCapabilities>;
|
||||
PROVIDER_LABELS: Record<ChatProvider, string>;
|
||||
withCredentialMutationLease: ReturnType<typeof chatCredentialMutationLease>;
|
||||
getPublicBaseUrl: () => string | null;
|
||||
getWebhookPublicBaseUrl: () => string | null;
|
||||
slackRegistration: Pick<ReturnType<typeof slackChatRegistrationService>, "cleanup" | "managerGrants">;
|
||||
};
|
||||
export function createChatEndpoints(db: Db, options: ChatEndpointsOptions) {
|
||||
const { endpointRecord, CAPABILITIES, PROVIDER_LABELS, withCredentialMutationLease, getPublicBaseUrl, getWebhookPublicBaseUrl } = options;
|
||||
|
||||
|
||||
function iso(value: Date | null | undefined): string | null {
|
||||
return value ? value.toISOString() : null;
|
||||
}
|
||||
|
||||
function providerSetupState(
|
||||
endpoint: Pick<
|
||||
EndpointRow,
|
||||
| "provider"
|
||||
| "providerAccountId"
|
||||
| "publicId"
|
||||
| "status"
|
||||
| "setup"
|
||||
| "botUsername"
|
||||
>,
|
||||
publicBaseUrl: string | null,
|
||||
assignedAgentName?: string | null,
|
||||
) {
|
||||
const path = endpoint.provider === "speko" ? `/api/voice-webhooks/${endpoint.publicId}/tools` : `/api/chat-webhooks/${endpoint.publicId}/${endpoint.provider}`;
|
||||
const webhookUrl = publicBaseUrl ? `${publicBaseUrl}${path}` : null;
|
||||
const step = endpoint.status === "active" ? "complete" : endpoint.setup.step;
|
||||
switch (endpoint.provider) {
|
||||
case "speko":
|
||||
return { step, providerUrl: "https://platform.speko.ai/", webhookUrl } as const;
|
||||
case "imessage-photon": return { step, providerUrl: "https://photon.codes/", testStartedAt: endpoint.setup.testStartedAt } as const;
|
||||
case "agentmail": return endpoint.setup;
|
||||
case "slack": {
|
||||
return slackSetupState({ endpoint, webhookUrl, step, assignedAgentName });
|
||||
}
|
||||
case "github":
|
||||
return {
|
||||
step,
|
||||
github: endpoint.setup.github,
|
||||
testStartedAt: endpoint.setup.testStartedAt,
|
||||
testSkipped: endpoint.setup.testSkipped,
|
||||
authorizationUrl: "https://github.com/settings/apps/new",
|
||||
providerUrl: "https://github.com/settings/installations",
|
||||
webhookUrl,
|
||||
webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null,
|
||||
} as const;
|
||||
case "discord":
|
||||
return {
|
||||
step,
|
||||
authorizationUrl: "https://discord.com/developers/applications",
|
||||
providerUrl: endpoint.providerAccountId
|
||||
? `https://discord.com/channels/${encodeURIComponent(endpoint.providerAccountId)}`
|
||||
: "https://discord.com/channels/@me",
|
||||
webhookUrl,
|
||||
} as const;
|
||||
case "microsoft-teams":
|
||||
return {
|
||||
step,
|
||||
authorizationUrl: "https://dev.teams.microsoft.com/apps",
|
||||
providerUrl: "https://teams.microsoft.com/",
|
||||
messagingEndpoint: webhookUrl,
|
||||
} as const;
|
||||
case "telegram":
|
||||
return {
|
||||
step,
|
||||
providerUrl: endpoint.botUsername
|
||||
? `https://t.me/${encodeURIComponent(endpoint.botUsername)}`
|
||||
: "https://t.me/BotFather",
|
||||
webhookUrl,
|
||||
} as const;
|
||||
}
|
||||
}
|
||||
|
||||
function serializeEndpoint(
|
||||
row: Awaited<ReturnType<typeof endpointRecord>>,
|
||||
): ChatEndpoint {
|
||||
if (!row) throw notFound("Chat endpoint not found");
|
||||
const endpoint = row.endpoint;
|
||||
return {
|
||||
id: endpoint.id,
|
||||
companyId: endpoint.companyId,
|
||||
connectionId: endpoint.connectionId,
|
||||
publicationMode: endpoint.publicationMode,
|
||||
externalExecutionPolicy: endpoint.externalExecutionPolicy,
|
||||
provider: endpoint.provider,
|
||||
publicId: endpoint.publicId,
|
||||
status: endpoint.status,
|
||||
deploymentMode: endpoint.deploymentMode,
|
||||
assignedAgentId: endpoint.assignedAgentId,
|
||||
assignedAgentName: row.assignedAgentName,
|
||||
sponsorUserId: endpoint.sponsorUserId,
|
||||
providerAccountId: endpoint.providerAccountId,
|
||||
providerAccountLabel: endpoint.providerAccountLabel,
|
||||
botExternalId: endpoint.botExternalId,
|
||||
botUsername: endpoint.botUsername,
|
||||
botLabel: endpoint.botDisplayName ?? row.assignedAgentName,
|
||||
botAvatarUrl: endpoint.botAvatarUrl,
|
||||
communicationInstructions: endpoint.communicationInstructions,
|
||||
executionDefaults: endpoint.executionDefaults,
|
||||
...(endpoint.provider === "imessage-photon" && endpoint.botExternalId ? { photonAllocation: endpoint.botExternalId.startsWith("photon-project:") ? "shared" as const : "dedicated" as const } : {}),
|
||||
allowDirectMessages: endpoint.allowDirectMessages,
|
||||
requireAtMention: endpoint.requireAtMention,
|
||||
allowGroupChats: endpoint.allowGroupChats,
|
||||
allowUnlinkedPeople: endpoint.allowUnlinkedPeople,
|
||||
replyMode: "subscribed",
|
||||
capabilities: endpoint.capabilities,
|
||||
setup: {
|
||||
...providerSetupState(
|
||||
endpoint,
|
||||
getWebhookPublicBaseUrl(),
|
||||
row.assignedAgentName,
|
||||
),
|
||||
...(endpoint.provider === "github"
|
||||
? {
|
||||
webhookSecretConfigured: row.credentialSecretRefs.some(
|
||||
(ref) => ref.configPath === "credentials.webhookSecret",
|
||||
),
|
||||
}
|
||||
: endpoint.provider === "speko" ? {
|
||||
webhookSecretConfigured: row.credentialSecretRefs.some(
|
||||
(ref) => ref.configPath === "credentials.signingSecret",
|
||||
),
|
||||
} : {}),
|
||||
},
|
||||
healthMessage: endpoint.healthMessage,
|
||||
lastError: endpoint.lastError,
|
||||
lastActivityAt: iso(endpoint.lastEventAt),
|
||||
lastPublicationAt: iso(endpoint.lastPublicationAt),
|
||||
activatedAt: iso(endpoint.activatedAt),
|
||||
createdAt: endpoint.createdAt.toISOString(),
|
||||
updatedAt: endpoint.updatedAt.toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
async function list(companyId: string) {
|
||||
const rows = await db
|
||||
.select({
|
||||
endpoint: chatEndpoints,
|
||||
assignedAgentName: agents.name,
|
||||
connectionName: toolConnections.name,
|
||||
applicationId: toolConnections.applicationId,
|
||||
credentialSecretRefs: toolConnections.credentialSecretRefs,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.innerJoin(
|
||||
agents,
|
||||
and(
|
||||
eq(agents.id, chatEndpoints.assignedAgentId),
|
||||
eq(agents.companyId, chatEndpoints.companyId),
|
||||
),
|
||||
)
|
||||
.innerJoin(
|
||||
toolConnections,
|
||||
and(
|
||||
eq(toolConnections.id, chatEndpoints.connectionId),
|
||||
eq(toolConnections.companyId, chatEndpoints.companyId),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.companyId, companyId),
|
||||
ne(chatEndpoints.status, "archived"),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(chatEndpoints.updatedAt));
|
||||
return rows.map((row) => serializeEndpoint(row));
|
||||
}
|
||||
|
||||
async function get(endpointId: string) {
|
||||
const result = serializeEndpoint(await endpointRecord(endpointId));
|
||||
if (result.provider === "slack") {
|
||||
const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId));
|
||||
result.setup.slackRegistration = registration ? slackRegistrationProjection(registration) : undefined;
|
||||
const publicOrigin = getPublicBaseUrl();
|
||||
result.setup.slackOAuthCallbackUri = publicOrigin ? `${publicOrigin}/api/chat-slack/oauth/callback` : null;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function create(
|
||||
companyId: string,
|
||||
input: CreateChatEndpointInput,
|
||||
actorUserId?: string | null,
|
||||
) {
|
||||
if ((input.provider as string) === "agentmail") throw badRequest("Use the email inbox setup API for AgentMail");
|
||||
if ((input.slackApp || input.slackSetupMethod) && input.provider !== "slack") throw unprocessable("Slack app details only apply to Slack connections");
|
||||
if (input.slackSetupMethod === "managed" && options.slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable");
|
||||
const agent = await db
|
||||
.select({ id: agents.id, name: agents.name, status: agents.status })
|
||||
.from(agents)
|
||||
.where(
|
||||
and(
|
||||
eq(agents.companyId, companyId),
|
||||
eq(agents.id, input.assignedAgentId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!agent)
|
||||
throw unprocessable("The selected agent does not belong to this company");
|
||||
if (!isAgentStatusInvokable(agent.status)) {
|
||||
throw unprocessable(
|
||||
"The selected agent must be active before it can be connected to chat",
|
||||
{
|
||||
code: "chat_agent_not_invokable",
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const endpointId = randomUUID();
|
||||
const publicId = randomBytes(32).toString("base64url");
|
||||
const applicationId = input.applicationId ?? randomUUID();
|
||||
const connectionId = randomUUID();
|
||||
const suffix = endpointId.slice(0, 8);
|
||||
const name = (
|
||||
input.name?.trim() || `${PROVIDER_LABELS[input.provider]} — ${agent.name}`
|
||||
).slice(0, 145);
|
||||
await db.transaction(async (tx) => {
|
||||
if (input.applicationId) {
|
||||
const application = await tx
|
||||
.select({
|
||||
id: toolApplications.id,
|
||||
applicationKey: toolApplications.applicationKey,
|
||||
metadata: toolApplications.metadata,
|
||||
status: toolApplications.status,
|
||||
})
|
||||
.from(toolApplications)
|
||||
.where(
|
||||
and(
|
||||
eq(toolApplications.companyId, companyId),
|
||||
eq(toolApplications.id, input.applicationId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!application) throw notFound("App not found");
|
||||
const sourceTemplateKey =
|
||||
typeof application.metadata.sourceTemplateKey === "string"
|
||||
? application.metadata.sourceTemplateKey
|
||||
: null;
|
||||
if (
|
||||
application.status === "archived" ||
|
||||
(application.applicationKey !== input.provider &&
|
||||
sourceTemplateKey !== input.provider)
|
||||
) {
|
||||
throw unprocessable(
|
||||
`The selected App is not the ${PROVIDER_LABELS[input.provider]} connector`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
await tx.insert(toolApplications).values({
|
||||
id: applicationId,
|
||||
companyId,
|
||||
applicationKey: `chat:${input.provider}:${endpointId}`,
|
||||
name: `${name} ${suffix}`,
|
||||
description: `People talk to ${agent.name} through ${PROVIDER_LABELS[input.provider]}.`,
|
||||
type: "chat",
|
||||
status: "draft",
|
||||
metadata: { sourceTemplateKey: input.provider, purpose: "channel" },
|
||||
ownerUserId: actorUserId ?? null,
|
||||
});
|
||||
}
|
||||
await tx.insert(toolConnections).values({
|
||||
id: connectionId,
|
||||
companyId,
|
||||
applicationId,
|
||||
name,
|
||||
uid: `chat-${input.provider}-${endpointId}`,
|
||||
connectionKind: "managed",
|
||||
connectionPurpose: "channel",
|
||||
ownership: "customer",
|
||||
transport: input.provider === "speko" ? "voice" : "chat_sdk",
|
||||
authKind: "api_key",
|
||||
credentialPolicy: "shared",
|
||||
status: "draft",
|
||||
enabled: false,
|
||||
config: { provider: input.provider },
|
||||
transportConfig: {},
|
||||
createdByUserId: actorUserId ?? null,
|
||||
});
|
||||
await tx.insert(chatEndpoints).values({
|
||||
id: endpointId,
|
||||
companyId,
|
||||
connectionId,
|
||||
provider: input.provider,
|
||||
publicId,
|
||||
assignedAgentId: agent.id,
|
||||
sponsorUserId: actorUserId ?? null,
|
||||
// Group chat is a broad Teams reach grant rather than a discovered
|
||||
// destination toggle. Keep it closed until the operator explicitly
|
||||
// enables that surface, even when this process is running against a
|
||||
// database created before the column default was hardened.
|
||||
requireAtMention: input.provider === "slack",
|
||||
allowGroupChats: !["microsoft-teams", "speko"].includes(input.provider),
|
||||
allowUnlinkedPeople: !["slack", "imessage-photon", "speko"].includes(input.provider),
|
||||
capabilities: CAPABILITIES[input.provider],
|
||||
setup: {
|
||||
step: "provider_setup",
|
||||
...(input.provider === "slack"
|
||||
? { slackApp: input.slackApp ?? defaultSlackAppConfiguration(agent.name),
|
||||
command: (input.slackApp ?? defaultSlackAppConfiguration(agent.name)).command, slackSetupMethod: input.slackSetupMethod ?? options.slackRegistration.managerGrants.defaultMethod() }
|
||||
: {}),
|
||||
},
|
||||
});
|
||||
});
|
||||
await logActivity(db, {
|
||||
companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action: "chat_endpoint.created",
|
||||
entityType: "tool_connection",
|
||||
entityId: connectionId,
|
||||
details: {
|
||||
endpointId,
|
||||
provider: input.provider,
|
||||
assignedAgentId: agent.id,
|
||||
},
|
||||
});
|
||||
return get(endpointId);
|
||||
}
|
||||
|
||||
async function update(
|
||||
endpointId: string,
|
||||
input: UpdateChatEndpointInput,
|
||||
actorUserId?: string | null,
|
||||
) {
|
||||
const initial = await endpointRecord(endpointId);
|
||||
if (!initial) throw notFound("Chat endpoint not found");
|
||||
if (initial.endpoint.provider === "agentmail" && Object.keys(input).some((key) => key !== "executionDefaults")) throw badRequest("Use the email inbox API for AgentMail");
|
||||
await withCredentialMutationLease(
|
||||
initial.endpoint,
|
||||
async (credentialLease) => {
|
||||
// Reach changes share the provider-transport lane with publications.
|
||||
// Once a disable returns, no send that sampled the previous grant can
|
||||
// still begin or finish behind it.
|
||||
const existing = await endpointRecord(endpointId);
|
||||
if (!existing) throw notFound("Chat endpoint not found");
|
||||
const values: Partial<typeof chatEndpoints.$inferInsert> = {
|
||||
updatedAt: new Date(),
|
||||
};
|
||||
if (input.communicationInstructions !== undefined) {
|
||||
if (existing.endpoint.provider !== "slack") {
|
||||
throw unprocessable("Communication instructions are currently supported for Slack connections");
|
||||
}
|
||||
values.communicationInstructions = input.communicationInstructions;
|
||||
}
|
||||
if (input.slackSetupMethod === "managed" && options.slackRegistration.managerGrants.defaultMethod() !== "managed") throw conflict("Managed Slack setup is unavailable");
|
||||
if (input.slackSetupMethod && (existing.endpoint.provider !== "slack" || existing.endpoint.botExternalId || existing.endpoint.status !== "draft")) {
|
||||
throw conflict("Setup method can only change before connecting the app");
|
||||
}
|
||||
if (input.slackApp || input.slackSetupMethod) {
|
||||
const [registration] = await db.select().from(chatSlackRegistrations).where(eq(chatSlackRegistrations.endpointId, endpointId));
|
||||
if (registration && (registration.managerGrantId || registration.status !== "failed")) {
|
||||
if (registration.managerGrantId || input.slackSetupMethod === "managed" || input.slackApp || input.slackSetupMethod === "automatic") throw conflict("Slack app details are locked after creation starts");
|
||||
// Manual recovery fences pending authorization under the same credential lease.
|
||||
await options.slackRegistration.cleanup(endpointId, credentialLease);
|
||||
}
|
||||
}
|
||||
if (input.slackSetupMethod) {
|
||||
values.setup = { ...existing.endpoint.setup, slackSetupMethod: input.slackSetupMethod };
|
||||
}
|
||||
if (input.slackApp) {
|
||||
if (existing.endpoint.provider !== "slack") {
|
||||
throw unprocessable("Slack app details only apply to Slack connections");
|
||||
}
|
||||
if (existing.endpoint.status !== "draft" || existing.endpoint.botExternalId) {
|
||||
throw conflict("Slack app details can only be edited before connecting the app");
|
||||
}
|
||||
values.setup = {
|
||||
...(values.setup ?? existing.endpoint.setup),
|
||||
slackApp: input.slackApp,
|
||||
command: input.slackApp.command,
|
||||
};
|
||||
}
|
||||
if (input.allowDirectMessages !== undefined)
|
||||
values.allowDirectMessages = input.allowDirectMessages;
|
||||
if (input.requireAtMention !== undefined) {
|
||||
if (existing.endpoint.provider !== "slack")
|
||||
throw unprocessable("Require at-mention only applies to Slack connections");
|
||||
values.requireAtMention = input.requireAtMention;
|
||||
}
|
||||
if (input.allowGroupChats && existing.endpoint.provider === "imessage-photon" && (!existing.endpoint.botExternalId || existing.endpoint.botExternalId.startsWith("photon-project:")))
|
||||
throw unprocessable("Photon shared channels support direct messages only; groups require a dedicated channel");
|
||||
if (input.allowGroupChats !== undefined)
|
||||
values.allowGroupChats = input.allowGroupChats;
|
||||
if (input.allowUnlinkedPeople !== undefined)
|
||||
values.allowUnlinkedPeople = input.allowUnlinkedPeople;
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
if (input.executionDefaults !== undefined) {
|
||||
await assertChatExecutionDefaultsAccess(db, {
|
||||
companyId: existing.endpoint.companyId,
|
||||
assigneeAgentId: values.assignedAgentId ?? existing.endpoint.assignedAgentId,
|
||||
actor: { type: "board", userId: actorUserId, source: actorUserId ? "session" : "local_implicit" },
|
||||
defaults: input.executionDefaults,
|
||||
}, tx);
|
||||
values.executionDefaults = input.executionDefaults;
|
||||
}
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set(values)
|
||||
.where(eq(chatEndpoints.id, endpointId));
|
||||
await logActivity(tx as unknown as Db, {
|
||||
companyId: existing.endpoint.companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action: "chat_endpoint.updated",
|
||||
entityType: "tool_connection",
|
||||
entityId: existing.endpoint.connectionId,
|
||||
details: { endpointId, fields: Object.keys(input) },
|
||||
});
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
},
|
||||
);
|
||||
return get(endpointId);
|
||||
}
|
||||
return { list, get, create, update };
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
chatDeliveries,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import type {
|
||||
Message,
|
||||
} from "chat";
|
||||
|
||||
type GitHubMessageProvenanceOptions = {
|
||||
githubManualMessages: WeakMap<object, { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }>;
|
||||
githubAutomaticMessages: WeakMap<object, { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
githubIssueMessages: WeakMap<object, { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
};
|
||||
export function createGitHubMessageProvenance(options: GitHubMessageProvenanceOptions) {
|
||||
const { githubManualMessages, githubAutomaticMessages, githubIssueMessages } = options;
|
||||
function restoreGitHubMessageProvenance(delivery: typeof chatDeliveries.$inferSelect, endpointRuntime: Pick<ChatEndpointRuntime, "provider">, message: Message): void {
|
||||
const githubManual = delivery.normalizedEvent.githubManual;
|
||||
if (endpointRuntime.provider === "github" && githubManual && typeof githubManual === "object") githubManualMessages.set(message, githubManual as { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" });
|
||||
const githubAutomatic = delivery.normalizedEvent.githubAutomatic;
|
||||
if (endpointRuntime.provider === "github" && githubAutomatic && typeof githubAutomatic === "object") {
|
||||
githubAutomaticMessages.set(message, githubAutomatic as { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy });
|
||||
}
|
||||
const githubIssue = delivery.normalizedEvent.githubIssue;
|
||||
if (endpointRuntime.provider === "github" && githubIssue && typeof githubIssue === "object") {
|
||||
githubIssueMessages.set(message, githubIssue as { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy });
|
||||
}
|
||||
}
|
||||
return { restoreGitHubMessageProvenance };
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
import { createGitHubAppSetup } from "./app-setup.js";
|
||||
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import { githubAppJwt } from "../../chat-github-client.js";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
forbidden,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import {
|
||||
logActivity,
|
||||
} from "../../activity-log.js";
|
||||
import {
|
||||
resyncGitHubAppWebhook,
|
||||
} from "../../chat-github-webhook-config.js";
|
||||
|
||||
type GitHubSetupOptions = {
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
getWebhookPublicBaseUrl: () => string | null;
|
||||
githubWizard: Pick<ReturnType<typeof createGitHubAppSetup>["githubWizard"], "repairCloud">;
|
||||
};
|
||||
export function createGitHubSetup(db: Db, options: GitHubSetupOptions) {
|
||||
const { fetchImpl, getWebhookPublicBaseUrl } = options;
|
||||
|
||||
async function repairGitHubCloudSetup(endpoint: EndpointRow, credentialLease: CredentialMutationLeaseGuard, actorUserId?: string | null) {
|
||||
if (!actorUserId) throw forbidden("A configuring member is required to recover this App");
|
||||
await credentialLease.assertOwned();
|
||||
await options.githubWizard.repairCloud(endpoint.id, actorUserId);
|
||||
await credentialLease.assertOwned();
|
||||
}
|
||||
|
||||
async function repairGitHubWebhookSetup(endpoint: EndpointRow, credentials: Record<string, string>, credentialLease: CredentialMutationLeaseGuard, actorUserId?: string | null) {
|
||||
// The immutable App and its installation were verified above. Repair
|
||||
// only its callback using the stored secret; never reinstall it or
|
||||
// change repository access. Keep historical signed-ping evidence, but
|
||||
// require the fresh test round trip established above before activation.
|
||||
const auditWebhookSync = async (action: string) => {
|
||||
await db.transaction(async (tx) => {
|
||||
await credentialLease.assertOwned(tx);
|
||||
await logActivity(tx as unknown as Db, {
|
||||
companyId: endpoint.companyId,
|
||||
actorType: "user",
|
||||
actorId: actorUserId ?? "board",
|
||||
action,
|
||||
entityType: "tool_connection",
|
||||
entityId: endpoint.connectionId,
|
||||
details: { endpointId: endpoint.id, provider: "github" },
|
||||
});
|
||||
await credentialLease.assertOwned(tx);
|
||||
});
|
||||
};
|
||||
await auditWebhookSync("chat_endpoint.webhook_sync_started");
|
||||
await credentialLease.assertOwned();
|
||||
await resyncGitHubAppWebhook({
|
||||
fetch: fetchImpl,
|
||||
appToken: githubAppJwt(credentials.appId, credentials.privateKey),
|
||||
webhookUrl: `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/github`,
|
||||
webhookSecret: credentials.webhookSecret,
|
||||
});
|
||||
await auditWebhookSync("chat_endpoint.webhook_synced");
|
||||
}
|
||||
|
||||
return { repairGitHubCloudSetup, repairGitHubWebhookSetup };
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import type { Thread } from "chat";
|
||||
|
||||
import { chatEndpointResources } from "@paperclipai/db";
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js";
|
||||
|
||||
type DestinationAdmissionInput = {
|
||||
endpoint: EndpointRow;
|
||||
currentEndpoint: Pick<EndpointRow, "status" | "allowGroupChats">;
|
||||
thread: Thread;
|
||||
|
||||
};
|
||||
type DestinationAdmissionOptions = {
|
||||
ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise<typeof chatEndpointResources.$inferSelect>;
|
||||
};
|
||||
export function createPhotonDestinationAdmission(options: DestinationAdmissionOptions) {
|
||||
const { ensureResource } = options;
|
||||
async function admitPhotonDestination(input: DestinationAdmissionInput, tx: DbTransaction) {
|
||||
const { endpoint, currentEndpoint, thread } = input;
|
||||
let destinationAccepting = true;
|
||||
|
||||
const resource = await ensureResource(endpoint, thread, false, tx);
|
||||
destinationAccepting =
|
||||
resource.enabled &&
|
||||
resource.availability === "available" &&
|
||||
currentEndpoint.allowGroupChats;
|
||||
return destinationAccepting;
|
||||
}
|
||||
return { admitPhotonDestination };
|
||||
}
|
||||
@@ -1,4 +1,285 @@
|
||||
import type { SafeChatPublicationPayload } from "@paperclipai/shared";
|
||||
import { createChatPublicationReplacement } from "../publication-replacement.js";
|
||||
|
||||
import { type ChatRuntime } from "../../chat-runtime.js";
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
|
||||
import { nativePhotonInteraction, publishPhotonPrompt, type PhotonPromptReceipt, type PhotonInteractionBinding } from "../../photon/interactions.js";
|
||||
import { PhotonError } from "../../photon/cloud.js";
|
||||
import { PhotonChatAdapter } from "../../photon/adapter.js";
|
||||
import {
|
||||
and,
|
||||
desc,
|
||||
eq,
|
||||
or,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
chatActions,
|
||||
chatEndpointResources,
|
||||
chatMessageLinks,
|
||||
heartbeatRuns,
|
||||
issueThreadInteractions,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatEndpointSetupState,
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
type ChatSdkRuntime,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
import { safeChatTaskUrl } from "../../chat-task-url.js";
|
||||
|
||||
import { issueThreadInteractionService } from "../../issue-thread-interactions.js";
|
||||
|
||||
import type {
|
||||
FileUpload,
|
||||
} from "chat";
|
||||
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
|
||||
import type { PublicationTransportInput } from "../publications/types.js";
|
||||
import type { Thread } from "chat";
|
||||
type PhotonPublicationDeliveryOptions = {
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
runtimeGeneration: (setup: ChatEndpointSetupState) => number;
|
||||
runtime: Pick<ChatRuntime | ChatSdkRuntime, "get">;
|
||||
receiptReactionCompletionRunId: ReturnType<typeof createChatPublicationReplacement>["receiptReactionCompletionRunId"];
|
||||
getTaskBaseUrl: () => string | null | undefined;
|
||||
};
|
||||
export function createPhotonPublicationDelivery(db: Db, options: PhotonPublicationDeliveryOptions) {
|
||||
const { endpointRecord, runtimeGeneration, runtime, receiptReactionCompletionRunId, getTaskBaseUrl } = options;
|
||||
async function deliverPhotonPublication(input: PublicationTransportInput, endpointRuntime: ChatEndpointRuntime, thread: Pick<Thread, "id">, text: string, files: FileUpload[]) {
|
||||
const adapter = endpointRuntime.getProviderAdapter();
|
||||
if (!(adapter instanceof PhotonChatAdapter))
|
||||
throw new Error("Photon publication runtime unavailable");
|
||||
const assertCurrent = async () => {
|
||||
await input.beforePhotonWrite?.();
|
||||
const current = await endpointRecord(input.endpoint.id);
|
||||
if (
|
||||
!current ||
|
||||
!["verifying", "active"].includes(current.endpoint.status) ||
|
||||
runtimeGeneration(current.endpoint.setup) !==
|
||||
runtimeGeneration(input.endpoint.setup) ||
|
||||
runtime.get(input.endpoint.id) !== endpointRuntime
|
||||
)
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"Photon publication authority changed",
|
||||
);
|
||||
const resource = await db
|
||||
.select()
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.companyId, current.endpoint.companyId),
|
||||
eq(chatEndpointResources.endpointId, current.endpoint.id),
|
||||
eq(chatEndpointResources.id, input.conversation.resourceId!),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]);
|
||||
if (
|
||||
resource?.availability !== "available" ||
|
||||
(!input.conversation.isDirectMessage && !resource.enabled) ||
|
||||
(input.conversation.isDirectMessage &&
|
||||
!current.endpoint.allowDirectMessages)
|
||||
)
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"Photon conversation is no longer enabled",
|
||||
);
|
||||
};
|
||||
const retry = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.endpointId, input.endpoint.id),
|
||||
eq(chatActions.kind, "photon_publication_retry"),
|
||||
eq(
|
||||
chatActions.providerActionId,
|
||||
`photon-retry:${input.publication.id}:${input.publication.attempts + 1}`,
|
||||
),
|
||||
eq(chatActions.status, "processed"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]);
|
||||
const retryUnknown = Boolean(retry);
|
||||
const nextQuestion =
|
||||
/^photon-question:([a-zA-Z0-9_-]{8,24}):(\d{1,2})$/.exec(
|
||||
input.publication.idempotencyKey,
|
||||
);
|
||||
if (
|
||||
input.payload.interactionId &&
|
||||
(input.publication.idempotencyKey ===
|
||||
`interaction:${input.payload.interactionId}:${input.endpoint.id}` ||
|
||||
nextQuestion)
|
||||
) {
|
||||
const action = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, input.endpoint.companyId),
|
||||
eq(chatActions.endpointId, input.endpoint.id),
|
||||
eq(chatActions.kind, "photon_interaction"),
|
||||
nextQuestion
|
||||
? eq(chatActions.providerActionId, `photon:${nextQuestion[1]}`)
|
||||
: eq(
|
||||
sql<string>`${chatActions.payload}->>'publicationId'`,
|
||||
input.publication.id,
|
||||
),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]);
|
||||
const interaction = (
|
||||
await issueThreadInteractionService(db).listForIssue(
|
||||
input.publication.issueId,
|
||||
)
|
||||
).find((candidate) => candidate.id === input.payload.interactionId);
|
||||
if (
|
||||
action &&
|
||||
interaction?.status === "pending" &&
|
||||
nativePhotonInteraction(interaction)
|
||||
) {
|
||||
const promptGuard = async () => {
|
||||
await assertCurrent();
|
||||
const current = await db
|
||||
.select({ status: issueThreadInteractions.status })
|
||||
.from(issueThreadInteractions)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
issueThreadInteractions.companyId,
|
||||
input.endpoint.companyId,
|
||||
),
|
||||
eq(issueThreadInteractions.id, interaction.id),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]);
|
||||
if (current?.status !== "pending")
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"This interaction has already been resolved",
|
||||
);
|
||||
};
|
||||
const receipt = await publishPhotonPrompt({
|
||||
adapter,
|
||||
threadId: thread.id,
|
||||
binding: action.payload as unknown as PhotonInteractionBinding,
|
||||
interaction,
|
||||
questionIndex: nextQuestion ? Number(nextQuestion[2]) : 0,
|
||||
taskUrl: safeChatTaskUrl(
|
||||
getTaskBaseUrl(),
|
||||
input.publication.issueId,
|
||||
),
|
||||
assertCurrent: promptGuard,
|
||||
retryUnknown,
|
||||
});
|
||||
return { id: receipt.promptMessageGuid };
|
||||
}
|
||||
if (nextQuestion)
|
||||
throw new PhotonError(
|
||||
"rejected",
|
||||
"This question is no longer available",
|
||||
);
|
||||
}
|
||||
const reply = await adapter.state.read<{ guid: string | null }>(
|
||||
`publication-reply:${input.publication.id}`,
|
||||
);
|
||||
let replyTo = reply?.guid ?? null;
|
||||
if (!reply) {
|
||||
const sourceRunId = await receiptReactionCompletionRunId(
|
||||
db,
|
||||
input.publication,
|
||||
input.payload,
|
||||
);
|
||||
if (sourceRunId) {
|
||||
const source = await db
|
||||
.select({ guid: chatMessageLinks.providerMessageId })
|
||||
.from(heartbeatRuns)
|
||||
.innerJoin(
|
||||
chatMessageLinks,
|
||||
and(
|
||||
eq(chatMessageLinks.companyId, input.endpoint.companyId),
|
||||
eq(chatMessageLinks.endpointId, input.endpoint.id),
|
||||
eq(chatMessageLinks.conversationId, input.conversation.id),
|
||||
eq(chatMessageLinks.direction, "inbound"),
|
||||
or(
|
||||
sql`${chatMessageLinks.commentId}::text = ${heartbeatRuns.contextSnapshot}->>'wakeCommentId'`,
|
||||
sql`coalesce(${heartbeatRuns.contextSnapshot}->'wakeCommentIds', '[]'::jsonb) ? ${chatMessageLinks.commentId}::text`,
|
||||
),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRuns.id, sourceRunId),
|
||||
eq(heartbeatRuns.companyId, input.endpoint.companyId),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(chatMessageLinks.createdAt))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0]);
|
||||
replyTo = source?.guid ?? null;
|
||||
}
|
||||
replyTo = (
|
||||
await adapter.state.update<{ guid: string | null }>(
|
||||
`publication-reply:${input.publication.id}`,
|
||||
(current) => current ?? { guid: replyTo },
|
||||
)
|
||||
).guid;
|
||||
}
|
||||
const sent = await adapter.publish(
|
||||
thread.id,
|
||||
input.publication.id,
|
||||
{ markdown: text, files },
|
||||
{ assertCurrent, retryUnknown, replyTo: replyTo ?? undefined },
|
||||
);
|
||||
if (input.publication.idempotencyKey.startsWith("photon-response:")) {
|
||||
const notice = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, input.endpoint.companyId),
|
||||
eq(chatActions.endpointId, input.endpoint.id),
|
||||
eq(chatActions.kind, "photon_response_notice"),
|
||||
eq(
|
||||
chatActions.providerActionId,
|
||||
`photon-notice:${input.publication.idempotencyKey}`,
|
||||
),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]);
|
||||
if (
|
||||
notice &&
|
||||
typeof notice.payload.reference === "string" &&
|
||||
typeof notice.payload.questionIndex === "number" &&
|
||||
typeof notice.payload.publicationId === "string"
|
||||
) {
|
||||
const receipt: PhotonPromptReceipt = {
|
||||
schema: 1,
|
||||
reference: notice.payload.reference,
|
||||
questionIndex: notice.payload.questionIndex,
|
||||
publicationId: notice.payload.publicationId,
|
||||
promptMessageGuid: sent.id,
|
||||
promptMessageGuids: sent.messageIds,
|
||||
options: {},
|
||||
};
|
||||
for (const guid of sent.messageIds)
|
||||
await adapter.state.update<PhotonPromptReceipt>(
|
||||
`prompt-message:${guid}`,
|
||||
(current) => current ?? receipt,
|
||||
);
|
||||
}
|
||||
}
|
||||
return sent;
|
||||
}
|
||||
return { deliverPhotonPublication };
|
||||
}
|
||||
|
||||
|
||||
export function imessageProgressSupersessionReason(
|
||||
progress: SafeChatPublicationPayload["progressState"],
|
||||
@@ -6,4 +287,4 @@ export function imessageProgressSupersessionReason(
|
||||
return progress && ["queued", "working"].includes(progress)
|
||||
? "iMessage uses typing instead of progress bubbles"
|
||||
: null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
import type { createPhotonCredentialChecks } from "./credentials.js";
|
||||
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import { photonSharedScope, photonSharedIdentity } from "../../photon/cloud.js";
|
||||
import { photonChannelConfigurationSchema } from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
ne,
|
||||
} from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
chatEndpoints,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ConfigureChatEndpointInput,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
conflict,
|
||||
notFound,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
|
||||
type PhotonSetupInspectionOptions = {
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
inspectPhotonCredentials: ReturnType<typeof createPhotonCredentialChecks>["inspectPhotonCredentials"];
|
||||
};
|
||||
export function createPhotonSetupInspection(db: Db, options: PhotonSetupInspectionOptions) {
|
||||
const { endpointRecord, inspectPhotonCredentials } = options;
|
||||
|
||||
|
||||
async function inspectPhoton(
|
||||
endpointId: string,
|
||||
input: { projectId: string; projectSecret: string },
|
||||
) {
|
||||
const record = await endpointRecord(endpointId);
|
||||
if (!record || record.endpoint.provider !== "imessage-photon")
|
||||
throw notFound("iMessage Photon endpoint not found");
|
||||
const inspection = structuredClone(await inspectPhotonCredentials(
|
||||
input.projectId,
|
||||
input.projectSecret,
|
||||
));
|
||||
const reserved = await db
|
||||
.select({ number: chatEndpoints.botExternalId })
|
||||
.from(chatEndpoints)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.provider, "imessage-photon"),
|
||||
ne(chatEndpoints.id, endpointId),
|
||||
ne(chatEndpoints.status, "archived"),
|
||||
),
|
||||
);
|
||||
for (const line of inspection.lines) {
|
||||
if (reserved.some((row) => row.number === line.phoneNumber)) {
|
||||
line.eligible = false;
|
||||
line.unavailableReason =
|
||||
"This number already belongs to another channel";
|
||||
}
|
||||
}
|
||||
inspection.eligible = inspection.allocation === "shared" ? inspection.eligible && !reserved.some((row) => row.number === photonSharedIdentity(inspection.projectId)) : inspection.lines.some((line) => line.eligible);
|
||||
return inspection;
|
||||
}
|
||||
return { inspectPhoton };
|
||||
}
|
||||
export function preparePhotonConfiguration(endpoint: EndpointRow, input: ConfigureChatEndpointInput, credentials: Record<string, string>): Record<string, string> {
|
||||
const configuration = photonChannelConfigurationSchema.parse(input.photon ?? (credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { projectId: credentials.projectId, lineId: credentials.lineId }));
|
||||
const lineId = configuration.allocation === "shared" ? photonSharedScope(configuration.projectId) : configuration.lineId;
|
||||
if (endpoint.botExternalId && (configuration.projectId !== endpoint.providerAccountId || credentials.lineId && lineId !== credentials.lineId)) throw conflict("A different Photon identity requires a new channel");
|
||||
return { ...credentials, ...configuration, lineId };
|
||||
}
|
||||
|
||||
export function createPhotonConfiguration(db: Db, options: { invalidateRuntime: (endpointId: string) => Promise<boolean> }) {
|
||||
const { invalidateRuntime } = options;
|
||||
async function persistPhotonConfiguration(endpoint: EndpointRow, credentials: Record<string, string>, credentialLease: CredentialMutationLeaseGuard) {
|
||||
await invalidateRuntime(endpoint.id);
|
||||
await credentialLease.assertOwned();
|
||||
if (credentials.allocation === "shared") await db.update(chatEndpoints).set({ allowGroupChats: false }).where(eq(chatEndpoints.id, endpoint.id));
|
||||
await db.update(toolConnections).set({ config: { provider: endpoint.provider, photon: credentials.allocation === "shared" ? { allocation: "shared", projectId: credentials.projectId } : { allocation: "dedicated", projectId: credentials.projectId, lineId: credentials.lineId } } }).where(and(eq(toolConnections.companyId, endpoint.companyId), eq(toolConnections.id, endpoint.connectionId)));
|
||||
}
|
||||
return { persistPhotonConfiguration };
|
||||
}
|
||||
@@ -0,0 +1,399 @@
|
||||
import { createTeamsDestinationAdmission } from "../microsoft-teams/destination-admission.js";
|
||||
import { createTelegramDestinationAdmission } from "../telegram/destination-admission.js";
|
||||
import { createPhotonDestinationAdmission } from "../imessage-photon/destination-admission.js";
|
||||
|
||||
import { deliveryContentWasRedacted } from "./delivery-ledger.js";
|
||||
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction, InboundRuntimeContext } from "../runtime-types.js";
|
||||
|
||||
import { notifyChatDeliveryWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import {
|
||||
githubMessageAddressesAnotherBot,
|
||||
} from "../../chat-github-events.js";
|
||||
|
||||
import {
|
||||
createHash,
|
||||
} from "node:crypto";
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatDeliveries,
|
||||
chatEndpointResources,
|
||||
chatEndpoints,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatEndpointSetupState,
|
||||
ToolCredentialSecretRef,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import type {
|
||||
Message,
|
||||
Thread,
|
||||
} from "chat";
|
||||
|
||||
import type { DiscordGatewayOwnership } from "../runtime-types.js";
|
||||
import type { InboundEnvelope } from "./envelope.js";
|
||||
import type { ChatSdkMessageCallbackEvent } from "../../chat-sdk-runtime.js";
|
||||
|
||||
type InboundAdmissionInput = {
|
||||
endpoint: EndpointRow;
|
||||
thread: Thread;
|
||||
message: Message;
|
||||
trigger: ChatSdkMessageCallbackEvent["trigger"];
|
||||
runtimeContext?: InboundRuntimeContext;
|
||||
admittedDeliveryId: string | null;
|
||||
deferDrainUntilFollowup: boolean;
|
||||
scheduledAt: number | null;
|
||||
envelope: Pick<InboundEnvelope, "providerEventId" | "addressed" | "eventKind" | "normalized" | "redactedDestinationNormalized" | "githubAutomatic" | "githubIssue">;
|
||||
};
|
||||
type GatewayAdmissionState = {
|
||||
lost?: DiscordGatewayOwnership;
|
||||
renewed?: { expiresAt: Date; ownership: DiscordGatewayOwnership };
|
||||
};
|
||||
type InboundAdmissionOptions = {
|
||||
runtimeGeneration: (setup: ChatEndpointSetupState) => number;
|
||||
credentialFingerprint: (refs: ToolCredentialSecretRef[]) => string;
|
||||
leasedChatProvider: (provider: string) => boolean;
|
||||
renewDiscordGatewayOwnershipForMessageAdmission: (tx: DbTransaction, endpointId: string, context: InboundRuntimeContext) => Promise<{ kind: "owned"; expiresAt: Date; ownership: DiscordGatewayOwnership } | { kind: "lost"; ownership: DiscordGatewayOwnership | null }>;
|
||||
ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise<typeof chatEndpointResources.$inferSelect>;
|
||||
DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS: number;
|
||||
};
|
||||
export function createInboundAdmission(db: Db, options: InboundAdmissionOptions) {
|
||||
const { runtimeGeneration, credentialFingerprint, leasedChatProvider, renewDiscordGatewayOwnershipForMessageAdmission, ensureResource, DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS } = options;
|
||||
const { admitTeamsDestination } = createTeamsDestinationAdmission({ ensureResource });
|
||||
const { admitTelegramDestination } = createTelegramDestinationAdmission({ ensureResource });
|
||||
const { admitPhotonDestination } = createPhotonDestinationAdmission({ ensureResource });
|
||||
async function admitInboundMessage(input: InboundAdmissionInput, discordGatewayAdmissionState: GatewayAdmissionState) {
|
||||
const { endpoint, thread, message, trigger, runtimeContext, admittedDeliveryId, deferDrainUntilFollowup, scheduledAt, envelope } = input;
|
||||
const { providerEventId, addressed, eventKind, normalized, redactedDestinationNormalized, githubAutomatic, githubIssue } = envelope;
|
||||
return await db.transaction(async (tx) => {
|
||||
const currentEndpoint = await tx
|
||||
.select({
|
||||
status: chatEndpoints.status,
|
||||
setup: chatEndpoints.setup,
|
||||
allowDirectMessages: chatEndpoints.allowDirectMessages,
|
||||
requireAtMention: chatEndpoints.requireAtMention,
|
||||
allowGroupChats: chatEndpoints.allowGroupChats,
|
||||
botUsername: chatEndpoints.botUsername,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!currentEndpoint) return null;
|
||||
// An SDK callback can finish parsing after a pause/resume cycle. Each
|
||||
// runtime captures the durable endpoint generation at registration;
|
||||
// resume advances it under the same row lock used by admission. This
|
||||
// rejects stale callbacks without relying on provider/server clock sync.
|
||||
const currentRuntimeGeneration = runtimeGeneration(currentEndpoint.setup);
|
||||
const currentCredentialFingerprint = runtimeContext
|
||||
? await tx
|
||||
.select({ refs: toolConnections.credentialSecretRefs })
|
||||
.from(toolConnections)
|
||||
.where(
|
||||
and(
|
||||
eq(toolConnections.companyId, endpoint.companyId),
|
||||
eq(toolConnections.id, endpoint.connectionId),
|
||||
),
|
||||
)
|
||||
.then((rows) => credentialFingerprint(rows[0]?.refs ?? []))
|
||||
: null;
|
||||
const staleActivation =
|
||||
runtimeContext !== undefined &&
|
||||
(runtimeContext.generation !== currentRuntimeGeneration ||
|
||||
runtimeContext.credentialFingerprint !==
|
||||
currentCredentialFingerprint);
|
||||
const endpointAccepting =
|
||||
["verifying", "active"].includes(currentEndpoint.status) &&
|
||||
!staleActivation;
|
||||
if (
|
||||
endpointAccepting &&
|
||||
leasedChatProvider(endpoint.provider) &&
|
||||
runtimeContext &&
|
||||
!admittedDeliveryId
|
||||
) {
|
||||
// The first process-local ownership check can precede a host pause.
|
||||
// Reclaim the same durable token while the endpoint admission row is
|
||||
// locked so a standby takeover cannot be followed by an obsolete
|
||||
// callback write. This renews only the exact token that authenticated
|
||||
// the callback; generation and credential checks above remain intact.
|
||||
const gatewayOwnership =
|
||||
await renewDiscordGatewayOwnershipForMessageAdmission(
|
||||
tx,
|
||||
endpoint.id,
|
||||
runtimeContext,
|
||||
);
|
||||
if (gatewayOwnership.kind === "lost") {
|
||||
if (gatewayOwnership.ownership) {
|
||||
discordGatewayAdmissionState.lost = gatewayOwnership.ownership;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
discordGatewayAdmissionState.renewed = gatewayOwnership;
|
||||
}
|
||||
let provisionalTeamsSetupReply = false;
|
||||
let destinationAccepting = true;
|
||||
if (endpointAccepting && thread.isDM) {
|
||||
destinationAccepting = currentEndpoint.allowDirectMessages;
|
||||
}
|
||||
if (
|
||||
endpointAccepting &&
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
!thread.isDM
|
||||
) {
|
||||
({ destinationAccepting, provisionalTeamsSetupReply } = await admitTeamsDestination({ endpoint, currentEndpoint, thread, message, addressed }, tx));
|
||||
}
|
||||
if (
|
||||
endpointAccepting &&
|
||||
endpoint.provider === "telegram" &&
|
||||
!thread.isDM
|
||||
) {
|
||||
destinationAccepting = await admitTelegramDestination({ endpoint, currentEndpoint, thread, addressed }, tx);
|
||||
}
|
||||
if (
|
||||
endpointAccepting &&
|
||||
endpoint.provider === "imessage-photon" &&
|
||||
!thread.isDM
|
||||
) {
|
||||
destinationAccepting = await admitPhotonDestination({ endpoint, currentEndpoint, thread }, tx);
|
||||
}
|
||||
const mentionRequired =
|
||||
endpoint.provider === "slack" &&
|
||||
currentEndpoint.requireAtMention &&
|
||||
trigger !== "mention" &&
|
||||
message.isMention !== true;
|
||||
// GitHub sends the same comment to every installed App. Subscribed
|
||||
// conversations are a fallback for unaddressed replies, not permission
|
||||
// to wake every bot when the person names a different connected App.
|
||||
// Recheck inside durable admission so recovered deliveries follow the
|
||||
// same company-scoped routing decision before any task or reaction.
|
||||
const githubAddressedElsewhere =
|
||||
endpointAccepting &&
|
||||
endpoint.provider === "github" &&
|
||||
!githubAutomatic &&
|
||||
!githubIssue &&
|
||||
(await githubMessageAddressesAnotherBot(tx, {
|
||||
id: endpoint.id,
|
||||
companyId: endpoint.companyId,
|
||||
botUsername: currentEndpoint.botUsername,
|
||||
}, message.text));
|
||||
const accepting = endpointAccepting && destinationAccepting && !githubAddressedElsewhere && !mentionRequired;
|
||||
const redactDestinationDelivery =
|
||||
githubAddressedElsewhere || mentionRequired || (!accepting && thread.isDM) ||
|
||||
|
||||
(!thread.isDM &&
|
||||
(endpoint.provider === "microsoft-teams" ||
|
||||
endpoint.provider === "telegram" ||
|
||||
endpoint.provider === "imessage-photon") &&
|
||||
(!accepting || provisionalTeamsSetupReply));
|
||||
const ignoredAt = accepting ? null : new Date();
|
||||
const inactiveReason = !endpointAccepting
|
||||
? staleActivation
|
||||
? "Connection activation changed before admission"
|
||||
: "Connection is not active"
|
||||
: githubAddressedElsewhere
|
||||
? "GitHub message explicitly mentions a different connected bot"
|
||||
: mentionRequired
|
||||
? "Message did not @mention the bot"
|
||||
: endpoint.provider === "telegram" && !thread.isDM && !addressed
|
||||
? "Message did not address the agent"
|
||||
: "Destination is not enabled in Paperclip";
|
||||
let candidate = admittedDeliveryId
|
||||
? await tx
|
||||
.select()
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.id, admittedDeliveryId),
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
eq(chatDeliveries.providerEventId, providerEventId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
if (!admittedDeliveryId) {
|
||||
await notifyChatDeliveryWork(tx);
|
||||
const [delivery] = await tx
|
||||
.insert(chatDeliveries)
|
||||
.values({
|
||||
companyId: endpoint.companyId,
|
||||
endpointId: endpoint.id,
|
||||
providerEventId,
|
||||
deduplicationKey: createHash("sha256")
|
||||
.update(providerEventId)
|
||||
.digest("hex"),
|
||||
eventKind,
|
||||
normalizedEvent: redactDestinationDelivery
|
||||
? redactedDestinationNormalized
|
||||
: deferDrainUntilFollowup && accepting
|
||||
? {
|
||||
...normalized,
|
||||
providerThreadPending: true,
|
||||
}
|
||||
: normalized,
|
||||
state: accepting
|
||||
? deferDrainUntilFollowup
|
||||
? "retry"
|
||||
: "received"
|
||||
: "filtered",
|
||||
nextAttemptAt:
|
||||
accepting && deferDrainUntilFollowup
|
||||
? new Date(
|
||||
Date.now() + DISCORD_ROOT_THREAD_CONFIRMATION_DELAY_MS,
|
||||
)
|
||||
: accepting && scheduledAt
|
||||
? new Date(scheduledAt)
|
||||
: null,
|
||||
redactedError:
|
||||
accepting && deferDrainUntilFollowup
|
||||
? "Waiting for Discord thread creation confirmation"
|
||||
: accepting
|
||||
? null
|
||||
: inactiveReason,
|
||||
processedAt: ignoredAt,
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning();
|
||||
candidate = delivery ?? null;
|
||||
}
|
||||
if (!candidate && !admittedDeliveryId) {
|
||||
const existingDelivery = await tx
|
||||
.select()
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
eq(chatDeliveries.providerEventId, providerEventId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (existingDelivery) {
|
||||
const hydrateTeamsDelivery =
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
!thread.isDM &&
|
||||
accepting &&
|
||||
!provisionalTeamsSetupReply &&
|
||||
["received", "retry", "processing"].includes(
|
||||
existingDelivery.state,
|
||||
) &&
|
||||
deliveryContentWasRedacted(existingDelivery.normalizedEvent);
|
||||
await notifyChatDeliveryWork(tx);
|
||||
[candidate] = await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
// Increment under PostgreSQL's row lock so simultaneous handler
|
||||
// fanout and provider retries cannot lose duplicate telemetry.
|
||||
normalizedEvent: redactDestinationDelivery
|
||||
? sql`${JSON.stringify(redactedDestinationNormalized)}::jsonb
|
||||
|| jsonb_build_object(
|
||||
'deduplication',
|
||||
jsonb_build_object(
|
||||
'duplicateCount',
|
||||
coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1,
|
||||
'lastDuplicateAt',
|
||||
${new Date().toISOString()}::text
|
||||
)
|
||||
)`
|
||||
: hydrateTeamsDelivery
|
||||
? sql`${JSON.stringify(normalized)}::jsonb
|
||||
|| jsonb_build_object(
|
||||
'deduplication',
|
||||
coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb)
|
||||
|| jsonb_build_object(
|
||||
'duplicateCount',
|
||||
coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1,
|
||||
'lastDuplicateAt',
|
||||
${new Date().toISOString()}::text
|
||||
)
|
||||
)`
|
||||
: sql`coalesce(${chatDeliveries.normalizedEvent}, '{}'::jsonb)
|
||||
|| jsonb_build_object(
|
||||
'deduplication',
|
||||
coalesce(${chatDeliveries.normalizedEvent}->'deduplication', '{}'::jsonb)
|
||||
|| jsonb_build_object(
|
||||
'duplicateCount',
|
||||
coalesce((${chatDeliveries.normalizedEvent}#>>'{deduplication,duplicateCount}')::integer, 0) + 1,
|
||||
'lastDuplicateAt',
|
||||
${new Date().toISOString()}::text
|
||||
)
|
||||
)`,
|
||||
...(!accepting &&
|
||||
["received", "retry"].includes(existingDelivery.state)
|
||||
? {
|
||||
state: "filtered" as const,
|
||||
nextAttemptAt: null,
|
||||
redactedError: inactiveReason,
|
||||
processedAt: ignoredAt,
|
||||
}
|
||||
: {}),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatDeliveries.id, existingDelivery.id))
|
||||
.returning();
|
||||
}
|
||||
}
|
||||
if (
|
||||
candidate &&
|
||||
admittedDeliveryId &&
|
||||
accepting &&
|
||||
!provisionalTeamsSetupReply &&
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
!thread.isDM &&
|
||||
["received", "retry", "processing"].includes(candidate.state) &&
|
||||
deliveryContentWasRedacted(candidate.normalizedEvent)
|
||||
) {
|
||||
[candidate] = await tx
|
||||
.update(chatDeliveries)
|
||||
.set({ normalizedEvent: normalized, updatedAt: new Date() })
|
||||
.where(eq(chatDeliveries.id, candidate.id))
|
||||
.returning();
|
||||
}
|
||||
if (
|
||||
candidate &&
|
||||
admittedDeliveryId &&
|
||||
!accepting &&
|
||||
["received", "retry", "processing"].includes(candidate.state)
|
||||
) {
|
||||
await notifyChatDeliveryWork(tx);
|
||||
[candidate] = await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
...(redactDestinationDelivery
|
||||
? {
|
||||
normalizedEvent: redactedDestinationNormalized,
|
||||
principalId: null,
|
||||
}
|
||||
: {}),
|
||||
nextAttemptAt: null,
|
||||
redactedError: inactiveReason,
|
||||
processedAt: ignoredAt,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatDeliveries.id, candidate.id))
|
||||
.returning();
|
||||
}
|
||||
if (!accepting && ignoredAt) {
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({ lastEventAt: ignoredAt, updatedAt: ignoredAt })
|
||||
.where(eq(chatEndpoints.id, endpoint.id));
|
||||
}
|
||||
return {
|
||||
accepting,
|
||||
candidate: candidate ?? null,
|
||||
provisionalTeamsSetupReply,
|
||||
};
|
||||
});
|
||||
}
|
||||
return { admitInboundMessage };
|
||||
}
|
||||
@@ -0,0 +1,429 @@
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction, LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
stableExternalPrincipalId,
|
||||
} from "../message-lifecycle.js";
|
||||
|
||||
import { voiceSessionStore } from "../../voice/voice-session-store.js";
|
||||
|
||||
import { githubChatPrincipalAccess } from "../../chat-github-access.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
inArray,
|
||||
ne,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatConversations,
|
||||
chatEndpointResources,
|
||||
chatExternalPrincipals,
|
||||
chatIdentityLinks,
|
||||
companyMemberships,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
HttpError,
|
||||
forbidden,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import type {
|
||||
Author,
|
||||
Thread,
|
||||
} from "chat";
|
||||
import { canonicalProviderResourceId, chatSurfaceKind, providerResourceLabelFromThread, providerResourceType, nonDirectDestinationAllowed } from "./surfaces.js";
|
||||
type ResourceRow = typeof chatEndpointResources.$inferSelect;
|
||||
type InboundAuthorityOptions = {
|
||||
voiceStore: Pick<ReturnType<typeof voiceSessionStore>, "authorizePrincipal">;
|
||||
runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise<EndpointRow | null>;
|
||||
};
|
||||
export function createChatInboundAuthority(db: Db, options: InboundAuthorityOptions) {
|
||||
const { voiceStore, runtimeCallbackEndpoint } = options;
|
||||
|
||||
|
||||
async function findAuthorizedIdentityLink(endpoint: EndpointRow, userId: string) {
|
||||
return db.select({ id: chatIdentityLinks.id }).from(chatIdentityLinks)
|
||||
.innerJoin(companyMemberships, and(
|
||||
eq(companyMemberships.companyId, chatIdentityLinks.companyId), eq(companyMemberships.principalType, "user"),
|
||||
eq(companyMemberships.principalId, userId), eq(companyMemberships.status, "active"), ne(companyMemberships.membershipRole, "viewer"),
|
||||
))
|
||||
.where(and(eq(chatIdentityLinks.companyId, endpoint.companyId), eq(chatIdentityLinks.endpointId, endpoint.id),
|
||||
eq(chatIdentityLinks.paperclipUserId, userId), eq(chatIdentityLinks.status, "linked")))
|
||||
.limit(1).then((rows) => rows[0] ?? null);
|
||||
}
|
||||
|
||||
async function ensurePrincipal(
|
||||
endpoint: EndpointRow,
|
||||
author: Author,
|
||||
raw?: unknown,
|
||||
) {
|
||||
const providerAccountId = endpoint.providerAccountId ?? "unknown";
|
||||
const externalId = stableExternalPrincipalId(
|
||||
endpoint.provider,
|
||||
author,
|
||||
raw,
|
||||
);
|
||||
const [principal] = await db
|
||||
.insert(chatExternalPrincipals)
|
||||
.values({
|
||||
companyId: endpoint.companyId,
|
||||
provider: endpoint.provider,
|
||||
providerAccountId,
|
||||
externalId,
|
||||
kind:
|
||||
author.isBot === true ? "bot" : author.isSystem ? "system" : "user",
|
||||
displayName: author.fullName,
|
||||
handle: author.userName,
|
||||
isBot: author.isBot === true || author.isMe,
|
||||
lastSeenAt: new Date(),
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: [
|
||||
chatExternalPrincipals.companyId,
|
||||
chatExternalPrincipals.provider,
|
||||
chatExternalPrincipals.providerAccountId,
|
||||
chatExternalPrincipals.externalId,
|
||||
],
|
||||
set: {
|
||||
displayName: author.fullName,
|
||||
handle: author.userName,
|
||||
isBot: author.isBot === true || author.isMe,
|
||||
lastSeenAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
})
|
||||
.returning();
|
||||
if (endpoint.provider === "speko") {
|
||||
try {
|
||||
const authorization = await db.transaction((tx) => voiceStore.authorizePrincipal(tx, endpoint.companyId, endpoint.id, externalId));
|
||||
return { principal, userId: authorization.userId, linkedDenied: false };
|
||||
} catch (error) {
|
||||
if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error;
|
||||
return { principal, userId: null, linkedDenied: true };
|
||||
}
|
||||
}
|
||||
const link = await db
|
||||
.select({
|
||||
userId: chatIdentityLinks.paperclipUserId,
|
||||
status: chatIdentityLinks.status,
|
||||
})
|
||||
.from(chatIdentityLinks)
|
||||
.where(
|
||||
and(
|
||||
eq(chatIdentityLinks.endpointId, endpoint.id),
|
||||
eq(chatIdentityLinks.principalId, principal.id),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (link?.status === "linked" && link.userId) {
|
||||
const membership = await db
|
||||
.select({
|
||||
status: companyMemberships.status,
|
||||
membershipRole: companyMemberships.membershipRole,
|
||||
})
|
||||
.from(companyMemberships)
|
||||
.where(
|
||||
and(
|
||||
eq(companyMemberships.companyId, endpoint.companyId),
|
||||
eq(companyMemberships.principalType, "user"),
|
||||
eq(companyMemberships.principalId, link.userId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
// External chat performs the same task mutations as the board. A linked
|
||||
// viewer therefore remains read-only instead of gaining write authority
|
||||
// merely by using a provider account.
|
||||
if (
|
||||
membership?.status !== "active" ||
|
||||
membership.membershipRole === "viewer"
|
||||
) {
|
||||
return { principal, userId: null, linkedDenied: true };
|
||||
}
|
||||
return { principal, userId: link.userId, linkedDenied: false };
|
||||
}
|
||||
return { principal, userId: null, linkedDenied: false };
|
||||
}
|
||||
|
||||
async function sponsorAllowsGuest(endpoint: EndpointRow): Promise<boolean> {
|
||||
// A null sponsor is the local board/operator context used by self-hosted
|
||||
// instances. Cloud endpoints retain the creating user and must revalidate
|
||||
// that user's current company authority on every sponsored guest turn.
|
||||
if (!endpoint.sponsorUserId) return true;
|
||||
const membership = await db
|
||||
.select({
|
||||
status: companyMemberships.status,
|
||||
membershipRole: companyMemberships.membershipRole,
|
||||
})
|
||||
.from(companyMemberships)
|
||||
.where(
|
||||
and(
|
||||
eq(companyMemberships.companyId, endpoint.companyId),
|
||||
eq(companyMemberships.principalType, "user"),
|
||||
eq(companyMemberships.principalId, endpoint.sponsorUserId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
return (
|
||||
membership?.status === "active" && membership.membershipRole !== "viewer"
|
||||
);
|
||||
}
|
||||
|
||||
async function lockCurrentPrincipalAuthorization(
|
||||
tx: DbOrTransaction,
|
||||
endpoint: EndpointRow,
|
||||
principalId: string,
|
||||
): Promise<{
|
||||
allowed: boolean;
|
||||
linkedDenied: boolean;
|
||||
userId: string | null;
|
||||
sponsorUserId?: string | null;
|
||||
}> {
|
||||
if (endpoint.provider === "speko") {
|
||||
const principal = await tx.select({ externalId: chatExternalPrincipals.externalId }).from(chatExternalPrincipals)
|
||||
.where(and(eq(chatExternalPrincipals.id, principalId), eq(chatExternalPrincipals.companyId, endpoint.companyId), eq(chatExternalPrincipals.provider, "speko"))).then((rows) => rows[0]);
|
||||
if (!principal) return { allowed: false, linkedDenied: true, userId: null };
|
||||
try { return await voiceStore.authorizePrincipal(tx as import("../../voice/voice-session-store.js").VoiceTransaction, endpoint.companyId, endpoint.id, principal.externalId); }
|
||||
catch (error) {
|
||||
if (!(error instanceof HttpError && [403, 404].includes(error.status))) throw error;
|
||||
return { allowed: false, linkedDenied: true, userId: null };
|
||||
}
|
||||
}
|
||||
const githubAccess = await githubChatPrincipalAccess(tx, endpoint, principalId);
|
||||
if (githubAccess) return githubAccess;
|
||||
// Link confirmation already uses this transaction-scoped identity key.
|
||||
// Taking it at the final task mutation boundary prevents a newly confirmed
|
||||
// identity from racing the authorization snapshot. Row locks below also
|
||||
// serialize revocation and membership changes that do not use this key.
|
||||
await tx.execute(
|
||||
sql`select pg_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0))`,
|
||||
);
|
||||
const link = await tx
|
||||
.select({
|
||||
status: chatIdentityLinks.status,
|
||||
userId: chatIdentityLinks.paperclipUserId,
|
||||
})
|
||||
.from(chatIdentityLinks)
|
||||
.where(
|
||||
and(
|
||||
eq(chatIdentityLinks.companyId, endpoint.companyId),
|
||||
eq(chatIdentityLinks.endpointId, endpoint.id),
|
||||
eq(chatIdentityLinks.principalId, principalId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (link?.status === "linked") {
|
||||
if (!link.userId) {
|
||||
return { allowed: false, linkedDenied: true, userId: null };
|
||||
}
|
||||
const membership = await tx
|
||||
.select({
|
||||
status: companyMemberships.status,
|
||||
membershipRole: companyMemberships.membershipRole,
|
||||
})
|
||||
.from(companyMemberships)
|
||||
.where(
|
||||
and(
|
||||
eq(companyMemberships.companyId, endpoint.companyId),
|
||||
eq(companyMemberships.principalType, "user"),
|
||||
eq(companyMemberships.principalId, link.userId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
const allowed =
|
||||
membership?.status === "active" &&
|
||||
membership.membershipRole !== "viewer";
|
||||
return {
|
||||
allowed,
|
||||
linkedDenied: !allowed,
|
||||
userId: allowed ? link.userId : null,
|
||||
};
|
||||
}
|
||||
if (!endpoint.allowUnlinkedPeople) {
|
||||
return { allowed: false, linkedDenied: false, userId: null };
|
||||
}
|
||||
if (!endpoint.sponsorUserId) {
|
||||
return { allowed: true, linkedDenied: false, userId: null };
|
||||
}
|
||||
const sponsorMembership = await tx
|
||||
.select({
|
||||
status: companyMemberships.status,
|
||||
membershipRole: companyMemberships.membershipRole,
|
||||
})
|
||||
.from(companyMemberships)
|
||||
.where(
|
||||
and(
|
||||
eq(companyMemberships.companyId, endpoint.companyId),
|
||||
eq(companyMemberships.principalType, "user"),
|
||||
eq(companyMemberships.principalId, endpoint.sponsorUserId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
return {
|
||||
allowed:
|
||||
sponsorMembership?.status === "active" &&
|
||||
sponsorMembership.membershipRole !== "viewer",
|
||||
linkedDenied: false,
|
||||
userId: null,
|
||||
};
|
||||
}
|
||||
|
||||
async function requireCurrentExternalActionAuthorization(
|
||||
tx: DbTransaction,
|
||||
input: {
|
||||
conversationId: string;
|
||||
endpointId: string;
|
||||
expectedUserId: string;
|
||||
principalId: string;
|
||||
runtimeContext: LifecycleRuntimeFence;
|
||||
},
|
||||
): Promise<void> {
|
||||
const endpoint = await runtimeCallbackEndpoint(
|
||||
tx,
|
||||
input.endpointId,
|
||||
input.runtimeContext,
|
||||
["active", "verifying"],
|
||||
);
|
||||
// Photon setup already admits linked senders and publishes agent prompts.
|
||||
// Let those prompts resolve so a clarifying question cannot deadlock the
|
||||
// actual-reply qualification. Other providers retain their active-only gate.
|
||||
if (!endpoint || (endpoint.status !== "active" &&
|
||||
(endpoint.provider !== "imessage-photon" || endpoint.setup.step !== "test"))) {
|
||||
throw forbidden("This chat action is no longer authorized", {
|
||||
code: "chat_action_authorization_changed",
|
||||
});
|
||||
}
|
||||
const conversation = await tx
|
||||
.select()
|
||||
.from(chatConversations)
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, endpoint.companyId),
|
||||
eq(chatConversations.endpointId, endpoint.id),
|
||||
eq(chatConversations.id, input.conversationId),
|
||||
inArray(chatConversations.state, ["active", "waiting"]),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!conversation) {
|
||||
throw forbidden("This chat action is no longer authorized", {
|
||||
code: "chat_action_authorization_changed",
|
||||
});
|
||||
}
|
||||
const resource = conversation.resourceId
|
||||
? await tx
|
||||
.select()
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.companyId, endpoint.companyId),
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.id, conversation.resourceId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const destinationAllowed = conversation.isDirectMessage
|
||||
? endpoint.allowDirectMessages
|
||||
: nonDirectDestinationAllowed(endpoint, resource);
|
||||
const authorization = await lockCurrentPrincipalAuthorization(
|
||||
tx,
|
||||
endpoint,
|
||||
input.principalId,
|
||||
);
|
||||
// Governed actions never fall back to sponsored-guest authority. If the
|
||||
// provider identity was relinked while this callback waited, reject this
|
||||
// stale actor snapshot and let a fresh callback use the new identity.
|
||||
if (
|
||||
!destinationAllowed ||
|
||||
!authorization.allowed ||
|
||||
authorization.userId !== input.expectedUserId
|
||||
) {
|
||||
throw forbidden("This chat action is no longer authorized", {
|
||||
code: "chat_action_authorization_changed",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function isExternalActionAuthorizationChange(error: unknown): boolean {
|
||||
if (!error || typeof error !== "object") return false;
|
||||
const value = error as { details?: unknown };
|
||||
return (
|
||||
Boolean(value.details) &&
|
||||
typeof value.details === "object" &&
|
||||
(value.details as { code?: unknown }).code ===
|
||||
"chat_action_authorization_changed"
|
||||
);
|
||||
}
|
||||
|
||||
async function ensureResource(
|
||||
endpoint: EndpointRow,
|
||||
thread: Thread,
|
||||
enabledBySetupActivation: boolean,
|
||||
database: DbOrTransaction = db,
|
||||
) {
|
||||
const type = providerResourceType(
|
||||
endpoint.provider,
|
||||
chatSurfaceKind(endpoint.provider, thread),
|
||||
);
|
||||
const providerResourceId = canonicalProviderResourceId(
|
||||
endpoint.provider,
|
||||
thread,
|
||||
);
|
||||
const resourceLabel = providerResourceLabelFromThread(
|
||||
endpoint.provider,
|
||||
thread,
|
||||
);
|
||||
const [resource] = await database
|
||||
.insert(chatEndpointResources)
|
||||
.values({
|
||||
companyId: endpoint.companyId,
|
||||
endpointId: endpoint.id,
|
||||
type,
|
||||
providerResourceId,
|
||||
label: resourceLabel.label,
|
||||
availability: "available",
|
||||
// Slack can deliver the invitation's app_mention before the membership
|
||||
// callback. Give either arrival order the same new-channel default.
|
||||
enabled:
|
||||
thread.isDM ||
|
||||
enabledBySetupActivation ||
|
||||
(endpoint.provider === "slack" && type === "channel"),
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: [
|
||||
chatEndpointResources.endpointId,
|
||||
chatEndpointResources.type,
|
||||
chatEndpointResources.providerResourceId,
|
||||
],
|
||||
set: {
|
||||
// Provider inventory and membership lifecycle own availability.
|
||||
// A delayed message must not resurrect a destination after the bot
|
||||
// was removed or an installation lost access.
|
||||
// Slack callbacks are not guaranteed to carry a channel name. Keep
|
||||
// an inventory- or conversations.info-derived label instead of
|
||||
// downgrading it to C… / slack:C… when a delayed message arrives.
|
||||
...(resourceLabel.fallback
|
||||
? {}
|
||||
: {
|
||||
label: sql<string>`case
|
||||
when ${chatEndpointResources.availability} = 'available' then ${resourceLabel.label}
|
||||
else ${chatEndpointResources.label}
|
||||
end`,
|
||||
}),
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
})
|
||||
.returning();
|
||||
return resource;
|
||||
}
|
||||
return { findAuthorizedIdentityLink, ensurePrincipal, sponsorAllowsGuest, lockCurrentPrincipalAuthorization, requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, ensureResource };
|
||||
}
|
||||
@@ -0,0 +1,767 @@
|
||||
import { createGitHubMessageProvenance } from "../github/message-provenance.js";
|
||||
import { createDiscordRootThreads } from "../discord/root-threads.js";
|
||||
|
||||
import type { EndpointRow, RuntimeContext, InboundRuntimeContext } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
telegramDeliveryHasZeroMessageId,
|
||||
} from "../telegram/messages.js";
|
||||
|
||||
import { notifyChatDeliveryWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
|
||||
import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
createHash,
|
||||
randomUUID,
|
||||
} from "node:crypto";
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
inArray,
|
||||
lte,
|
||||
notInArray,
|
||||
or,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import {
|
||||
externalThreadIdentityCondition,
|
||||
} from "../../chat-control-chronology.js";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatDeliveries,
|
||||
chatEndpointLeases,
|
||||
chatEndpoints,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
normalizeContentType,
|
||||
} from "../../../attachment-types.js";
|
||||
|
||||
import { logger } from "../../../middleware/logger.js";
|
||||
|
||||
import {
|
||||
type ChatSdkMessageCallbackEvent,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
|
||||
import { isExternalChatWaitAuthorizationContention } from "../../native-runtime/chat-attachment-reuse.js";
|
||||
|
||||
import {
|
||||
restoreGitHubAttachmentLimitOmissions,
|
||||
} from "../../chat-github-attachments.js";
|
||||
|
||||
import type {
|
||||
Attachment,
|
||||
Message,
|
||||
Thread,
|
||||
} from "chat";
|
||||
import { DELIVERY_PROCESSING_STALE_MS, deliveryContentWasRedacted, setupDestinationActivationSuppressed, normalizedDeliveryThreadId, normalizedLifecycleTargetEventId, normalizedLifecycleEffect, deliveryReady, conversationDeliveryOrder, pendingInboundWakeupCondition, lifecycleRuntimeFence } from "./delivery-ledger.js";
|
||||
import type { createChatMessageLifecycle } from "./lifecycle.js";
|
||||
import type { LiveInboundMessage } from "../runtime-types.js";
|
||||
type DeliveryRow = typeof chatDeliveries.$inferSelect;
|
||||
type ConversationDeliveryQueueOptions = {
|
||||
DELIVERY_DRAIN_LIMIT: number;
|
||||
DELIVERY_LEASE_TTL_MS: number;
|
||||
discordRootThreadPending: ReturnType<typeof createDiscordRootThreads>["discordRootThreadPending"];
|
||||
reconcileDiscordRootThread: ReturnType<typeof createDiscordRootThreads>["reconcileDiscordRootThread"];
|
||||
durableExternalThreadIdentity: (threadId: string) => string;
|
||||
liveInboundMessages: Map<string, LiveInboundMessage>;
|
||||
processInboundWakeup: (deliveryId: string) => Promise<boolean>;
|
||||
processLifecycleDelivery: ReturnType<typeof createChatMessageLifecycle>["processLifecycleDelivery"];
|
||||
processMessage: (endpoint: EndpointRow, thread: Thread, message: Message, trigger: ChatSdkMessageCallbackEvent["trigger"], ingressOnly?: boolean, recoveredProviderUrl?: string | null, runtimeContext?: InboundRuntimeContext, providerUpdateId?: number, admittedDeliveryId?: string | null, receiptReactionSupported?: boolean, preauthorizedUserId?: string | null, deferDrainUntilFollowup?: boolean, suppressSetupDestinationActivation?: boolean, slackSlashControl?: boolean) => Promise<void>;
|
||||
redactError: (error: unknown) => string;
|
||||
runtimeContexts: WeakMap<object, RuntimeContext>;
|
||||
runtimeFor: (endpoint: EndpointRow) => Promise<ChatEndpointRuntime>;
|
||||
settleRejectedInboundWakeups: (onlyDeliveryId?: string) => Promise<unknown>;
|
||||
sanitizeFilename: (value: string | undefined) => string | null;
|
||||
githubManualMessages: WeakMap<object, { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }>;
|
||||
githubAutomaticMessages: WeakMap<object, { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
githubIssueMessages: WeakMap<object, { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
conversationDrainKey: (endpointId: string, threadId: string) => string;
|
||||
conversationLeaseRenewalIntervalMs?: number;
|
||||
renewConversationDeliveryLease?: (input: { endpointId: string; leaseKey: string; token: string; expiresAt: Date }) => Promise<boolean>;
|
||||
};
|
||||
export function createChatConversationDeliveryQueue(db: Db, options: ConversationDeliveryQueueOptions) {
|
||||
const { DELIVERY_DRAIN_LIMIT, DELIVERY_LEASE_TTL_MS, discordRootThreadPending, reconcileDiscordRootThread, durableExternalThreadIdentity, liveInboundMessages, processInboundWakeup, processLifecycleDelivery, processMessage, redactError, runtimeContexts, runtimeFor, settleRejectedInboundWakeups, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages, conversationDrainKey } = options;
|
||||
const { restoreGitHubMessageProvenance } = createGitHubMessageProvenance({ githubManualMessages, githubAutomaticMessages, githubIssueMessages });
|
||||
|
||||
|
||||
async function earliestOpenConversationDelivery(
|
||||
endpointId: string,
|
||||
threadId: string,
|
||||
): Promise<DeliveryRow | null> {
|
||||
const externalThreadId = sql<string>`${chatDeliveries.normalizedEvent}->'conversation'->>'externalThreadId'`;
|
||||
const candidate = await db
|
||||
.select()
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, endpointId),
|
||||
or(
|
||||
inArray(chatDeliveries.state, ["received", "retry", "processing"]),
|
||||
and(
|
||||
eq(chatDeliveries.state, "processed"),
|
||||
pendingInboundWakeupCondition(),
|
||||
),
|
||||
),
|
||||
notInArray(chatDeliveries.eventKind, [
|
||||
"reaction_added",
|
||||
"reaction_removed",
|
||||
]),
|
||||
externalThreadIdentityCondition(externalThreadId, threadId),
|
||||
),
|
||||
)
|
||||
.orderBy(...conversationDeliveryOrder())
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!candidate) return null;
|
||||
|
||||
// A lifecycle callback can reach Paperclip before its create callback. If
|
||||
// the root becomes durable while the lifecycle row is waiting, always
|
||||
// return that exact dependency first even when a coarse or malformed
|
||||
// provider timestamp would otherwise put the edit/delete at the head.
|
||||
const targetProviderEventId = normalizedLifecycleTargetEventId(candidate);
|
||||
if (!targetProviderEventId) return candidate;
|
||||
return db
|
||||
.select()
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, endpointId),
|
||||
eq(chatDeliveries.providerEventId, targetProviderEventId),
|
||||
inArray(chatDeliveries.state, ["received", "retry", "processing"]),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? candidate);
|
||||
}
|
||||
|
||||
async function acquireConversationDeliveryLease(input: {
|
||||
companyId: string;
|
||||
endpointId: string;
|
||||
threadId: string;
|
||||
}): Promise<{ leaseKey: string; token: string } | null> {
|
||||
const now = new Date();
|
||||
const token = randomUUID();
|
||||
const leaseKey = `inbound:${createHash("sha256").update(durableExternalThreadIdentity(input.threadId)).digest("hex")}`;
|
||||
const expiresAt = new Date(now.getTime() + DELIVERY_LEASE_TTL_MS);
|
||||
const inserted = await db
|
||||
.insert(chatEndpointLeases)
|
||||
.values({
|
||||
companyId: input.companyId,
|
||||
endpointId: input.endpointId,
|
||||
leaseKey,
|
||||
token,
|
||||
expiresAt,
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning({ id: chatEndpointLeases.id });
|
||||
if (inserted.length > 0) return { leaseKey, token };
|
||||
const reclaimed = await db
|
||||
.update(chatEndpointLeases)
|
||||
.set({ token, expiresAt, updatedAt: now })
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointLeases.companyId, input.companyId),
|
||||
eq(chatEndpointLeases.endpointId, input.endpointId),
|
||||
eq(chatEndpointLeases.leaseKey, leaseKey),
|
||||
lte(chatEndpointLeases.expiresAt, now),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatEndpointLeases.id });
|
||||
return reclaimed.length > 0 ? { leaseKey, token } : null;
|
||||
}
|
||||
|
||||
async function releaseConversationDeliveryLease(input: {
|
||||
endpointId: string;
|
||||
leaseKey: string;
|
||||
token: string;
|
||||
}): Promise<void> {
|
||||
await db
|
||||
.delete(chatEndpointLeases)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointLeases.endpointId, input.endpointId),
|
||||
eq(chatEndpointLeases.leaseKey, input.leaseKey),
|
||||
eq(chatEndpointLeases.token, input.token),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function messageFromDelivery(
|
||||
delivery: DeliveryRow,
|
||||
thread: Thread,
|
||||
endpointRuntime: ChatEndpointRuntime,
|
||||
): {
|
||||
message: Message;
|
||||
receiptReactionSupported: boolean;
|
||||
suppressSetupDestinationActivation: boolean;
|
||||
trigger: ChatSdkMessageCallbackEvent["trigger"];
|
||||
providerUrl: string | null;
|
||||
} | null {
|
||||
if (telegramDeliveryHasZeroMessageId(delivery, endpointRuntime.provider))
|
||||
return null;
|
||||
const normalized = delivery.normalizedEvent as {
|
||||
acknowledgement?: { receiptReactionSupported?: unknown };
|
||||
kind?: unknown;
|
||||
trigger?: unknown;
|
||||
principal?: {
|
||||
externalId?: unknown;
|
||||
displayName?: unknown;
|
||||
handle?: unknown;
|
||||
};
|
||||
message?: {
|
||||
providerMessageId?: unknown;
|
||||
sourceProviderMessageId?: unknown;
|
||||
text?: unknown;
|
||||
mentionedBot?: unknown;
|
||||
attachments?: Array<{
|
||||
name?: unknown;
|
||||
mimeType?: unknown;
|
||||
size?: unknown;
|
||||
recovery?: unknown;
|
||||
}>;
|
||||
attachmentLimitOmissions?: unknown;
|
||||
photonReply?: { guid?: unknown; part?: unknown };
|
||||
};
|
||||
conversation?: { providerUrl?: unknown };
|
||||
};
|
||||
const providerMessageId = normalized.message?.providerMessageId;
|
||||
const externalId = normalized.principal?.externalId;
|
||||
if (typeof providerMessageId !== "string" || typeof externalId !== "string")
|
||||
return null;
|
||||
const isTeams = endpointRuntime.provider === "microsoft-teams";
|
||||
const referenceOnlyTeamsAttachments = isTeams && !thread.isDM;
|
||||
const originFence = lifecycleRuntimeFence(delivery);
|
||||
const storedAttachments = normalized.message?.attachments ?? [];
|
||||
const attachments = (
|
||||
isTeams ? storedAttachments.slice(0, 20) : storedAttachments
|
||||
)
|
||||
.map((attachment) => {
|
||||
if (!attachment || typeof attachment !== "object") return null;
|
||||
const rehydrated =
|
||||
attachment.recovery &&
|
||||
(!referenceOnlyTeamsAttachments ||
|
||||
(typeof attachment.recovery === "object" &&
|
||||
attachment.recovery !== null &&
|
||||
"locator" in attachment.recovery &&
|
||||
typeof attachment.recovery.locator === "object" &&
|
||||
attachment.recovery.locator !== null &&
|
||||
"kind" in attachment.recovery.locator &&
|
||||
attachment.recovery.locator.kind === "teams_inline_image"))
|
||||
? endpointRuntime.rehydrateAttachment(attachment.recovery, {
|
||||
threadId: thread.id,
|
||||
messageId: typeof normalized.message?.sourceProviderMessageId === "string" ? normalized.message.sourceProviderMessageId : providerMessageId,
|
||||
runtimeGeneration: originFence?.generation,
|
||||
credentialFingerprint: originFence?.credentialFingerprint,
|
||||
principalExternalId: externalId,
|
||||
isDirectMessage: thread.isDM,
|
||||
})
|
||||
: null;
|
||||
if (rehydrated) return rehydrated;
|
||||
if (
|
||||
isTeams ||
|
||||
endpointRuntime.provider === "imessage-photon" ||
|
||||
(endpointRuntime.provider === "telegram" &&
|
||||
typeof attachment.recovery === "object" &&
|
||||
attachment.recovery !== null &&
|
||||
"locator" in attachment.recovery &&
|
||||
typeof attachment.recovery.locator === "object" &&
|
||||
attachment.recovery.locator !== null &&
|
||||
"kind" in attachment.recovery.locator &&
|
||||
attachment.recovery.locator.kind === "telegram_media")
|
||||
) {
|
||||
// Non-personal references and personal files whose safe download
|
||||
// capability did not survive restart still belong to this input.
|
||||
// Keep bounded metadata so ingestion records an explicit omission;
|
||||
// never persist bearer URLs or fabricate a download capability.
|
||||
// Even a legacy URL locator cannot enable non-personal downloads.
|
||||
// A Telegram media locator which fails its exact source proof also
|
||||
// remains an unavailable input, never an empty attachment-free turn.
|
||||
return {
|
||||
type: "file",
|
||||
name:
|
||||
sanitizeFilename(
|
||||
typeof attachment.name === "string"
|
||||
? attachment.name
|
||||
: undefined,
|
||||
) ?? undefined,
|
||||
mimeType: normalizeContentType(
|
||||
typeof attachment.mimeType === "string"
|
||||
? attachment.mimeType
|
||||
: undefined,
|
||||
).slice(0, 255),
|
||||
size:
|
||||
typeof attachment.size === "number" &&
|
||||
Number.isSafeInteger(attachment.size) &&
|
||||
attachment.size >= 0
|
||||
? attachment.size
|
||||
: undefined,
|
||||
} as Attachment;
|
||||
}
|
||||
return null;
|
||||
})
|
||||
.filter((attachment): attachment is Attachment => Boolean(attachment));
|
||||
const message = {
|
||||
id: providerMessageId,
|
||||
threadId: thread.id,
|
||||
text:
|
||||
typeof normalized.message?.text === "string"
|
||||
? normalized.message.text
|
||||
: "",
|
||||
formatted: { type: "root", children: [] },
|
||||
raw:
|
||||
endpointRuntime.provider === "imessage-photon" &&
|
||||
typeof normalized.message?.photonReply?.guid === "string"
|
||||
? {
|
||||
replyTargetGuid: normalized.message.photonReply.guid,
|
||||
threadOriginatorPart: normalized.message.photonReply.part,
|
||||
}
|
||||
: endpointRuntime.provider === "github" && typeof normalized.message?.sourceProviderMessageId === "string"
|
||||
? { githubSourceMessageId: normalized.message.sourceProviderMessageId } : {},
|
||||
author: {
|
||||
userId: externalId,
|
||||
userName:
|
||||
typeof normalized.principal?.handle === "string"
|
||||
? normalized.principal.handle
|
||||
: externalId,
|
||||
fullName:
|
||||
typeof normalized.principal?.displayName === "string"
|
||||
? normalized.principal.displayName
|
||||
: externalId,
|
||||
isBot: false,
|
||||
isMe: false,
|
||||
isSystem: false,
|
||||
},
|
||||
metadata: { dateSent: delivery.receivedAt, edited: false },
|
||||
attachments,
|
||||
links: [],
|
||||
isMention: normalized.message?.mentionedBot === true,
|
||||
} as unknown as Message;
|
||||
restoreGitHubMessageProvenance(delivery, endpointRuntime, message);
|
||||
if (
|
||||
endpointRuntime.provider === "github" ||
|
||||
endpointRuntime.provider === "imessage-photon"
|
||||
) {
|
||||
restoreGitHubAttachmentLimitOmissions(
|
||||
message,
|
||||
normalized.message?.attachmentLimitOmissions,
|
||||
);
|
||||
}
|
||||
const normalizedTrigger = normalized.trigger;
|
||||
const trigger: ChatSdkMessageCallbackEvent["trigger"] =
|
||||
normalizedTrigger === "direct_message" ||
|
||||
normalizedTrigger === "mention" ||
|
||||
normalizedTrigger === "subscribed_message" ||
|
||||
normalizedTrigger === "unaddressed_message"
|
||||
? normalizedTrigger
|
||||
: normalized.kind === "direct_message"
|
||||
? "direct_message"
|
||||
: normalized.kind === "mention"
|
||||
? "mention"
|
||||
: "subscribed_message";
|
||||
return {
|
||||
message,
|
||||
// Legacy deliveries all originated from native provider messages and
|
||||
// therefore retain the historical default.
|
||||
receiptReactionSupported:
|
||||
normalized.acknowledgement?.receiptReactionSupported !== false,
|
||||
suppressSetupDestinationActivation: setupDestinationActivationSuppressed(
|
||||
delivery.normalizedEvent,
|
||||
),
|
||||
trigger,
|
||||
providerUrl:
|
||||
typeof normalized.conversation?.providerUrl === "string"
|
||||
? normalized.conversation.providerUrl
|
||||
: null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Drain one external conversation in durable receipt order. The database
|
||||
* lease is endpoint + thread scoped, so separate server processes can work
|
||||
* on different conversations concurrently but can never mutate the same
|
||||
* Paperclip task from two inbound turns at once.
|
||||
*/
|
||||
async function drainConversationDeliveries(
|
||||
endpointId: string,
|
||||
threadId: string,
|
||||
): Promise<boolean> {
|
||||
const endpoint = await db
|
||||
.select()
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpointId))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!endpoint) return false;
|
||||
if (endpoint.status === "paused" || endpoint.status === "attention")
|
||||
return false;
|
||||
|
||||
const lease = await acquireConversationDeliveryLease({
|
||||
companyId: endpoint.companyId,
|
||||
endpointId,
|
||||
threadId,
|
||||
});
|
||||
if (!lease) return false;
|
||||
|
||||
let leaseOwned = true;
|
||||
let renewal: Promise<void> | null = null;
|
||||
const renewTimer = setInterval(
|
||||
() => {
|
||||
if (!leaseOwned || renewal) return;
|
||||
const expiresAt = new Date(Date.now() + DELIVERY_LEASE_TTL_MS);
|
||||
renewal = Promise.resolve()
|
||||
.then(async () => {
|
||||
const renewed = options.renewConversationDeliveryLease
|
||||
? await options.renewConversationDeliveryLease({
|
||||
endpointId,
|
||||
leaseKey: lease.leaseKey,
|
||||
token: lease.token,
|
||||
expiresAt,
|
||||
})
|
||||
: await db
|
||||
.update(chatEndpointLeases)
|
||||
.set({ expiresAt, updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointLeases.endpointId, endpointId),
|
||||
eq(chatEndpointLeases.leaseKey, lease.leaseKey),
|
||||
eq(chatEndpointLeases.token, lease.token),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatEndpointLeases.id })
|
||||
.then((rows) => rows.length > 0);
|
||||
if (!renewed) leaseOwned = false;
|
||||
})
|
||||
.catch((error) => {
|
||||
// A failed renewal has unknown durability. Stop this local drain at
|
||||
// the next delivery boundary so another worker can never overlap
|
||||
// later turns after the original lease expires or is reclaimed.
|
||||
leaseOwned = false;
|
||||
logger.warn(
|
||||
{ endpointId, error: redactError(error) },
|
||||
"could not renew external chat conversation lease",
|
||||
);
|
||||
})
|
||||
.finally(() => {
|
||||
renewal = null;
|
||||
});
|
||||
},
|
||||
options.conversationLeaseRenewalIntervalMs ?? DELIVERY_LEASE_TTL_MS / 3,
|
||||
);
|
||||
renewTimer.unref?.();
|
||||
|
||||
try {
|
||||
for (
|
||||
let processed = 0;
|
||||
processed < DELIVERY_DRAIN_LIMIT;
|
||||
processed += 1
|
||||
) {
|
||||
if (!leaseOwned) break;
|
||||
const delivery = await earliestOpenConversationDelivery(
|
||||
endpointId,
|
||||
threadId,
|
||||
);
|
||||
if (!delivery || !deliveryReady(delivery, new Date())) break;
|
||||
if (delivery.state === "processed") {
|
||||
if (!(await processInboundWakeup(delivery.id))) break;
|
||||
continue;
|
||||
}
|
||||
if (telegramDeliveryHasZeroMessageId(delivery, endpoint.provider)) {
|
||||
const filtered = await db
|
||||
.transaction(async (tx) => {
|
||||
// Do not round-trip a PostgreSQL microsecond timestamp through
|
||||
// JS Date for a CAS. Lock the exact receipt, then revalidate the
|
||||
// current source and claim before changing it. A competing live
|
||||
// claim is a retry boundary, not permission to settle its work.
|
||||
const [current] = await tx
|
||||
.select()
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.id, delivery.id),
|
||||
eq(chatDeliveries.companyId, endpoint.companyId),
|
||||
eq(chatDeliveries.endpointId, endpoint.id),
|
||||
),
|
||||
)
|
||||
.for("update", { noWait: true });
|
||||
const filteredAt = new Date();
|
||||
if (
|
||||
!current ||
|
||||
current.state !== delivery.state ||
|
||||
current.eventKind !== delivery.eventKind ||
|
||||
!["received", "retry", "processing"].includes(current.state) ||
|
||||
normalizedDeliveryThreadId(current) !== threadId ||
|
||||
!deliveryReady(current, filteredAt) ||
|
||||
!telegramDeliveryHasZeroMessageId(current, endpoint.provider)
|
||||
)
|
||||
return false;
|
||||
await notifyChatDeliveryWork(tx);
|
||||
const changed = await tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
nextAttemptAt: null,
|
||||
processedAt: filteredAt,
|
||||
updatedAt: filteredAt,
|
||||
redactedError:
|
||||
"Telegram message ID zero cannot authorize ordinary work",
|
||||
})
|
||||
.where(eq(chatDeliveries.id, current.id))
|
||||
.returning({ id: chatDeliveries.id });
|
||||
return changed.length === 1;
|
||||
})
|
||||
.catch((error: unknown) => {
|
||||
if (isExternalChatWaitAuthorizationContention(error))
|
||||
return false;
|
||||
throw error;
|
||||
});
|
||||
if (!filtered) break;
|
||||
// Retain original evidence/comments; filtering adds no user content
|
||||
// and cannot hydrate attachments, invoke commands, or send feedback.
|
||||
await settleRejectedInboundWakeups(delivery.id);
|
||||
liveInboundMessages.delete(delivery.id);
|
||||
continue;
|
||||
}
|
||||
if (endpoint.status === "archived" || endpoint.status === "revoked") {
|
||||
await db.transaction(async (tx) => {
|
||||
await notifyChatDeliveryWork(tx);
|
||||
return tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
redactedError: "Connection was removed before processing",
|
||||
processedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatDeliveries.id, delivery.id));
|
||||
});
|
||||
liveInboundMessages.delete(delivery.id);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (
|
||||
endpoint.provider === "discord" &&
|
||||
discordRootThreadPending(delivery)
|
||||
) {
|
||||
// This is the only recovery path for the narrow crash window after
|
||||
// pre-admission but before Chat's ordinary callback. A successful
|
||||
// read proves the root thread exists; an explicit 404/410 proves it
|
||||
// does not. Ambiguous transport and auth failures remain retryable.
|
||||
if (await reconcileDiscordRootThread(endpoint, delivery, threadId)) {
|
||||
continue;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
const live = liveInboundMessages.get(delivery.id);
|
||||
try {
|
||||
if (
|
||||
delivery.eventKind === "message_updated" ||
|
||||
delivery.eventKind === "message_deleted" ||
|
||||
delivery.eventKind === "message_restored"
|
||||
) {
|
||||
await processLifecycleDelivery(endpoint, delivery);
|
||||
} else if (live) {
|
||||
await processMessage(
|
||||
endpoint,
|
||||
live.thread,
|
||||
live.message,
|
||||
live.trigger,
|
||||
false,
|
||||
live.providerUrl,
|
||||
live.runtimeContext,
|
||||
undefined,
|
||||
delivery.id,
|
||||
live.receiptReactionSupported,
|
||||
undefined,
|
||||
false,
|
||||
setupDestinationActivationSuppressed(delivery.normalizedEvent),
|
||||
);
|
||||
} else {
|
||||
if (deliveryContentWasRedacted(delivery.normalizedEvent)) {
|
||||
const filteredAt = new Date();
|
||||
await db.transaction(async (tx) => {
|
||||
await notifyChatDeliveryWork(tx);
|
||||
return tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "filtered",
|
||||
principalId: null,
|
||||
nextAttemptAt: null,
|
||||
processedAt: filteredAt,
|
||||
redactedError:
|
||||
"Provisional Teams setup reply could not be hydrated from a current provider event",
|
||||
updatedAt: filteredAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.id, delivery.id),
|
||||
inArray(chatDeliveries.state, [
|
||||
"received",
|
||||
"retry",
|
||||
"processing",
|
||||
]),
|
||||
),
|
||||
);
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const endpointRuntime = await runtimeFor(endpoint);
|
||||
const deliveryRuntimeFence = lifecycleRuntimeFence(delivery);
|
||||
const currentRuntimeContext = runtimeContexts.get(
|
||||
endpointRuntime as object,
|
||||
);
|
||||
const thread = endpointRuntime.thread(threadId);
|
||||
const reconstructed = messageFromDelivery(
|
||||
delivery,
|
||||
thread,
|
||||
endpointRuntime,
|
||||
);
|
||||
if (!reconstructed) {
|
||||
await db.transaction(async (tx) => {
|
||||
await notifyChatDeliveryWork(tx);
|
||||
return tx
|
||||
.update(chatDeliveries)
|
||||
.set({
|
||||
state: "failed",
|
||||
redactedError: "Normalized delivery is incomplete",
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatDeliveries.id, delivery.id));
|
||||
});
|
||||
continue;
|
||||
}
|
||||
await processMessage(
|
||||
endpoint,
|
||||
thread,
|
||||
reconstructed.message,
|
||||
reconstructed.trigger,
|
||||
false,
|
||||
reconstructed.providerUrl,
|
||||
deliveryRuntimeFence
|
||||
? {
|
||||
...deliveryRuntimeFence,
|
||||
endpointRuntime,
|
||||
}
|
||||
: currentRuntimeContext,
|
||||
undefined,
|
||||
delivery.id,
|
||||
reconstructed.receiptReactionSupported,
|
||||
undefined,
|
||||
false,
|
||||
reconstructed.suppressSetupDestinationActivation,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
{
|
||||
endpointId,
|
||||
deliveryId: delivery.id,
|
||||
error: redactError(error),
|
||||
},
|
||||
"external chat delivery will retry in receipt order",
|
||||
);
|
||||
}
|
||||
|
||||
const state = await db
|
||||
.select({ state: chatDeliveries.state })
|
||||
.from(chatDeliveries)
|
||||
.where(eq(chatDeliveries.id, delivery.id))
|
||||
.then((rows) => rows[0]?.state ?? null);
|
||||
if (
|
||||
state === "processed" ||
|
||||
state === "filtered" ||
|
||||
state === "failed"
|
||||
) {
|
||||
if (state !== "processed")
|
||||
await settleRejectedInboundWakeups(delivery.id);
|
||||
liveInboundMessages.delete(delivery.id);
|
||||
continue;
|
||||
}
|
||||
// A retry delay or another still-live processing claim is a strict
|
||||
// head-of-line barrier; later replies may not overtake it.
|
||||
break;
|
||||
}
|
||||
} finally {
|
||||
clearInterval(renewTimer);
|
||||
await renewal;
|
||||
await releaseConversationDeliveryLease({ endpointId, ...lease });
|
||||
}
|
||||
|
||||
const next = await earliestOpenConversationDelivery(endpointId, threadId);
|
||||
if (next?.state === "processed") {
|
||||
const [ready] = await db
|
||||
.select({ id: chatDeliveries.id })
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.id, next.id),
|
||||
pendingInboundWakeupCondition(true),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
return Boolean(ready);
|
||||
}
|
||||
return Boolean(next && deliveryReady(next, new Date()));
|
||||
}
|
||||
|
||||
async function nextInboundDeliveryAt(): Promise<number | null> {
|
||||
const rows = await db.select({ delivery: {
|
||||
id: chatDeliveries.id, endpointId: chatDeliveries.endpointId, providerEventId: chatDeliveries.providerEventId,
|
||||
state: chatDeliveries.state, eventKind: chatDeliveries.eventKind,
|
||||
// Retain scheduling metadata only, not message bodies or attachment data.
|
||||
normalizedEvent: sql<DeliveryRow["normalizedEvent"]>`jsonb_build_object(
|
||||
'conversation', ${chatDeliveries.normalizedEvent}->'conversation',
|
||||
'lifecycle', ${chatDeliveries.normalizedEvent}->'lifecycle',
|
||||
'message', jsonb_build_object(
|
||||
'targetProviderEventId', ${chatDeliveries.normalizedEvent}->'message'->'targetProviderEventId',
|
||||
'admissionDependencyEventId', ${chatDeliveries.normalizedEvent}->'message'->'admissionDependencyEventId'))`,
|
||||
}, at: sql<string | null>`case
|
||||
when ${chatDeliveries.state} = 'processing' then ${chatDeliveries.updatedAt} + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond'
|
||||
when ${chatDeliveries.state} = 'processed' then (
|
||||
select min(case when a.status = 'processing' then a.updated_at + ${DELIVERY_PROCESSING_STALE_MS} * interval '1 millisecond'
|
||||
else coalesce((a.result->>'retryAt')::timestamptz, now()) end)
|
||||
from chat_actions a where a.delivery_id = chat_deliveries.id
|
||||
and a.company_id = chat_deliveries.company_id and a.endpoint_id = chat_deliveries.endpoint_id
|
||||
and a.kind = 'inbound_wakeup' and a.status in ('issued', 'processing'))
|
||||
else coalesce(${chatDeliveries.nextAttemptAt}, now()) end` })
|
||||
.from(chatDeliveries).where(and(
|
||||
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.provider = 'agentmail')`,
|
||||
// Provider lifecycle receipts can restore an unavailable endpoint.
|
||||
// The ordinary conversation drain still waits for operator resume.
|
||||
or(sql`${chatDeliveries.normalizedEvent}->'lifecycle' is not null`,
|
||||
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatDeliveries.endpointId} and e.status in ('paused', 'attention'))`),
|
||||
or(inArray(chatDeliveries.state, ["received", "retry", "processing"]),
|
||||
and(eq(chatDeliveries.state, "processed"), pendingInboundWakeupCondition())),
|
||||
)).orderBy(...conversationDeliveryOrder());
|
||||
// Only an ordered conversation head can run. Reaction and provider
|
||||
// lifecycle receipts are independent; lifecycle edits may select their
|
||||
// exact admission dependency before the chronologically earliest row.
|
||||
const byEvent = new Map(rows.map(row => [`${row.delivery.endpointId}:${row.delivery.providerEventId}`, row]));
|
||||
const conversations = new Set<string>();
|
||||
let next: number | null = null;
|
||||
for (let row of rows) {
|
||||
const delivery = row.delivery;
|
||||
const threadId = normalizedDeliveryThreadId(delivery);
|
||||
if (threadId && !normalizedLifecycleEffect(delivery) && !["reaction_added", "reaction_removed"].includes(delivery.eventKind)) {
|
||||
const key = conversationDrainKey(delivery.endpointId, threadId);
|
||||
if (conversations.has(key)) continue;
|
||||
conversations.add(key);
|
||||
const dependency = normalizedLifecycleTargetEventId(delivery);
|
||||
const target = dependency ? byEvent.get(`${delivery.endpointId}:${dependency}`) : undefined;
|
||||
if (target && ["received", "retry", "processing"].includes(target.delivery.state)) row = target;
|
||||
}
|
||||
if (row.at !== null) {
|
||||
const at = new Date(row.at).getTime();
|
||||
next = next === null ? at : Math.min(next, at);
|
||||
}
|
||||
}
|
||||
return next;
|
||||
}
|
||||
return { drainConversationDeliveries, nextInboundDeliveryAt };
|
||||
}
|
||||
@@ -0,0 +1,241 @@
|
||||
import type { LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
asc,
|
||||
sql,
|
||||
type SQL,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import {
|
||||
chatActions,
|
||||
chatDeliveries,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
type ChatProviderLifecycleEffect,
|
||||
} from "../../chat-provider-lifecycle.js";
|
||||
|
||||
type DeliveryRow = typeof chatDeliveries.$inferSelect;
|
||||
|
||||
const DELIVERY_PROCESSING_STALE_MS = 60_000;
|
||||
|
||||
function deliveryContentWasRedacted(normalizedEvent: unknown): boolean {
|
||||
if (
|
||||
!normalizedEvent ||
|
||||
typeof normalizedEvent !== "object" ||
|
||||
Array.isArray(normalizedEvent)
|
||||
)
|
||||
return false;
|
||||
const filtering = (normalizedEvent as { filtering?: unknown }).filtering;
|
||||
return (
|
||||
Boolean(filtering) &&
|
||||
typeof filtering === "object" &&
|
||||
!Array.isArray(filtering) &&
|
||||
(filtering as { contentRetained?: unknown }).contentRetained === false
|
||||
);
|
||||
}
|
||||
|
||||
function setupDestinationActivationSuppressed(
|
||||
normalizedEvent: unknown,
|
||||
): boolean {
|
||||
if (
|
||||
!normalizedEvent ||
|
||||
typeof normalizedEvent !== "object" ||
|
||||
Array.isArray(normalizedEvent)
|
||||
)
|
||||
return false;
|
||||
const normalized = normalizedEvent as {
|
||||
admission?: { origin?: unknown };
|
||||
admissionAuthorization?: { version?: unknown };
|
||||
};
|
||||
return (
|
||||
normalized.admission?.origin === "provider_confirmed_action" ||
|
||||
normalized.admissionAuthorization?.version === 1
|
||||
);
|
||||
}
|
||||
|
||||
function normalizedDeliveryThreadId(delivery: Pick<DeliveryRow, "normalizedEvent">): string | null {
|
||||
const normalized = delivery.normalizedEvent as {
|
||||
conversation?: { externalThreadId?: unknown };
|
||||
};
|
||||
return typeof normalized.conversation?.externalThreadId === "string"
|
||||
? normalized.conversation.externalThreadId
|
||||
: null;
|
||||
}
|
||||
|
||||
function normalizedLifecycleTargetEventId(
|
||||
delivery: Pick<DeliveryRow, "eventKind" | "normalizedEvent">,
|
||||
): string | null {
|
||||
if (
|
||||
delivery.eventKind !== "message_updated" &&
|
||||
delivery.eventKind !== "message_deleted" &&
|
||||
delivery.eventKind !== "message_restored"
|
||||
)
|
||||
return null;
|
||||
const normalized = delivery.normalizedEvent as {
|
||||
message?: { targetProviderEventId?: unknown; admissionDependencyEventId?: unknown };
|
||||
};
|
||||
// An edit can also introduce a new addressed request. Process that durable
|
||||
// admission before waiting for the old, potentially unadmitted source.
|
||||
if (typeof normalized.message?.admissionDependencyEventId === "string")
|
||||
return normalized.message.admissionDependencyEventId;
|
||||
return typeof normalized.message?.targetProviderEventId === "string"
|
||||
? normalized.message.targetProviderEventId
|
||||
: null;
|
||||
}
|
||||
|
||||
function normalizedLifecycleEffect(
|
||||
delivery: Pick<DeliveryRow, "normalizedEvent">,
|
||||
): ChatProviderLifecycleEffect | null {
|
||||
const normalized = delivery.normalizedEvent as { lifecycle?: unknown };
|
||||
const value = normalized.lifecycle;
|
||||
if (!value || typeof value !== "object" || Array.isArray(value))
|
||||
return null;
|
||||
const effect = value as Record<string, unknown>;
|
||||
if (
|
||||
(effect.provider !== "slack" &&
|
||||
effect.provider !== "github" &&
|
||||
effect.provider !== "discord" &&
|
||||
effect.provider !== "microsoft-teams" &&
|
||||
effect.provider !== "telegram") ||
|
||||
typeof effect.providerEventId !== "string"
|
||||
)
|
||||
return null;
|
||||
if (
|
||||
effect.kind === "endpoint" &&
|
||||
(effect.availability === "available" ||
|
||||
effect.availability === "attention" ||
|
||||
effect.availability === "revoked") &&
|
||||
typeof effect.reason === "string"
|
||||
)
|
||||
return effect as ChatProviderLifecycleEffect;
|
||||
if (
|
||||
effect.kind === "resource" &&
|
||||
(effect.availability === "available" ||
|
||||
effect.availability === "unavailable" ||
|
||||
effect.availability === "removed") &&
|
||||
typeof effect.providerResourceId === "string" &&
|
||||
(effect.previousProviderResourceId === undefined ||
|
||||
typeof effect.previousProviderResourceId === "string") &&
|
||||
typeof effect.resourceType === "string" &&
|
||||
typeof effect.label === "string"
|
||||
)
|
||||
return effect as ChatProviderLifecycleEffect;
|
||||
return null;
|
||||
}
|
||||
|
||||
function deliveryReady(delivery: DeliveryRow, now: Date): boolean {
|
||||
// Accepted input may still own the conversation head through its wake
|
||||
// outbox. The action dispatcher checks its own retry/claim deadline.
|
||||
if (delivery.state === "processed") return true;
|
||||
if (delivery.state === "received")
|
||||
return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now;
|
||||
if (delivery.state === "retry") {
|
||||
return !delivery.nextAttemptAt || delivery.nextAttemptAt <= now;
|
||||
}
|
||||
return (
|
||||
delivery.state === "processing" &&
|
||||
delivery.updatedAt <=
|
||||
new Date(now.getTime() - DELIVERY_PROCESSING_STALE_MS)
|
||||
);
|
||||
}
|
||||
|
||||
function conversationDeliveryOrder() {
|
||||
return [
|
||||
asc(
|
||||
sql`coalesce(nullif(${chatDeliveries.normalizedEvent}->'message'->>'providerSentAt', '')::timestamptz, ${chatDeliveries.receivedAt})`,
|
||||
),
|
||||
// GitHub timestamps and Telegram message dates have one-second
|
||||
// resolution. GitHub's numeric comment id and Telegram's message_id
|
||||
// are monotonic within one conversation, so use them before receipt
|
||||
// order. The Telegram update_id is a final provider-native tie-breaker
|
||||
// for unusual payloads that lack a usable message_id.
|
||||
asc(sql`coalesce(
|
||||
case
|
||||
when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence' ~ '^[0-9]+$'
|
||||
then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageSequence')::numeric
|
||||
else null
|
||||
end,
|
||||
case
|
||||
when ${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId' ~ '^[0-9]+$'
|
||||
then (${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId')::numeric
|
||||
else null
|
||||
end
|
||||
)`),
|
||||
asc(sql`case
|
||||
when ${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId' ~ '^[0-9]+$'
|
||||
then (${chatDeliveries.normalizedEvent}->'message'->>'providerUpdateId')::numeric
|
||||
else null
|
||||
end`),
|
||||
// GitHub exposes no sortable webhook sequence. When an edit and delete
|
||||
// share its whole-second updated_at value, preserve the only valid
|
||||
// lifecycle state transition: update before delete. Provider-native
|
||||
// update ids (Telegram) remain the stronger preceding key.
|
||||
asc(sql`case ${chatDeliveries.eventKind}
|
||||
when 'message_updated' then 1
|
||||
when 'message_deleted' then 2
|
||||
when 'message_restored' then 3
|
||||
else 0
|
||||
end`),
|
||||
asc(chatDeliveries.receivedAt),
|
||||
asc(chatDeliveries.id),
|
||||
];
|
||||
}
|
||||
|
||||
function pendingInboundWakeupCondition(readyOnly = false): SQL {
|
||||
return sql`exists (
|
||||
select 1 from ${chatActions}
|
||||
where ${chatActions.deliveryId} = ${chatDeliveries.id}
|
||||
and ${chatActions.endpointId} = ${chatDeliveries.endpointId}
|
||||
and ${chatActions.companyId} = ${chatDeliveries.companyId}
|
||||
and ${chatActions.kind} = 'inbound_wakeup'
|
||||
and ${chatActions.status} in ('issued', 'processing')
|
||||
${
|
||||
readyOnly
|
||||
? sql`and (
|
||||
(${chatActions.status} = 'issued' and (${chatActions.result}->>'retryAt' is null or (${chatActions.result}->>'retryAt')::timestamptz <= now()))
|
||||
or (${chatActions.status} = 'processing' and ${chatActions.updatedAt} <= ${new Date(Date.now() - DELIVERY_PROCESSING_STALE_MS).toISOString()}::timestamptz)
|
||||
)`
|
||||
: sql``
|
||||
}
|
||||
)`;
|
||||
}
|
||||
|
||||
function lifecycleRuntimeFence(
|
||||
delivery: DeliveryRow,
|
||||
): LifecycleRuntimeFence | null {
|
||||
const normalized = delivery.normalizedEvent;
|
||||
if (
|
||||
!normalized ||
|
||||
typeof normalized !== "object" ||
|
||||
Array.isArray(normalized)
|
||||
)
|
||||
return null;
|
||||
const value = (normalized as Record<string, unknown>).runtimeContext;
|
||||
if (!value || typeof value !== "object" || Array.isArray(value))
|
||||
return null;
|
||||
const context = value as Record<string, unknown>;
|
||||
return typeof context.generation === "number" &&
|
||||
Number.isSafeInteger(context.generation) &&
|
||||
typeof context.credentialFingerprint === "string"
|
||||
? {
|
||||
generation: context.generation as number,
|
||||
credentialFingerprint: context.credentialFingerprint,
|
||||
}
|
||||
: null;
|
||||
}
|
||||
|
||||
function normalizedLifecycleWithRuntimeFence(
|
||||
delivery: DeliveryRow,
|
||||
runtimeContext: LifecycleRuntimeFence,
|
||||
): Record<string, unknown> {
|
||||
const normalized =
|
||||
delivery.normalizedEvent &&
|
||||
typeof delivery.normalizedEvent === "object" &&
|
||||
!Array.isArray(delivery.normalizedEvent)
|
||||
? (delivery.normalizedEvent as Record<string, unknown>)
|
||||
: {};
|
||||
return { ...normalized, runtimeContext };
|
||||
}
|
||||
|
||||
export { DELIVERY_PROCESSING_STALE_MS, deliveryContentWasRedacted, setupDestinationActivationSuppressed, normalizedDeliveryThreadId, normalizedLifecycleTargetEventId, normalizedLifecycleEffect, deliveryReady, conversationDeliveryOrder, pendingInboundWakeupCondition, lifecycleRuntimeFence, normalizedLifecycleWithRuntimeFence };
|
||||
@@ -0,0 +1,360 @@
|
||||
import { providerResourceType, durableExternalThreadIdentity, canonicalProviderResourceId, chatSurfaceKind } from "./surfaces.js";
|
||||
|
||||
import type { EndpointRow, InboundRuntimeContext } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
MAX_INBOUND_TEXT,
|
||||
stableExternalPrincipalId,
|
||||
} from "../message-lifecycle.js";
|
||||
import {
|
||||
telegramMessageSequence,
|
||||
telegramMessageSentAt,
|
||||
} from "../telegram/messages.js";
|
||||
import {
|
||||
slackMessageSentAt,
|
||||
} from "../slack/messages.js";
|
||||
import {
|
||||
teamsActivitySentAt,
|
||||
} from "../microsoft-teams/messages.js";
|
||||
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
|
||||
import { chatGitHubConfigurations } from "@paperclipai/db";
|
||||
import type { GitHubReviewEventContext, GitHubIssueEventContext, GitHubReviewPolicy } from "@paperclipai/shared";
|
||||
|
||||
import { photonReplyReference } from "../../photon/adapter.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatEndpointResources,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatEventKind,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
normalizeContentType,
|
||||
} from "../../../attachment-types.js";
|
||||
import {
|
||||
deriveTeamsPersonalRecipient,
|
||||
} from "../../chat-teams-personal-recipient.js";
|
||||
|
||||
import {
|
||||
type ChatSdkMessageCallbackEvent,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
|
||||
import {
|
||||
githubAttachmentLimitOmissions,
|
||||
} from "../../chat-github-attachments.js";
|
||||
|
||||
import { chatProviderConversationUrl } from "../../chat-provider-links.js";
|
||||
|
||||
import type {
|
||||
Attachment,
|
||||
Message,
|
||||
Thread,
|
||||
} from "chat";
|
||||
|
||||
export type InboundEnvelopeInput = {
|
||||
endpoint: EndpointRow;
|
||||
thread: Thread;
|
||||
message: Message;
|
||||
trigger: ChatSdkMessageCallbackEvent["trigger"];
|
||||
receiptReactionSupported: boolean;
|
||||
recoveredProviderUrl: string | null;
|
||||
runtimeContext?: InboundRuntimeContext;
|
||||
providerUpdateId?: number;
|
||||
admittedDeliveryId: string | null;
|
||||
suppressSetupDestinationActivation: boolean;
|
||||
slackSlashControl: boolean;
|
||||
};
|
||||
type InboundEnvelopeOptions = {
|
||||
runtimeFor: (endpoint: EndpointRow) => Promise<ChatEndpointRuntime>;
|
||||
sanitizeFilename: (value: string | undefined) => string | null;
|
||||
githubManualMessages: WeakMap<object, { policy: GitHubReviewPolicy; revision: number; event: "mention" | "comment" }>;
|
||||
githubAutomaticMessages: WeakMap<object, { context: GitHubReviewEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
githubIssueMessages: WeakMap<object, { context: GitHubIssueEventContext; revision: number; policy: GitHubReviewPolicy }>;
|
||||
};
|
||||
export function createInboundEnvelope(db: Db, options: InboundEnvelopeOptions) {
|
||||
const { runtimeFor, sanitizeFilename, githubManualMessages, githubAutomaticMessages, githubIssueMessages } = options;
|
||||
async function prepareInboundEnvelope(input: InboundEnvelopeInput) {
|
||||
const { endpoint, thread, message, trigger, receiptReactionSupported, recoveredProviderUrl, runtimeContext, providerUpdateId, admittedDeliveryId, suppressSetupDestinationActivation, slackSlashControl } = input;
|
||||
const githubAutomatic = endpoint.provider === "github" ? githubAutomaticMessages.get(message) : undefined;
|
||||
const githubIssue = endpoint.provider === "github" ? githubIssueMessages.get(message) : undefined;
|
||||
let githubManual = githubManualMessages.get(message);
|
||||
if (endpoint.provider === "github" && !githubAutomatic && !githubIssue && !githubManual) {
|
||||
const [config] = await db.select().from(chatGitHubConfigurations).where(eq(chatGitHubConfigurations.endpointId, endpoint.id));
|
||||
if (config) {
|
||||
const repository = thread.channelId.replace(/^github:/, "").toLowerCase();
|
||||
const [repo] = await db.select().from(chatEndpointResources).where(and(eq(chatEndpointResources.endpointId, endpoint.id), eq(chatEndpointResources.providerResourceId, repository)));
|
||||
const repositoryId = String(repo?.metadata?.providerRepositoryId ?? "");
|
||||
githubManual = { policy: { ...config.configuration.defaults, ...config.configuration.repositories[repositoryId] }, revision: config.revision, event: trigger === "mention" || message.isMention ? "mention" : "comment" };
|
||||
}
|
||||
}
|
||||
const providerEventId = `${durableExternalThreadIdentity(thread.id)}:${message.id}`;
|
||||
const surfaceKind = chatSurfaceKind(endpoint.provider, thread);
|
||||
const addressed =
|
||||
endpoint.provider === "imessage-photon" ||
|
||||
trigger === "mention" ||
|
||||
trigger === "direct_message" ||
|
||||
message.isMention === true;
|
||||
const eventKind: ChatEventKind =
|
||||
trigger === "direct_message"
|
||||
? "direct_message"
|
||||
: trigger === "mention"
|
||||
? "mention"
|
||||
: "message";
|
||||
// A callback remains tied to the runtime that authenticated and parsed it.
|
||||
// In particular, do not resolve the registry again here: pause or rotation
|
||||
// may already have removed that instance, and recreating a runtime before
|
||||
// the durable admission fence would either throw or bind old payload data
|
||||
// to new credentials.
|
||||
const endpointRuntime =
|
||||
runtimeContext?.endpointRuntime ?? (await runtimeFor(endpoint));
|
||||
// Non-personal generic files still require a separate Graph grant. Only
|
||||
// exact inline image resources from the authenticated original activity
|
||||
// receive a scope-bound, URL-free recovery locator. Reconstructed input
|
||||
// can retain that proof, but cannot mint one from legacy fetch metadata.
|
||||
const teamsNonPersonal =
|
||||
endpoint.provider === "microsoft-teams" && !thread.isDM;
|
||||
const attachmentSource = {
|
||||
threadId: thread.id,
|
||||
messageId: message.id,
|
||||
runtimeGeneration: runtimeContext?.generation,
|
||||
credentialFingerprint: runtimeContext?.credentialFingerprint,
|
||||
principalExternalId: stableExternalPrincipalId(
|
||||
endpoint.provider,
|
||||
message.author,
|
||||
message.raw,
|
||||
),
|
||||
isDirectMessage: thread.isDM,
|
||||
...(runtimeContext?.endpointRuntime && !admittedDeliveryId
|
||||
? { message }
|
||||
: {}),
|
||||
};
|
||||
const inlineRecovery = new Map<
|
||||
Attachment,
|
||||
ReturnType<ChatEndpointRuntime["attachmentRecoveryDescriptor"]>
|
||||
>();
|
||||
const nativeInboundAttachments = teamsNonPersonal
|
||||
? message.attachments.slice(0, 20).flatMap((attachment) => {
|
||||
const descriptor = endpointRuntime.attachmentRecoveryDescriptor(
|
||||
attachment,
|
||||
attachmentSource,
|
||||
);
|
||||
if (descriptor?.locator.kind !== "teams_inline_image") return [];
|
||||
const restored = endpointRuntime.rehydrateAttachment(
|
||||
descriptor,
|
||||
attachmentSource,
|
||||
);
|
||||
if (!restored) return [];
|
||||
inlineRecovery.set(attachment, descriptor);
|
||||
return [restored];
|
||||
})
|
||||
: message.attachments;
|
||||
const unavailableTeamsReferences = teamsNonPersonal
|
||||
? Math.min(message.attachments.length, 20) -
|
||||
nativeInboundAttachments.length
|
||||
: 0;
|
||||
const providerSentAt =
|
||||
endpoint.provider === "microsoft-teams"
|
||||
? teamsActivitySentAt(message.raw)
|
||||
: endpoint.provider === "telegram"
|
||||
? telegramMessageSentAt(message.raw)
|
||||
: endpoint.provider === "slack"
|
||||
? slackMessageSentAt(message.raw, message.id)
|
||||
: message.metadata.dateSent instanceof Date &&
|
||||
Number.isFinite(message.metadata.dateSent.getTime())
|
||||
? message.metadata.dateSent
|
||||
: null;
|
||||
const providerSentAtSource = providerSentAt
|
||||
? endpoint.provider === "microsoft-teams"
|
||||
? "teams_activity_timestamp"
|
||||
: endpoint.provider === "telegram"
|
||||
? "telegram_message_date"
|
||||
: endpoint.provider === "slack"
|
||||
? "slack_message_ts"
|
||||
: null
|
||||
: null;
|
||||
const providerUrl =
|
||||
(githubAutomatic
|
||||
? `https://github.com/${githubAutomatic.context.repository}/pull/${githubAutomatic.context.pullNumber}`
|
||||
: githubIssue ? `https://github.com/${githubIssue.context.repository}/issues/${githubIssue.context.issueNumber}` : null) ??
|
||||
// Description/review requests and edits have synthetic ledger IDs, not
|
||||
// native comment IDs. Their verified source URL must not acquire a fake
|
||||
// issuecomment anchor from that ledger identity.
|
||||
(endpoint.provider === "github" && githubManual && !receiptReactionSupported ? recoveredProviderUrl : null) ??
|
||||
chatProviderConversationUrl({
|
||||
provider: endpoint.provider,
|
||||
providerAccountId: endpoint.providerAccountId,
|
||||
botUsername: endpoint.botUsername,
|
||||
threadId: thread.id,
|
||||
providerMessageId: message.id,
|
||||
raw: message.raw,
|
||||
}) ?? recoveredProviderUrl;
|
||||
// Preserve only the closed personal-recipient proof from the runtime that
|
||||
// authenticated this original activity. A reconstructed delivery's AAD-only
|
||||
// author and SDK route caches cannot recreate its Bot Framework recipient.
|
||||
const teamsPersonalRecipient =
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
runtimeContext?.endpointRuntime &&
|
||||
!admittedDeliveryId &&
|
||||
endpoint.providerAccountId &&
|
||||
endpoint.botExternalId
|
||||
? deriveTeamsPersonalRecipient(message, {
|
||||
companyId: endpoint.companyId,
|
||||
endpointId: endpoint.id,
|
||||
runtimeGeneration: runtimeContext.generation,
|
||||
credentialFingerprint: runtimeContext.credentialFingerprint,
|
||||
tenantId: endpoint.providerAccountId,
|
||||
botAppId: endpoint.botExternalId,
|
||||
providerEventId,
|
||||
threadId: thread.id,
|
||||
isDirectMessage: thread.isDM,
|
||||
})
|
||||
: null;
|
||||
const normalized = {
|
||||
providerEventId,
|
||||
kind: eventKind,
|
||||
trigger,
|
||||
...(teamsPersonalRecipient ? { teamsPersonalRecipient } : {}),
|
||||
...(slackSlashControl
|
||||
? { admission: { origin: "slack_slash_control" } }
|
||||
: suppressSetupDestinationActivation
|
||||
? // A slash-command root is provider-confirmed only after an enabled
|
||||
// destination authorized its transport. Persist that closed origin so
|
||||
// crash recovery can never reinterpret it as first-time setup traffic
|
||||
// and undo a later operator reach revocation.
|
||||
{ admission: { origin: "provider_confirmed_action" } }
|
||||
: {}),
|
||||
...(runtimeContext
|
||||
? {
|
||||
runtimeContext: {
|
||||
credentialFingerprint: runtimeContext.credentialFingerprint,
|
||||
generation: runtimeContext.generation,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
acknowledgement: {
|
||||
// Some provider callbacks represent an auditable user command without
|
||||
// a provider message that can receive a reaction. Persist the
|
||||
// distinction so a crash/retry cannot mistake the synthetic ledger id
|
||||
// for a native message id.
|
||||
receiptReactionSupported,
|
||||
},
|
||||
...(githubAutomatic ? { githubAutomatic } : {}),
|
||||
...(githubIssue ? { githubIssue } : {}),
|
||||
...(githubManual ? { githubManual } : {}),
|
||||
principal: {
|
||||
// Bot/system authors have already been rejected at this intake boundary.
|
||||
isBot: false,
|
||||
externalId: stableExternalPrincipalId(
|
||||
endpoint.provider,
|
||||
message.author,
|
||||
message.raw,
|
||||
),
|
||||
displayName: message.author.fullName,
|
||||
handle: message.author.userName,
|
||||
},
|
||||
resource: {
|
||||
type: providerResourceType(endpoint.provider, surfaceKind),
|
||||
providerResourceId: canonicalProviderResourceId(
|
||||
endpoint.provider,
|
||||
thread,
|
||||
),
|
||||
label: thread.channel.name ?? thread.channelId,
|
||||
},
|
||||
conversation: {
|
||||
externalConversationId: thread.channelId,
|
||||
externalThreadId: thread.id,
|
||||
label: thread.channel.name ?? thread.channelId,
|
||||
isDirectMessage: thread.isDM,
|
||||
providerUrl,
|
||||
},
|
||||
message: {
|
||||
providerMessageId: message.id,
|
||||
...(endpoint.provider === "github" && typeof (message.raw as { githubSourceMessageId?: unknown })?.githubSourceMessageId === "string" && /^[1-9][0-9]*$/.test((message.raw as { githubSourceMessageId: string }).githubSourceMessageId)
|
||||
? { sourceProviderMessageId: (message.raw as { githubSourceMessageId: string }).githubSourceMessageId } : {}),
|
||||
...(endpoint.provider === "imessage-photon"
|
||||
? { photonReply: photonReplyReference(message.raw) }
|
||||
: {}),
|
||||
providerMessageSequence:
|
||||
endpoint.provider === "telegram"
|
||||
? telegramMessageSequence(message.raw)
|
||||
: null,
|
||||
providerUpdateId:
|
||||
endpoint.provider === "telegram" ? (providerUpdateId ?? null) : null,
|
||||
text: message.text.slice(0, MAX_INBOUND_TEXT),
|
||||
mentionedBot: message.isMention === true,
|
||||
providerSentAt: providerSentAt?.toISOString() ?? null,
|
||||
...(providerSentAtSource ? { providerSentAtSource } : {}),
|
||||
...((endpoint.provider === "github" ||
|
||||
endpoint.provider === "imessage-photon") &&
|
||||
(githubAttachmentLimitOmissions(message) ||
|
||||
Math.max(0, message.attachments.length - 20))
|
||||
? {
|
||||
attachmentLimitOmissions:
|
||||
githubAttachmentLimitOmissions(message) ||
|
||||
Math.max(0, message.attachments.length - 20),
|
||||
}
|
||||
: {}),
|
||||
attachments: message.attachments.slice(0, 20).map((attachment) => ({
|
||||
name: sanitizeFilename(attachment.name),
|
||||
mimeType: normalizeContentType(attachment.mimeType),
|
||||
size: attachment.size ?? null,
|
||||
recovery: teamsNonPersonal
|
||||
? (inlineRecovery.get(attachment) ?? null)
|
||||
: nativeInboundAttachments.includes(attachment)
|
||||
? endpointRuntime.attachmentRecoveryDescriptor(
|
||||
attachment,
|
||||
["telegram", "imessage-photon"].includes(endpoint.provider)
|
||||
? attachmentSource
|
||||
: undefined,
|
||||
)
|
||||
: null,
|
||||
})),
|
||||
},
|
||||
};
|
||||
// Teams RSC can deliver messages from every installed channel, and a
|
||||
// Telegram bot can receive addressed traffic from a provider-available
|
||||
// group that the operator has not enabled in Paperclip. Until the reach
|
||||
// gate admits that destination, retain only identifiers required for
|
||||
// deduplication and operator-visible filtering—not user text, attachment
|
||||
// metadata, or an external principal profile.
|
||||
const redactedDestinationNormalized = {
|
||||
providerEventId,
|
||||
kind: eventKind,
|
||||
trigger,
|
||||
...(runtimeContext
|
||||
? {
|
||||
runtimeContext: {
|
||||
credentialFingerprint: runtimeContext.credentialFingerprint,
|
||||
generation: runtimeContext.generation,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
resource: {
|
||||
type: providerResourceType(endpoint.provider, surfaceKind),
|
||||
providerResourceId: canonicalProviderResourceId(
|
||||
endpoint.provider,
|
||||
thread,
|
||||
),
|
||||
},
|
||||
conversation: { externalThreadId: thread.id },
|
||||
message: {
|
||||
providerMessageId: message.id,
|
||||
providerSentAt: providerSentAt?.toISOString() ?? null,
|
||||
},
|
||||
filtering: { contentRetained: false },
|
||||
};
|
||||
return { githubAutomatic, githubIssue, githubManual, providerEventId, surfaceKind, addressed, eventKind, endpointRuntime, nativeInboundAttachments, unavailableTeamsReferences, providerSentAt, providerUrl, normalized, redactedDestinationNormalized };
|
||||
}
|
||||
return { prepareInboundEnvelope };
|
||||
}
|
||||
export type InboundEnvelope = Awaited<ReturnType<ReturnType<typeof createInboundEnvelope>["prepareInboundEnvelope"]>>;
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,113 @@
|
||||
import { createChatInboundAuthority } from "./authorization.js";
|
||||
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
ne,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatEndpointResources,
|
||||
chatEndpoints,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
notFound,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import type {
|
||||
Thread,
|
||||
} from "chat";
|
||||
|
||||
import type { ChatSurfaceKind } from "./surfaces.js";
|
||||
|
||||
type MessageResourceAdmissionInput = {
|
||||
endpoint: EndpointRow;
|
||||
thread: Thread;
|
||||
addressed: boolean;
|
||||
suppressSetupDestinationActivation: boolean;
|
||||
surfaceKind: ChatSurfaceKind;
|
||||
};
|
||||
export function createMessageResourceAdmission(db: Db, options: { ensureResource: ReturnType<typeof createChatInboundAuthority>["ensureResource"] }) {
|
||||
const { ensureResource } = options;
|
||||
async function admitMessageResource(input: MessageResourceAdmissionInput) {
|
||||
const { endpoint, thread, addressed, suppressSetupDestinationActivation, surfaceKind } = input;
|
||||
const mayEnableSetupDestination =
|
||||
// Slack channels start enabled. Never use first-test activation to
|
||||
// override an existing channel the operator explicitly disabled.
|
||||
endpoint.provider !== "slack" &&
|
||||
endpoint.provider !== "imessage-photon" &&
|
||||
!thread.isDM &&
|
||||
endpoint.status === "verifying" &&
|
||||
addressed &&
|
||||
!suppressSetupDestinationActivation &&
|
||||
!(
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
surfaceKind === "linear_group"
|
||||
);
|
||||
const resourceAdmission = mayEnableSetupDestination
|
||||
? await db.transaction(async (tx) => {
|
||||
const currentEndpoint = await tx
|
||||
.select({ status: chatEndpoints.status })
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, endpoint.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!currentEndpoint) throw notFound("Chat endpoint not found");
|
||||
const enabledResourceCount = await tx
|
||||
.select({ count: sql<number>`count(*)::int` })
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.enabled, true),
|
||||
eq(chatEndpointResources.availability, "available"),
|
||||
ne(chatEndpointResources.type, "direct_message"),
|
||||
ne(chatEndpointResources.type, "group_chat"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]?.count ?? 0);
|
||||
const enableSetupDestination =
|
||||
currentEndpoint.status === "verifying" &&
|
||||
enabledResourceCount === 0;
|
||||
let resource = await ensureResource(
|
||||
endpoint,
|
||||
thread,
|
||||
enableSetupDestination,
|
||||
tx,
|
||||
);
|
||||
if (enableSetupDestination && !resource.enabled) {
|
||||
resource = await tx
|
||||
.update(chatEndpointResources)
|
||||
.set({ enabled: true, updatedAt: new Date() })
|
||||
.where(eq(chatEndpointResources.id, resource.id))
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? resource);
|
||||
}
|
||||
return { enabledResourceCount, resource };
|
||||
})
|
||||
: {
|
||||
enabledResourceCount: await db
|
||||
.select({ count: sql<number>`count(*)::int` })
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.enabled, true),
|
||||
eq(chatEndpointResources.availability, "available"),
|
||||
ne(chatEndpointResources.type, "direct_message"),
|
||||
ne(chatEndpointResources.type, "group_chat"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]?.count ?? 0),
|
||||
resource: await ensureResource(endpoint, thread, false),
|
||||
};
|
||||
return resourceAdmission;
|
||||
}
|
||||
return { admitMessageResource };
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
import { slackThreadChannelId } from "../slack/messages.js";
|
||||
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
|
||||
import { baseTeamsConversationId } from "../microsoft-teams/resources.js";
|
||||
import { slackResourceLabelIsFallback } from "../slack/resources.js";
|
||||
import { telegramResourceLabelIsFallback } from "../telegram/resources.js";
|
||||
|
||||
import {
|
||||
canonicalTeamsThreadId,
|
||||
teamsConversationId,
|
||||
} from "../../chat-control-chronology.js";
|
||||
|
||||
import {
|
||||
chatEndpointResources,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatProvider,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import type {
|
||||
Thread,
|
||||
} from "chat";
|
||||
|
||||
type ResourceRow = typeof chatEndpointResources.$inferSelect;
|
||||
|
||||
|
||||
type ChatSurfaceKind = "direct_message" | "linear_group" | "native_thread";
|
||||
|
||||
function providerResourceType(
|
||||
provider: ChatProvider,
|
||||
surfaceKind: ChatSurfaceKind,
|
||||
): string {
|
||||
if (surfaceKind === "direct_message") return "direct_message";
|
||||
if (provider === "github") return "repository";
|
||||
if (provider === "imessage-photon") return "group_chat";
|
||||
if (provider === "discord") return "channel";
|
||||
if (provider === "microsoft-teams")
|
||||
return surfaceKind === "linear_group" ? "group_chat" : "channel";
|
||||
// Telegram topics are task/thread boundaries inside one group resource. The
|
||||
// operator grants access to the chat once rather than having to rediscover
|
||||
// and enable every topic independently.
|
||||
if (provider === "telegram") return "chat";
|
||||
return "channel";
|
||||
}
|
||||
|
||||
|
||||
function sameTeamsThreadIdentity(left: string, right: string): boolean {
|
||||
const leftConversationId = teamsConversationId(left);
|
||||
const rightConversationId = teamsConversationId(right);
|
||||
return (
|
||||
leftConversationId !== null &&
|
||||
rightConversationId !== null &&
|
||||
leftConversationId === rightConversationId
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
function durableExternalThreadIdentity(threadId: string): string {
|
||||
return canonicalTeamsThreadId(threadId) ?? threadId;
|
||||
}
|
||||
|
||||
function teamsThreadRootMessageId(threadId: string): string | null {
|
||||
const conversationId = teamsConversationId(threadId);
|
||||
if (!conversationId) return null;
|
||||
return /;messageid=([^;]+)/i.exec(conversationId)?.[1] ?? null;
|
||||
}
|
||||
|
||||
function canonicalProviderResourceId(
|
||||
provider: ChatProvider,
|
||||
thread: Pick<Thread, "channelId" | "id">,
|
||||
): string {
|
||||
if (provider === "slack") return thread.channelId.replace(/^slack:/, "");
|
||||
if (provider === "github")
|
||||
return thread.channelId.replace(/^github:/, "").toLowerCase();
|
||||
if (provider === "discord") {
|
||||
const parts = thread.channelId.split(":");
|
||||
return parts[2] ?? thread.channelId;
|
||||
}
|
||||
if (provider === "microsoft-teams") {
|
||||
const conversationId =
|
||||
teamsConversationId(thread.channelId) ?? teamsConversationId(thread.id);
|
||||
return conversationId
|
||||
? baseTeamsConversationId(conversationId)
|
||||
: baseTeamsConversationId(thread.channelId);
|
||||
}
|
||||
if (provider === "telegram")
|
||||
return thread.channelId.replace(/^telegram:/, "");
|
||||
return thread.channelId;
|
||||
}
|
||||
|
||||
function providerResourceLabelFromThread(
|
||||
provider: ChatProvider,
|
||||
thread: Pick<Thread, "channelId" | "id" | "channel">,
|
||||
): { label: string; fallback: boolean } {
|
||||
const providerResourceId = canonicalProviderResourceId(provider, thread);
|
||||
const candidate = thread.channel.name?.trim();
|
||||
if (!candidate) return { label: providerResourceId, fallback: true };
|
||||
if (
|
||||
(provider === "slack" &&
|
||||
slackResourceLabelIsFallback(providerResourceId, candidate)) ||
|
||||
(provider === "telegram" &&
|
||||
telegramResourceLabelIsFallback(providerResourceId, candidate))
|
||||
) {
|
||||
return { label: providerResourceId, fallback: true };
|
||||
}
|
||||
return { label: candidate, fallback: false };
|
||||
}
|
||||
|
||||
/**
|
||||
* Native channel threads keep one issue forever. Linear surfaces instead
|
||||
* advance through Paperclip task generations because their provider id is
|
||||
* reused after a task completes or the user explicitly starts a new task.
|
||||
*/
|
||||
function chatSurfaceKind(
|
||||
provider: ChatProvider,
|
||||
thread: Thread,
|
||||
): ChatSurfaceKind {
|
||||
if (thread.isDM) return "direct_message";
|
||||
if (provider === "imessage-photon") return "linear_group";
|
||||
if (provider === "telegram") {
|
||||
return /^telegram:[^:]+:[^:]+$/.test(thread.id)
|
||||
? "native_thread"
|
||||
: "linear_group";
|
||||
}
|
||||
if (provider === "microsoft-teams") {
|
||||
const conversationId = teamsConversationId(thread.id);
|
||||
return conversationId?.includes(";messageid=")
|
||||
? "native_thread"
|
||||
: "linear_group";
|
||||
}
|
||||
return "native_thread";
|
||||
}
|
||||
|
||||
/**
|
||||
* Teams group chats are admitted as a class: installation at the provider and
|
||||
* the endpoint's group-chat toggle are the two gates the operator can
|
||||
* actually control. Teams channels, and every other non-DM destination,
|
||||
* retain the explicit per-resource enablement gate shown in Settings.
|
||||
*/
|
||||
function nonDirectDestinationAllowed(
|
||||
endpoint: EndpointRow,
|
||||
resource: ResourceRow | null | undefined,
|
||||
): boolean {
|
||||
if (!resource || resource.availability !== "available") return false;
|
||||
if (
|
||||
endpoint.provider === "microsoft-teams" &&
|
||||
resource.type === "group_chat"
|
||||
) {
|
||||
return endpoint.allowGroupChats;
|
||||
}
|
||||
return resource.enabled;
|
||||
}
|
||||
|
||||
function linearControlCommand(text: string): "new" | "close" | "status" | null {
|
||||
const match = /^\/(new|close|status)(?:@[\w.-]+)?\s*$/i.exec(text.trim());
|
||||
return (
|
||||
(match?.[1]?.toLowerCase() as "new" | "close" | "status" | undefined) ??
|
||||
null
|
||||
);
|
||||
}
|
||||
|
||||
function actionThreadMatchesConversation(
|
||||
provider: ChatProvider,
|
||||
actionThreadId: string,
|
||||
conversationThreadId: string,
|
||||
): boolean {
|
||||
if (provider === "microsoft-teams") {
|
||||
return sameTeamsThreadIdentity(actionThreadId, conversationThreadId);
|
||||
}
|
||||
if (provider !== "slack") return actionThreadId === conversationThreadId;
|
||||
const actionChannelId = slackThreadChannelId(actionThreadId);
|
||||
return (
|
||||
actionChannelId !== null &&
|
||||
actionChannelId === slackThreadChannelId(conversationThreadId)
|
||||
);
|
||||
}
|
||||
|
||||
export { providerResourceType, sameTeamsThreadIdentity, durableExternalThreadIdentity, teamsThreadRootMessageId, canonicalProviderResourceId, providerResourceLabelFromThread, chatSurfaceKind, nonDirectDestinationAllowed, linearControlCommand, actionThreadMatchesConversation };
|
||||
export type { ChatSurfaceKind };
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,432 @@
|
||||
import {
|
||||
and,
|
||||
asc,
|
||||
desc,
|
||||
eq,
|
||||
isNotNull,
|
||||
isNull,
|
||||
lte,
|
||||
notInArray,
|
||||
or,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
agentWakeupRequests,
|
||||
chatActions,
|
||||
chatEndpoints,
|
||||
chatMessageLinks,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import { isUniqueViolation } from "../../../db-errors.js";
|
||||
|
||||
import { logger } from "../../../middleware/logger.js";
|
||||
|
||||
import {
|
||||
type IssueAssignmentWakeupDeps,
|
||||
} from "../../issue-assignment-wakeup.js";
|
||||
|
||||
import { issueThreadInteractionService } from "../../issue-thread-interactions.js";
|
||||
|
||||
type InteractionContinuationsOptions = {
|
||||
heartbeat: IssueAssignmentWakeupDeps;
|
||||
redactError: (error: unknown) => string;
|
||||
CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS: number;
|
||||
};
|
||||
|
||||
export function createChatInteractionContinuations(db: Db, options: InteractionContinuationsOptions) {
|
||||
const { redactError, CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS } = options;
|
||||
|
||||
|
||||
async function processPendingInteractionWakeups(
|
||||
limit = 100,
|
||||
): Promise<number> {
|
||||
const staleBefore = new Date(Date.now() - 30_000);
|
||||
const retryBefore = new Date(
|
||||
Date.now() - CONFIRMATION_WAKEUP_RETRY_BACKOFF_MS,
|
||||
);
|
||||
const candidates = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.kind, "interaction_wakeup"),
|
||||
or(
|
||||
and(
|
||||
eq(chatActions.status, "issued"),
|
||||
or(
|
||||
isNull(chatActions.result),
|
||||
lte(chatActions.updatedAt, retryBefore),
|
||||
),
|
||||
),
|
||||
and(
|
||||
eq(chatActions.status, "processing"),
|
||||
lte(chatActions.updatedAt, staleBefore),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
.orderBy(asc(chatActions.updatedAt), asc(chatActions.id))
|
||||
.limit(limit);
|
||||
let processed = 0;
|
||||
for (const candidate of candidates) {
|
||||
const [claimed] = await db
|
||||
.update(chatActions)
|
||||
.set({ status: "processing", updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, candidate.id),
|
||||
or(
|
||||
and(
|
||||
eq(chatActions.status, "issued"),
|
||||
or(
|
||||
isNull(chatActions.result),
|
||||
lte(chatActions.updatedAt, retryBefore),
|
||||
),
|
||||
),
|
||||
and(
|
||||
eq(chatActions.status, "processing"),
|
||||
lte(chatActions.updatedAt, staleBefore),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
if (!claimed) continue;
|
||||
const payload = claimed.payload;
|
||||
const requestedByActorType =
|
||||
payload.requestedByActorType === "user" ||
|
||||
payload.requestedByActorType === "agent" ||
|
||||
payload.requestedByActorType === "system"
|
||||
? payload.requestedByActorType
|
||||
: typeof payload.requestedByUserId === "string"
|
||||
? "user"
|
||||
: null;
|
||||
const requestedByActorId =
|
||||
typeof payload.requestedByActorId === "string"
|
||||
? payload.requestedByActorId
|
||||
: typeof payload.requestedByUserId === "string"
|
||||
? payload.requestedByUserId
|
||||
: null;
|
||||
if (
|
||||
payload.version !== 1 ||
|
||||
typeof payload.interactionId !== "string" ||
|
||||
typeof payload.interactionKind !== "string" ||
|
||||
!["accepted", "rejected", "answered", "cancelled", "failed"].includes(
|
||||
String(payload.interactionStatus),
|
||||
) ||
|
||||
typeof payload.issueId !== "string" ||
|
||||
typeof payload.agentId !== "string" ||
|
||||
!requestedByActorType ||
|
||||
!requestedByActorId
|
||||
) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "failed",
|
||||
result: { code: "interaction_wakeup_payload_invalid" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, claimed.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const currentIssue = await db
|
||||
.select({
|
||||
assigneeAgentId: issues.assigneeAgentId,
|
||||
status: issues.status,
|
||||
})
|
||||
.from(issues)
|
||||
.where(
|
||||
and(
|
||||
eq(issues.companyId, claimed.companyId),
|
||||
eq(issues.id, payload.issueId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
const currentInteraction = currentIssue
|
||||
? (
|
||||
await issueThreadInteractionService(db).listForIssue(
|
||||
payload.issueId,
|
||||
)
|
||||
).find((interaction) => interaction.id === payload.interactionId)
|
||||
: null;
|
||||
if (
|
||||
!currentIssue ||
|
||||
!currentInteraction ||
|
||||
currentInteraction.companyId !== claimed.companyId ||
|
||||
currentInteraction.kind !== payload.interactionKind ||
|
||||
currentInteraction.status !== payload.interactionStatus ||
|
||||
currentIssue.assigneeAgentId !== payload.agentId ||
|
||||
currentIssue.status === "done" ||
|
||||
currentIssue.status === "cancelled"
|
||||
) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "processed",
|
||||
result: { code: "interaction_wakeup_no_longer_applicable" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, claimed.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
processed += 1;
|
||||
continue;
|
||||
}
|
||||
const idempotencyKey = `interaction:${payload.interactionId}:${payload.interactionStatus}`;
|
||||
try {
|
||||
const sourceCommentId = currentInteraction.sourceCommentId ?? null;
|
||||
const sourceRunId = currentInteraction.sourceRunId ?? null;
|
||||
// A resolved interaction is still part of the originating external
|
||||
// turn. Preserve one verified inbound comment edge on the continuation
|
||||
// run so queued/working/failure milestones return to this exact
|
||||
// endpoint and conversation instead of disappearing as an internal
|
||||
// automation run.
|
||||
const chatOrigin = claimed.conversationId
|
||||
? await db
|
||||
.select({
|
||||
commentId: chatMessageLinks.commentId,
|
||||
provider: chatEndpoints.provider,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.innerJoin(
|
||||
chatMessageLinks,
|
||||
and(
|
||||
eq(chatMessageLinks.companyId, chatEndpoints.companyId),
|
||||
eq(chatMessageLinks.endpointId, chatEndpoints.id),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.companyId, claimed.companyId),
|
||||
eq(chatEndpoints.id, claimed.endpointId),
|
||||
eq(chatMessageLinks.conversationId, claimed.conversationId),
|
||||
eq(chatMessageLinks.direction, "inbound"),
|
||||
isNotNull(chatMessageLinks.commentId),
|
||||
),
|
||||
)
|
||||
.orderBy(
|
||||
desc(chatMessageLinks.createdAt),
|
||||
desc(chatMessageLinks.id),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const wakeCommentId = sourceCommentId ?? chatOrigin?.commentId ?? null;
|
||||
const existing = await db
|
||||
.select({
|
||||
agentId: agentWakeupRequests.agentId,
|
||||
id: agentWakeupRequests.id,
|
||||
})
|
||||
.from(agentWakeupRequests)
|
||||
.where(
|
||||
and(
|
||||
eq(agentWakeupRequests.companyId, claimed.companyId),
|
||||
eq(agentWakeupRequests.idempotencyKey, idempotencyKey),
|
||||
notInArray(agentWakeupRequests.status, [
|
||||
"skipped",
|
||||
"failed",
|
||||
"cancelled",
|
||||
]),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
const queuedRun = !existing
|
||||
? await options.heartbeat.wakeup(payload.agentId, {
|
||||
source: "automation",
|
||||
triggerDetail: "system",
|
||||
reason: "issue_commented",
|
||||
payload: {
|
||||
issueId: payload.issueId,
|
||||
interactionId: payload.interactionId,
|
||||
interactionKind: payload.interactionKind,
|
||||
interactionStatus: payload.interactionStatus,
|
||||
sourceCommentId,
|
||||
sourceRunId,
|
||||
...(wakeCommentId
|
||||
? {
|
||||
wakeCommentId,
|
||||
wakeCommentIds: [wakeCommentId],
|
||||
}
|
||||
: {}),
|
||||
...(payload.planReviewInteraction &&
|
||||
typeof payload.planReviewInteraction === "object"
|
||||
? { planReviewInteraction: payload.planReviewInteraction }
|
||||
: {}),
|
||||
mutation: "interaction",
|
||||
},
|
||||
idempotencyKey,
|
||||
allowRunCoalescing: false,
|
||||
requestedByActorType,
|
||||
requestedByActorId,
|
||||
contextSnapshot: {
|
||||
issueId: payload.issueId,
|
||||
taskId: payload.issueId,
|
||||
interactionId: payload.interactionId,
|
||||
interactionKind: payload.interactionKind,
|
||||
interactionStatus: payload.interactionStatus,
|
||||
sourceCommentId,
|
||||
sourceRunId,
|
||||
...(payload.planReviewInteraction &&
|
||||
typeof payload.planReviewInteraction === "object"
|
||||
? { planReviewInteraction: payload.planReviewInteraction }
|
||||
: {}),
|
||||
wakeReason: "issue_commented",
|
||||
source: chatOrigin
|
||||
? `chat:${chatOrigin.provider}`
|
||||
: "external_chat.interaction.resolve",
|
||||
...(wakeCommentId
|
||||
? {
|
||||
wakeCommentId,
|
||||
wakeCommentIds: [wakeCommentId],
|
||||
}
|
||||
: {}),
|
||||
...(payload.forceFreshSession === true
|
||||
? { forceFreshSession: true }
|
||||
: {}),
|
||||
...(typeof payload.workspaceRefreshReason === "string"
|
||||
? {
|
||||
workspaceRefreshReason: payload.workspaceRefreshReason,
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
})
|
||||
: null;
|
||||
const durable =
|
||||
existing ??
|
||||
(await db
|
||||
.select({
|
||||
agentId: agentWakeupRequests.agentId,
|
||||
id: agentWakeupRequests.id,
|
||||
})
|
||||
.from(agentWakeupRequests)
|
||||
.where(
|
||||
and(
|
||||
eq(agentWakeupRequests.companyId, claimed.companyId),
|
||||
eq(agentWakeupRequests.idempotencyKey, idempotencyKey),
|
||||
notInArray(agentWakeupRequests.status, [
|
||||
"skipped",
|
||||
"failed",
|
||||
"cancelled",
|
||||
]),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null));
|
||||
if (!queuedRun && !durable) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "issued",
|
||||
result: { code: "interaction_wakeup_deferred" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, claimed.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "processed",
|
||||
result: {
|
||||
code: durable
|
||||
? durable.agentId === payload.agentId
|
||||
? "interaction_wakeup_already_durable"
|
||||
: "interaction_wakeup_coalesced_after_reassignment"
|
||||
: "interaction_wakeup_queued",
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, claimed.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
processed += 1;
|
||||
} catch (error) {
|
||||
const durable = isUniqueViolation(error)
|
||||
? await db
|
||||
.select({
|
||||
agentId: agentWakeupRequests.agentId,
|
||||
id: agentWakeupRequests.id,
|
||||
})
|
||||
.from(agentWakeupRequests)
|
||||
.where(
|
||||
and(
|
||||
eq(agentWakeupRequests.companyId, claimed.companyId),
|
||||
eq(agentWakeupRequests.idempotencyKey, idempotencyKey),
|
||||
notInArray(agentWakeupRequests.status, [
|
||||
"skipped",
|
||||
"failed",
|
||||
"cancelled",
|
||||
]),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const previousResult =
|
||||
claimed.result && typeof claimed.result === "object"
|
||||
? (claimed.result as Record<string, unknown>)
|
||||
: {};
|
||||
const previousAttempts =
|
||||
typeof previousResult.attemptCount === "number" &&
|
||||
Number.isInteger(previousResult.attemptCount) &&
|
||||
previousResult.attemptCount >= 0
|
||||
? previousResult.attemptCount
|
||||
: 0;
|
||||
const attemptCount = previousAttempts + 1;
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: durable ? "processed" : "issued",
|
||||
result: {
|
||||
code: durable
|
||||
? durable.agentId === payload.agentId
|
||||
? "interaction_wakeup_already_durable"
|
||||
: "interaction_wakeup_coalesced_after_reassignment"
|
||||
: "interaction_wakeup_failed",
|
||||
...(!durable ? { attemptCount } : {}),
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, claimed.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
if (!durable) {
|
||||
logger.warn(
|
||||
{
|
||||
endpointId: claimed.endpointId,
|
||||
interactionId: payload.interactionId,
|
||||
attemptCount,
|
||||
error: redactError(error),
|
||||
},
|
||||
"failed to queue external interaction continuation wake",
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
return processed;
|
||||
}
|
||||
return { processPendingInteractionWakeups };
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
import type { Message, Thread } from "chat";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { chatEndpointResources } from "@paperclipai/db";
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js";
|
||||
import { nonDirectDestinationAllowed, teamsThreadRootMessageId } from "../inbound/surfaces.js";
|
||||
|
||||
type DestinationAdmissionInput = {
|
||||
endpoint: EndpointRow;
|
||||
currentEndpoint: Pick<EndpointRow, "status" | "allowGroupChats">;
|
||||
thread: Thread;
|
||||
message: Message;
|
||||
addressed: boolean;
|
||||
};
|
||||
type DestinationAdmissionOptions = {
|
||||
ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise<typeof chatEndpointResources.$inferSelect>;
|
||||
};
|
||||
export function createTeamsDestinationAdmission(options: DestinationAdmissionOptions) {
|
||||
const { ensureResource } = options;
|
||||
async function admitTeamsDestination(input: DestinationAdmissionInput, tx: DbTransaction) {
|
||||
const { endpoint, currentEndpoint, thread, message, addressed } = input;
|
||||
let destinationAccepting = true;
|
||||
let provisionalTeamsSetupReply = false;
|
||||
|
||||
let resource = await ensureResource(endpoint, thread, false, tx);
|
||||
const enabledChannelCount = await tx
|
||||
.select({ count: sql<number>`count(*)::int` })
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.enabled, true),
|
||||
eq(chatEndpointResources.availability, "available"),
|
||||
eq(chatEndpointResources.type, "channel"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]?.count ?? 0);
|
||||
const canActivateFirstSetupChannel =
|
||||
currentEndpoint.status === "verifying" &&
|
||||
resource.type === "channel" &&
|
||||
resource.availability === "available" &&
|
||||
enabledChannelCount === 0 &&
|
||||
addressed;
|
||||
if (canActivateFirstSetupChannel && !resource.enabled) {
|
||||
resource = await tx
|
||||
.update(chatEndpointResources)
|
||||
.set({ enabled: true, updatedAt: new Date() })
|
||||
.where(eq(chatEndpointResources.id, resource.id))
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? resource);
|
||||
}
|
||||
const teamsRootMessageId = teamsThreadRootMessageId(thread.id);
|
||||
provisionalTeamsSetupReply =
|
||||
currentEndpoint.status === "verifying" &&
|
||||
resource.type === "channel" &&
|
||||
resource.availability === "available" &&
|
||||
enabledChannelCount === 0 &&
|
||||
!addressed &&
|
||||
Boolean(teamsRootMessageId) &&
|
||||
teamsRootMessageId !== message.id;
|
||||
destinationAccepting =
|
||||
nonDirectDestinationAllowed(
|
||||
{ ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats },
|
||||
resource,
|
||||
) || provisionalTeamsSetupReply;
|
||||
return { destinationAccepting, provisionalTeamsSetupReply };
|
||||
}
|
||||
return { admitTeamsDestination };
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import type { EndpointRow, DbTransaction, LifecycleRuntimeFence, InboundRuntimeContext } from "../runtime-types.js";
|
||||
|
||||
import { type ChatRuntime } from "../../chat-runtime.js";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
type ChatSdkRuntime,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
|
||||
type TeamsRoutesOptions = {
|
||||
runtime: Pick<ChatRuntime | ChatSdkRuntime, "get">;
|
||||
runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise<EndpointRow | null>;
|
||||
};
|
||||
|
||||
|
||||
|
||||
function microsoftTeamsServiceUrl(raw: unknown): string | null {
|
||||
if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
|
||||
const value = (raw as { serviceUrl?: unknown }).serviceUrl;
|
||||
return typeof value === "string" && value ? value : null;
|
||||
}
|
||||
export function createTeamsRoutes(db: Db, options: TeamsRoutesOptions) {
|
||||
const { runtime, runtimeCallbackEndpoint } = options;
|
||||
|
||||
|
||||
async function recordCurrentMicrosoftTeamsRoute(
|
||||
endpoint: EndpointRow,
|
||||
context: InboundRuntimeContext | undefined,
|
||||
threadId: string,
|
||||
raw: unknown,
|
||||
recordAcceptedActivity = true,
|
||||
): Promise<void> {
|
||||
if (endpoint.provider !== "microsoft-teams" || !context?.endpointRuntime) {
|
||||
return;
|
||||
}
|
||||
const serviceUrl = microsoftTeamsServiceUrl(raw);
|
||||
if (!serviceUrl) return;
|
||||
await db.transaction(async (tx) => {
|
||||
const current = await runtimeCallbackEndpoint(tx, endpoint.id, context, [
|
||||
"verifying",
|
||||
"active",
|
||||
]);
|
||||
if (!current || runtime.get(endpoint.id) !== context.endpointRuntime) {
|
||||
return;
|
||||
}
|
||||
await context.endpointRuntime.recordMicrosoftTeamsRoute(
|
||||
threadId,
|
||||
serviceUrl,
|
||||
recordAcceptedActivity ? raw : undefined,
|
||||
);
|
||||
});
|
||||
}
|
||||
return { recordCurrentMicrosoftTeamsRoute };
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
import type {
|
||||
ChatAdapterCapabilities,
|
||||
ChatProvider,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
const PROVIDER_LABELS: Record<ChatProvider, string> = {
|
||||
speko: "Speko",
|
||||
"imessage-photon": "iMessage Photon",
|
||||
agentmail: "AgentMail",
|
||||
slack: "Slack",
|
||||
github: "GitHub",
|
||||
discord: "Discord",
|
||||
"microsoft-teams": "Microsoft Teams",
|
||||
telegram: "Telegram",
|
||||
};
|
||||
|
||||
const CAPABILITIES: Record<ChatProvider, ChatAdapterCapabilities> = {
|
||||
speko: { threads: false, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: false, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false },
|
||||
"imessage-photon": { threads: false, directMessages: true, nativeStreaming: false, messageEdits: true, messageDeletes: false, reactions: false, files: true, cards: false, actions: true, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: false },
|
||||
agentmail: { threads: true, directMessages: true, nativeStreaming: false, messageEdits: false, messageDeletes: false, reactions: false, files: true, cards: false, actions: false, modals: false, slashCommands: false, ephemeralMessages: false, proactiveDirectMessages: true },
|
||||
slack: {
|
||||
threads: true,
|
||||
directMessages: true,
|
||||
nativeStreaming: true,
|
||||
messageEdits: true,
|
||||
messageDeletes: true,
|
||||
reactions: true,
|
||||
files: true,
|
||||
cards: true,
|
||||
actions: true,
|
||||
modals: true,
|
||||
slashCommands: true,
|
||||
ephemeralMessages: true,
|
||||
proactiveDirectMessages: false,
|
||||
},
|
||||
github: {
|
||||
threads: true,
|
||||
directMessages: false,
|
||||
nativeStreaming: false,
|
||||
messageEdits: true,
|
||||
messageDeletes: true,
|
||||
reactions: true,
|
||||
files: false,
|
||||
cards: true,
|
||||
actions: false,
|
||||
modals: false,
|
||||
slashCommands: false,
|
||||
ephemeralMessages: false,
|
||||
proactiveDirectMessages: false,
|
||||
},
|
||||
discord: {
|
||||
threads: true,
|
||||
directMessages: true,
|
||||
nativeStreaming: false,
|
||||
messageEdits: true,
|
||||
messageDeletes: true,
|
||||
reactions: true,
|
||||
files: true,
|
||||
cards: true,
|
||||
actions: true,
|
||||
modals: true,
|
||||
// Automatic registration upgrades these only after a durable provider
|
||||
// receipt. A draft or an uncertain/conflicting registration stays false.
|
||||
slashCommands: false,
|
||||
ephemeralMessages: false,
|
||||
proactiveDirectMessages: false,
|
||||
},
|
||||
"microsoft-teams": {
|
||||
threads: true,
|
||||
directMessages: true,
|
||||
// Production webhook processing is deferred into Paperclip's durable
|
||||
// ingress queue. The Teams adapter's request-scoped DM streamer is gone by
|
||||
// the time agent output is published, so advertise the durable behavior we
|
||||
// can actually provide. editMessage still lets one run coalesce its
|
||||
// queued, working, and final states in place.
|
||||
nativeStreaming: false,
|
||||
// Paperclip supplements the pinned adapter's missing inbound Bot Framework
|
||||
// messageUpdate/messageDelete dispatch after provider authentication.
|
||||
messageEdits: true,
|
||||
messageDeletes: true,
|
||||
reactions: true,
|
||||
files: true,
|
||||
cards: true,
|
||||
actions: true,
|
||||
modals: true,
|
||||
slashCommands: false,
|
||||
ephemeralMessages: true,
|
||||
proactiveDirectMessages: false,
|
||||
},
|
||||
telegram: {
|
||||
threads: true,
|
||||
directMessages: true,
|
||||
nativeStreaming: true,
|
||||
messageEdits: true,
|
||||
// Telegram's Bot API does not emit an update when a user deletes a
|
||||
// message, so this capability cannot be offered truthfully.
|
||||
messageDeletes: false,
|
||||
reactions: true,
|
||||
files: true,
|
||||
cards: true,
|
||||
actions: true,
|
||||
modals: false,
|
||||
slashCommands: false,
|
||||
ephemeralMessages: false,
|
||||
proactiveDirectMessages: false,
|
||||
},
|
||||
};
|
||||
export { PROVIDER_LABELS, CAPABILITIES };
|
||||
@@ -0,0 +1,344 @@
|
||||
import { createGitHubPublicationPolicy } from "../github/publication-policy.js";
|
||||
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction, LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
import { createChatFailedRunRetry } from "../failed-run-retry.js";
|
||||
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import { authorizeSlackBoardPublication } from "../../slack-board-authority.js";
|
||||
|
||||
import { slackPublicationAllowed } from "../../connectors/slack-access.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
gt,
|
||||
inArray,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatActions,
|
||||
chatConversations,
|
||||
chatEndpointResources,
|
||||
chatPublications,
|
||||
heartbeatRuns,
|
||||
issueComments,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
authorizeNativeChatReviewPresentation,
|
||||
NativeChatReviewPresentationContentionError,
|
||||
} from "../../native-runtime/native-chat-review-presentation.js";
|
||||
import { isExternalChatWaitAuthorizationContention } from "../../native-runtime/chat-attachment-reuse.js";
|
||||
|
||||
import { isExplicitOperatorPublication } from "./policy.js";
|
||||
import { attachmentFailureNoticeFence } from "./policy.js";
|
||||
import type { createChatPublicationTaskControl } from "./task-control.js";
|
||||
type ConversationRow = typeof chatConversations.$inferSelect;
|
||||
type PublicationAuthorizationOptions = {
|
||||
authorizeInboundWakePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise<boolean>;
|
||||
authorizeRetainedChatSourcePublication: (tx: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise<boolean>;
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
githubCurrentOmissionPreparedText: ReturnType<typeof createGitHubPublicationPolicy>["githubCurrentOmissionPreparedText"];
|
||||
hasCommittedTaskControlCompletion: ReturnType<typeof createChatPublicationTaskControl>["hasCommittedTaskControlCompletion"];
|
||||
lockCurrentPrincipalAuthorization: (tx: DbOrTransaction, endpoint: EndpointRow, principalId: string) => Promise<{ allowed: boolean; userId?: string | null }>;
|
||||
nonDirectDestinationAllowed: (endpoint: EndpointRow, resource: typeof chatEndpointResources.$inferSelect | null | undefined) => boolean;
|
||||
resolveCredentials: (endpoint: EndpointRow, tx?: DbOrTransaction) => Promise<Record<string, string>>;
|
||||
runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise<EndpointRow | null>;
|
||||
FailedChatRetryPublicationReadError: ReturnType<typeof createChatFailedRunRetry>["FailedChatRetryPublicationReadError"];
|
||||
};
|
||||
export function createChatPublicationAuthorization(db: Db, options: PublicationAuthorizationOptions) {
|
||||
const { authorizeInboundWakePublication, authorizeRetainedChatSourcePublication, fetchImpl, githubCurrentOmissionPreparedText, hasCommittedTaskControlCompletion, lockCurrentPrincipalAuthorization, nonDirectDestinationAllowed, resolveCredentials, runtimeCallbackEndpoint, FailedChatRetryPublicationReadError } = options;
|
||||
|
||||
|
||||
async function claimPublicationTransportAuthorization(input: {
|
||||
credentialLease: CredentialMutationLeaseGuard;
|
||||
endpoint: EndpointRow;
|
||||
publication: typeof chatPublications.$inferSelect;
|
||||
runtimeContext: LifecycleRuntimeFence;
|
||||
}): Promise<{
|
||||
authorizationActionId: string | null;
|
||||
conversation: ConversationRow;
|
||||
endpoint: EndpointRow;
|
||||
preparedText: string | null;
|
||||
} | null> {
|
||||
return db.transaction(async (tx) => {
|
||||
await input.credentialLease.assertOwned(tx);
|
||||
const currentPublication = await tx
|
||||
.select({ id: chatPublications.id })
|
||||
.from(chatPublications)
|
||||
.where(
|
||||
and(
|
||||
eq(chatPublications.id, input.publication.id),
|
||||
eq(chatPublications.state, "streaming"),
|
||||
eq(chatPublications.attempts, input.publication.attempts + 1),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!currentPublication) return null;
|
||||
try {
|
||||
if (
|
||||
!(await authorizeRetainedChatSourcePublication(tx, input.publication))
|
||||
)
|
||||
return null;
|
||||
} catch (error) {
|
||||
if (error instanceof NativeChatReviewPresentationContentionError)
|
||||
throw error;
|
||||
throw new FailedChatRetryPublicationReadError();
|
||||
}
|
||||
const attachmentFailureFence = attachmentFailureNoticeFence(
|
||||
input.publication.idempotencyKey,
|
||||
);
|
||||
if (
|
||||
attachmentFailureFence &&
|
||||
(attachmentFailureFence.generation !==
|
||||
input.runtimeContext.generation ||
|
||||
attachmentFailureFence.credentialFingerprint !==
|
||||
input.runtimeContext.credentialFingerprint)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const endpoint = await runtimeCallbackEndpoint(
|
||||
tx,
|
||||
input.publication.endpointId,
|
||||
input.runtimeContext,
|
||||
["verifying", "active"],
|
||||
);
|
||||
const conversation = endpoint
|
||||
? await tx
|
||||
.select()
|
||||
.from(chatConversations)
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, endpoint.companyId),
|
||||
eq(chatConversations.endpointId, endpoint.id),
|
||||
eq(chatConversations.id, input.publication.conversationId),
|
||||
inArray(chatConversations.state, [
|
||||
"active",
|
||||
"waiting",
|
||||
"completed",
|
||||
]),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const resource =
|
||||
endpoint && conversation?.resourceId && !conversation.isDirectMessage
|
||||
? await tx
|
||||
.select()
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.companyId, endpoint.companyId),
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.id, conversation.resourceId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const attachmentFailureNoticeSuperseded =
|
||||
attachmentFailureFence && conversation
|
||||
? await tx
|
||||
.select({ id: chatConversations.id })
|
||||
.from(chatConversations)
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, conversation.companyId),
|
||||
eq(chatConversations.endpointId, conversation.endpointId),
|
||||
eq(
|
||||
chatConversations.externalThreadId,
|
||||
conversation.externalThreadId,
|
||||
),
|
||||
gt(
|
||||
chatConversations.sessionGeneration,
|
||||
conversation.sessionGeneration,
|
||||
),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows.length > 0)
|
||||
: false;
|
||||
const isTaskControl =
|
||||
input.publication.idempotencyKey.startsWith("control:");
|
||||
const providerVisibleCompletion =
|
||||
conversation?.state === "completed" &&
|
||||
!isExplicitOperatorPublication(input.publication) &&
|
||||
(await hasCommittedTaskControlCompletion(conversation.id, tx));
|
||||
const authorizationAction =
|
||||
endpoint && isTaskControl
|
||||
? await tx
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, endpoint.companyId),
|
||||
eq(chatActions.endpointId, endpoint.id),
|
||||
eq(chatActions.conversationId, conversation?.id ?? ""),
|
||||
eq(chatActions.kind, "task_control_authorization"),
|
||||
eq(
|
||||
chatActions.providerActionId,
|
||||
`task-control-authorization:${input.publication.id}`,
|
||||
),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const destinationAllowed =
|
||||
endpoint !== null &&
|
||||
conversation !== null &&
|
||||
(endpoint.provider !== "slack" || await slackPublicationAllowed(tx, endpoint.companyId, endpoint.id, input.publication.issueId, conversation.externalConversationId, null)) &&
|
||||
(conversation.isDirectMessage
|
||||
? endpoint.allowDirectMessages
|
||||
: nonDirectDestinationAllowed(endpoint, resource));
|
||||
if (
|
||||
!endpoint ||
|
||||
!conversation ||
|
||||
(attachmentFailureFence &&
|
||||
((conversation.state !== "active" &&
|
||||
conversation.state !== "waiting") ||
|
||||
attachmentFailureNoticeSuperseded)) ||
|
||||
providerVisibleCompletion ||
|
||||
(isTaskControl && !authorizationAction?.principalId) ||
|
||||
!destinationAllowed
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
let authorizationActionId: string | null = null;
|
||||
if (endpoint.provider === "slack") {
|
||||
const credentials = await resolveCredentials(endpoint, tx);
|
||||
if (!await authorizeSlackBoardPublication(tx, endpoint, conversation, input.publication, credentials.botToken ?? "", fetchImpl)) return null;
|
||||
}
|
||||
if (
|
||||
input.publication.idempotencyKey.startsWith("wake:") &&
|
||||
!(await authorizeInboundWakePublication(tx, input.publication))
|
||||
)
|
||||
return null;
|
||||
// A native response alongside an older completion review has its own
|
||||
// narrow authorization. Recheck that exact proof, review and original
|
||||
// requester at the transport claim, including file publications whose
|
||||
// comment is the run's earlier internal attachment-selection comment.
|
||||
if (
|
||||
input.publication.commentId &&
|
||||
!isExplicitOperatorPublication(input.publication)
|
||||
) {
|
||||
const [origin] = await tx
|
||||
.select({
|
||||
runId: heartbeatRuns.id,
|
||||
resultJson: heartbeatRuns.resultJson,
|
||||
})
|
||||
.from(issueComments)
|
||||
.innerJoin(
|
||||
heartbeatRuns,
|
||||
and(
|
||||
eq(heartbeatRuns.id, issueComments.createdByRunId),
|
||||
eq(heartbeatRuns.companyId, issueComments.companyId),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(issueComments.id, input.publication.commentId),
|
||||
eq(issueComments.companyId, endpoint.companyId),
|
||||
eq(issueComments.issueId, input.publication.issueId),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (
|
||||
origin?.resultJson?.finalizationReasonCode ===
|
||||
"governed_response_waiting"
|
||||
) {
|
||||
try {
|
||||
if (
|
||||
!(await authorizeNativeChatReviewPresentation(
|
||||
tx as unknown as Db,
|
||||
{
|
||||
companyId: endpoint.companyId,
|
||||
issueId: input.publication.issueId,
|
||||
runId: origin.runId,
|
||||
resultJson: origin.resultJson,
|
||||
destination: {
|
||||
endpointId: endpoint.id,
|
||||
conversationId: conversation.id,
|
||||
},
|
||||
},
|
||||
"nonblocking",
|
||||
))
|
||||
)
|
||||
return null;
|
||||
} catch (error) {
|
||||
if (isExternalChatWaitAuthorizationContention(error))
|
||||
throw new NativeChatReviewPresentationContentionError();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
const preparedText = await githubCurrentOmissionPreparedText(
|
||||
tx,
|
||||
input.publication,
|
||||
endpoint,
|
||||
);
|
||||
if (preparedText === false) return null;
|
||||
if (authorizationAction?.principalId) {
|
||||
const authorization = await lockCurrentPrincipalAuthorization(
|
||||
tx,
|
||||
endpoint,
|
||||
authorizationAction.principalId,
|
||||
);
|
||||
if (!authorization.allowed) return null;
|
||||
const [claimedAuthorization] = await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "processing",
|
||||
result: {
|
||||
attempts: input.publication.attempts + 1,
|
||||
authorizedUserId: authorization.userId,
|
||||
credentialFingerprint: input.runtimeContext.credentialFingerprint,
|
||||
runtimeGeneration: input.runtimeContext.generation,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, authorizationAction.id),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatActions.id });
|
||||
if (!claimedAuthorization) return null;
|
||||
authorizationActionId = claimedAuthorization.id;
|
||||
}
|
||||
if (preparedText !== null) {
|
||||
// Persist the exact provider text before I/O. A retry must reuse this
|
||||
// body, not accumulate links or regenerate it from changed settings.
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({
|
||||
payload: { ...input.publication.payload, text: preparedText },
|
||||
})
|
||||
.where(eq(chatPublications.id, input.publication.id));
|
||||
}
|
||||
await input.credentialLease.assertOwned(tx);
|
||||
// The publication's streaming row plus the optional authorization action
|
||||
// are the durable transport claim. Their database locks are released
|
||||
// before provider I/O. The renewable credential lease intentionally
|
||||
// remains held so reconnect and reach mutations linearize after this
|
||||
// committed authorization snapshot and transport attempt.
|
||||
return {
|
||||
authorizationActionId,
|
||||
conversation,
|
||||
endpoint,
|
||||
preparedText,
|
||||
};
|
||||
});
|
||||
}
|
||||
return { claimPublicationTransportAuthorization };
|
||||
}
|
||||
@@ -0,0 +1,328 @@
|
||||
import { isOutboundAttachmentHydrationError, isOutboundAttachmentValidationError } from "../outbound-attachments.js";
|
||||
|
||||
import type { EndpointRow, DbTransaction, LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
import { createChatFailedRunRetry } from "../failed-run-retry.js";
|
||||
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
inArray,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatActions,
|
||||
chatConversations,
|
||||
chatEndpointResources,
|
||||
chatEndpoints,
|
||||
chatPublications,
|
||||
toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import { logger } from "../../../middleware/logger.js";
|
||||
import { redactSensitiveText } from "../../../redaction.js";
|
||||
|
||||
import {
|
||||
NativeChatReviewPresentationContentionError,
|
||||
} from "../../native-runtime/native-chat-review-presentation.js";
|
||||
|
||||
import { projectSafeChatPublication } from "../../chat-publication-projection.js";
|
||||
import { safeChatTaskUrl } from "../../chat-task-url.js";
|
||||
|
||||
import { classifyChatPublicationError } from "../../chat-publication-errors.js";
|
||||
|
||||
import { ATTACHMENT_FAILURE_NOTICE_PREFIX, definiteAttachmentProviderRejection, attachmentFailureKind } from "./policy.js";
|
||||
type PublicationFailuresOptions = {
|
||||
FailedChatRetryPublicationReadError: ReturnType<typeof createChatFailedRunRetry>["FailedChatRetryPublicationReadError"];
|
||||
MAX_ERROR_TEXT: number;
|
||||
getTaskBaseUrl: () => string | null | undefined;
|
||||
invalidateRuntime: (endpointId: string) => Promise<boolean>;
|
||||
runtimeCallbackEndpoint: (tx: DbTransaction, endpointId: string, context: LifecycleRuntimeFence, statuses: EndpointRow["status"][]) => Promise<EndpointRow | null>;
|
||||
};
|
||||
export function createChatPublicationFailures(db: Db, options: PublicationFailuresOptions) {
|
||||
const { FailedChatRetryPublicationReadError, MAX_ERROR_TEXT, getTaskBaseUrl, invalidateRuntime, runtimeCallbackEndpoint } = options;
|
||||
|
||||
|
||||
async function finalizePublicationFailure(
|
||||
publication: typeof chatPublications.$inferSelect,
|
||||
error: unknown,
|
||||
providerAccepted = false,
|
||||
runtimeContext?: LifecycleRuntimeFence | null,
|
||||
credentialLease?: CredentialMutationLeaseGuard,
|
||||
): Promise<boolean> {
|
||||
const attempts = publication.attempts + 1;
|
||||
const disposition = providerAccepted
|
||||
? {
|
||||
kind: "delivery_unknown" as const,
|
||||
reason:
|
||||
"Provider accepted the publication, but Paperclip could not confirm its durable result",
|
||||
}
|
||||
: error instanceof NativeChatReviewPresentationContentionError
|
||||
? { kind: "retry" as const, retryAfterMs: 250, reason: error.message }
|
||||
: error instanceof FailedChatRetryPublicationReadError
|
||||
? {
|
||||
kind: "retry" as const,
|
||||
retryAfterMs: 1_000,
|
||||
reason: error.message,
|
||||
}
|
||||
: isOutboundAttachmentValidationError(error)
|
||||
? {
|
||||
kind: "failed" as const,
|
||||
reason: error.message,
|
||||
}
|
||||
: isOutboundAttachmentHydrationError(error)
|
||||
? {
|
||||
kind: "retry" as const,
|
||||
retryAfterMs: Math.min(
|
||||
60_000,
|
||||
2 ** Math.max(0, attempts) * 1_000,
|
||||
),
|
||||
reason: error.message,
|
||||
}
|
||||
: classifyChatPublicationError(error, attempts);
|
||||
const failure = redactSensitiveText(disposition.reason).slice(
|
||||
0,
|
||||
MAX_ERROR_TEXT,
|
||||
);
|
||||
// Provider-directed rate limits are not delivery failures. Keep the
|
||||
// durable publication retryable for the provider's full interval instead
|
||||
// of exhausting the five-attempt transport budget while flood control is
|
||||
// still active.
|
||||
const terminalRetry =
|
||||
disposition.kind === "retry" &&
|
||||
disposition.providerRateLimit !== true &&
|
||||
attempts >= 5;
|
||||
const attachmentFailure = attachmentFailureKind(publication.payload);
|
||||
const attachmentFailureRuntimeContext =
|
||||
!providerAccepted &&
|
||||
disposition.kind === "failed" &&
|
||||
runtimeContext &&
|
||||
attachmentFailure &&
|
||||
definiteAttachmentProviderRejection(error)
|
||||
? runtimeContext
|
||||
: null;
|
||||
let invalidateCurrentRuntime = false;
|
||||
const finalized = await db.transaction(async (tx) => {
|
||||
await credentialLease?.assertOwned(tx);
|
||||
const state =
|
||||
disposition.kind === "retry" && !terminalRetry
|
||||
? "retry"
|
||||
: disposition.kind === "delivery_unknown"
|
||||
? "delivery_unknown"
|
||||
: disposition.kind === "resource_unavailable"
|
||||
? "cancelled"
|
||||
: "failed";
|
||||
await notifyChatPublicationWork(tx);
|
||||
const [ownedPublication] = await tx
|
||||
.update(chatPublications)
|
||||
.set({
|
||||
state,
|
||||
nextAttemptAt:
|
||||
disposition.kind === "retry" && !terminalRetry
|
||||
? new Date(Date.now() + disposition.retryAfterMs)
|
||||
: null,
|
||||
redactedError: failure,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatPublications.id, publication.id),
|
||||
eq(chatPublications.state, "streaming"),
|
||||
eq(chatPublications.attempts, attempts),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatPublications.id });
|
||||
if (!ownedPublication) return false;
|
||||
|
||||
if (publication.idempotencyKey.startsWith("control:")) {
|
||||
await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status:
|
||||
state === "delivery_unknown"
|
||||
? "delivery_unknown"
|
||||
: state === "cancelled"
|
||||
? "cancelled"
|
||||
: "issued",
|
||||
result: {
|
||||
code:
|
||||
state === "delivery_unknown"
|
||||
? "task_control_delivery_unknown"
|
||||
: state === "cancelled"
|
||||
? "task_control_destination_unavailable"
|
||||
: "task_control_transport_failed",
|
||||
attempts,
|
||||
retryable: state === "retry",
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.endpointId, publication.endpointId),
|
||||
eq(
|
||||
chatActions.providerActionId,
|
||||
`task-control-authorization:${publication.id}`,
|
||||
),
|
||||
eq(chatActions.status, "processing"),
|
||||
sql`(${chatActions.result}->>'attempts')::int = ${attempts}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
const currentEndpoint =
|
||||
runtimeContext &&
|
||||
(disposition.kind === "endpoint_attention" ||
|
||||
disposition.kind === "resource_unavailable")
|
||||
? await runtimeCallbackEndpoint(
|
||||
tx,
|
||||
publication.endpointId,
|
||||
runtimeContext,
|
||||
["verifying", "active"],
|
||||
)
|
||||
: null;
|
||||
|
||||
if (disposition.kind === "endpoint_attention" && currentEndpoint) {
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
status: "attention",
|
||||
healthMessage: "Provider credentials or permissions need attention",
|
||||
lastError: failure,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatEndpoints.id, currentEndpoint.id));
|
||||
await tx
|
||||
.update(toolConnections)
|
||||
.set({
|
||||
status: "disabled",
|
||||
enabled: false,
|
||||
healthStatus: "degraded",
|
||||
healthMessage: "Provider credentials or permissions need attention",
|
||||
lastError: failure,
|
||||
healthCheckedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(toolConnections.id, currentEndpoint.connectionId));
|
||||
invalidateCurrentRuntime = true;
|
||||
} else if (
|
||||
disposition.kind === "resource_unavailable" &&
|
||||
currentEndpoint
|
||||
) {
|
||||
const binding = await tx
|
||||
.select({ resourceId: chatConversations.resourceId })
|
||||
.from(chatConversations)
|
||||
.where(eq(chatConversations.id, publication.conversationId))
|
||||
.then((result) => result[0] ?? null);
|
||||
await tx
|
||||
.update(chatConversations)
|
||||
.set({ state: "unavailable", updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.id, publication.conversationId),
|
||||
eq(chatConversations.endpointId, currentEndpoint.id),
|
||||
),
|
||||
);
|
||||
if (binding?.resourceId) {
|
||||
await tx
|
||||
.update(chatEndpointResources)
|
||||
.set({ availability: "unavailable", updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.id, binding.resourceId),
|
||||
eq(chatEndpointResources.endpointId, currentEndpoint.id),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (attachmentFailureRuntimeContext && attachmentFailure) {
|
||||
const noticeEndpoint = await runtimeCallbackEndpoint(
|
||||
tx,
|
||||
publication.endpointId,
|
||||
attachmentFailureRuntimeContext,
|
||||
["verifying", "active"],
|
||||
);
|
||||
const noticeConversation = noticeEndpoint
|
||||
? await tx
|
||||
.select({
|
||||
id: chatConversations.id,
|
||||
state: chatConversations.state,
|
||||
})
|
||||
.from(chatConversations)
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, publication.companyId),
|
||||
eq(chatConversations.endpointId, publication.endpointId),
|
||||
eq(chatConversations.id, publication.conversationId),
|
||||
inArray(chatConversations.state, ["active", "waiting"]),
|
||||
),
|
||||
)
|
||||
.for("no key update")
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
const providerCanSendTextNotice =
|
||||
noticeEndpoint &&
|
||||
["slack", "discord", "telegram"].includes(noticeEndpoint.provider) &&
|
||||
(attachmentFailure.provider === null ||
|
||||
noticeEndpoint.provider === attachmentFailure.provider);
|
||||
if (providerCanSendTextNotice && noticeConversation) {
|
||||
const taskUrl = safeChatTaskUrl(
|
||||
getTaskBaseUrl(),
|
||||
publication.issueId,
|
||||
);
|
||||
const noticeText =
|
||||
attachmentFailure.kind === "generated_response"
|
||||
? "Paperclip could not send the response attachment. The complete response remains on its Paperclip task for an operator to retry."
|
||||
: "Paperclip could not send an attachment. The file remains on its Paperclip task for an operator to retry.";
|
||||
const idempotencyKey = `${ATTACHMENT_FAILURE_NOTICE_PREFIX}${publication.id}:${attachmentFailureRuntimeContext.generation}:${attachmentFailureRuntimeContext.credentialFingerprint}`;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.insert(chatPublications)
|
||||
.values({
|
||||
companyId: publication.companyId,
|
||||
endpointId: publication.endpointId,
|
||||
conversationId: publication.conversationId,
|
||||
issueId: publication.issueId,
|
||||
commentId: null,
|
||||
idempotencyKey,
|
||||
payload: projectSafeChatPublication({
|
||||
classification: "external",
|
||||
source: "safe_milestone",
|
||||
text: taskUrl
|
||||
? `${noticeText} Open task: ${taskUrl}`
|
||||
: noticeText,
|
||||
}),
|
||||
state: "pending",
|
||||
})
|
||||
.onConflictDoNothing();
|
||||
}
|
||||
}
|
||||
|
||||
await credentialLease?.assertOwned(tx);
|
||||
return true;
|
||||
});
|
||||
if (finalized && invalidateCurrentRuntime) {
|
||||
await credentialLease?.assertOwned();
|
||||
await invalidateRuntime(publication.endpointId).catch(() => undefined);
|
||||
}
|
||||
if (finalized) {
|
||||
logger.warn(
|
||||
{
|
||||
endpointId: publication.endpointId,
|
||||
publicationId: publication.id,
|
||||
error: failure,
|
||||
disposition: disposition.kind,
|
||||
},
|
||||
"chat publication failed",
|
||||
);
|
||||
}
|
||||
return finalized;
|
||||
}
|
||||
return { finalizePublicationFailure };
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
lt,
|
||||
or,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
import { type AnyPgColumn } from "drizzle-orm/pg-core";
|
||||
|
||||
type PublicationOrderTable = {
|
||||
createdAt: AnyPgColumn;
|
||||
id: AnyPgColumn;
|
||||
payload: AnyPgColumn;
|
||||
};
|
||||
|
||||
function publicationTransportOrderKey(table: PublicationOrderTable) {
|
||||
return sql<string>`coalesce(${table.payload}->'transportPart'->>'orderKey', ${table.id}::text)`;
|
||||
}
|
||||
|
||||
function publicationPrecedes(
|
||||
earlier: PublicationOrderTable,
|
||||
later: PublicationOrderTable,
|
||||
) {
|
||||
return or(
|
||||
lt(earlier.createdAt, later.createdAt),
|
||||
and(
|
||||
eq(earlier.createdAt, later.createdAt),
|
||||
lt(
|
||||
publicationTransportOrderKey(earlier),
|
||||
publicationTransportOrderKey(later),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export { publicationTransportOrderKey, publicationPrecedes };
|
||||
@@ -0,0 +1,132 @@
|
||||
import type { LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
|
||||
import {
|
||||
chatPublications,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
function isExplicitOperatorPublication(
|
||||
publication: Pick<typeof chatPublications.$inferSelect, "idempotencyKey">,
|
||||
): boolean {
|
||||
return /^(?:explicit:|explicit-board:)/.test(publication.idempotencyKey);
|
||||
}
|
||||
|
||||
const ATTACHMENT_FAILURE_NOTICE_PREFIX = "attachment-failure-notice:";
|
||||
|
||||
function attachmentFailureNoticeFence(
|
||||
idempotencyKey: string,
|
||||
): LifecycleRuntimeFence | null {
|
||||
const match =
|
||||
/^attachment-failure-notice:[0-9a-f-]{36}:(\d+):([0-9a-f]{64})$/i.exec(
|
||||
idempotencyKey,
|
||||
);
|
||||
if (!match) return null;
|
||||
const generation = Number(match[1]);
|
||||
if (!Number.isSafeInteger(generation) || generation < 0) return null;
|
||||
return { generation, credentialFingerprint: match[2]!.toLowerCase() };
|
||||
}
|
||||
|
||||
function definiteAttachmentProviderRejection(error: unknown): boolean {
|
||||
const pending: Array<{ depth: number; value: unknown }> = [
|
||||
{ depth: 0, value: error },
|
||||
];
|
||||
const seen = new Set<object>();
|
||||
const statuses: number[] = [];
|
||||
const names: string[] = [];
|
||||
const codes: string[] = [];
|
||||
while (pending.length > 0) {
|
||||
const current = pending.shift();
|
||||
if (
|
||||
!current ||
|
||||
current.depth > 4 ||
|
||||
!current.value ||
|
||||
typeof current.value !== "object" ||
|
||||
seen.has(current.value)
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
seen.add(current.value);
|
||||
const record = current.value as {
|
||||
cause?: unknown;
|
||||
code?: unknown;
|
||||
details?: { code?: unknown; providerStatus?: unknown };
|
||||
innerHttpError?: { statusCode?: unknown };
|
||||
name?: unknown;
|
||||
original?: unknown;
|
||||
originalError?: unknown;
|
||||
response?: { status?: unknown };
|
||||
status?: unknown;
|
||||
statusCode?: unknown;
|
||||
};
|
||||
for (const status of [
|
||||
record.status,
|
||||
record.statusCode,
|
||||
record.response?.status,
|
||||
record.details?.providerStatus,
|
||||
record.innerHttpError?.statusCode,
|
||||
]) {
|
||||
if (typeof status === "number" && Number.isInteger(status)) {
|
||||
statuses.push(status);
|
||||
}
|
||||
}
|
||||
if (typeof record.name === "string") names.push(record.name);
|
||||
for (const code of [record.code, record.details?.code]) {
|
||||
if (typeof code === "string") codes.push(code);
|
||||
}
|
||||
for (const nested of [
|
||||
record.cause,
|
||||
record.original,
|
||||
record.originalError,
|
||||
]) {
|
||||
pending.push({ depth: current.depth + 1, value: nested });
|
||||
}
|
||||
}
|
||||
if (
|
||||
names.some((name) =>
|
||||
["ValidationError", "NotImplementedError"].includes(name),
|
||||
) ||
|
||||
codes.some((code) =>
|
||||
[
|
||||
"CHAT_ADAPTER_COMPATIBILITY_ERROR",
|
||||
"CHAT_PROVIDER_PRETRANSPORT_REJECTED",
|
||||
"NOT_IMPLEMENTED",
|
||||
"VALIDATION_ERROR",
|
||||
].includes(code),
|
||||
)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
// Slack's Web API can reject a file operation with `ok: false` in an HTTP
|
||||
// 200 response. The SDK's structured platform-error code is still a
|
||||
// definitive provider rejection; the main disposition classifier has
|
||||
// already excluded auth, destination, and rate-limit outcomes.
|
||||
if (codes.includes("slack_webapi_platform_error")) return true;
|
||||
return statuses.some(
|
||||
(status) =>
|
||||
status >= 400 &&
|
||||
status < 500 &&
|
||||
![401, 403, 404, 410, 429].includes(status),
|
||||
);
|
||||
}
|
||||
|
||||
function attachmentFailureKind(
|
||||
payload: SafeChatPublicationPayload,
|
||||
):
|
||||
| { kind: "generated_response"; provider: "discord" | "telegram" }
|
||||
| { kind: "selected_file"; provider: null }
|
||||
| null {
|
||||
if (payload.transportPart?.mode === "discord_markdown_attachment") {
|
||||
return { kind: "generated_response", provider: "discord" };
|
||||
}
|
||||
if (payload.transportPart?.mode === "telegram_markdown_attachment") {
|
||||
return { kind: "generated_response", provider: "telegram" };
|
||||
}
|
||||
if (payload.attachmentIds?.length) {
|
||||
return { kind: "selected_file", provider: null };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export { isExplicitOperatorPublication, ATTACHMENT_FAILURE_NOTICE_PREFIX, attachmentFailureNoticeFence, definiteAttachmentProviderRejection, attachmentFailureKind };
|
||||
@@ -0,0 +1,150 @@
|
||||
import { createDiscordPublicationPreparation } from "../discord/publication-preparation.js";
|
||||
import { createTelegramPublicationPreparation } from "../telegram/publication-preparation.js";
|
||||
import { createChatPublicationReplacement } from "../publication-replacement.js";
|
||||
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import {
|
||||
eq,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatEndpoints,
|
||||
chatPublications,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
renderPublicationTransportText,
|
||||
splitNativePublicationText,
|
||||
} from "../../chat-publication-text-parts.js";
|
||||
|
||||
type PublicationPreparationOptions = {
|
||||
runPublicationToReplace: ReturnType<typeof createChatPublicationReplacement>["runPublicationToReplace"];
|
||||
publicationTransportPreparationBarrier?: (input: { publicationId: string }) => Promise<void>;
|
||||
};
|
||||
export function createChatPublicationPreparation(db: Db, options: PublicationPreparationOptions) {
|
||||
const { runPublicationToReplace } = options;
|
||||
const { prepareDiscordPublication } = createDiscordPublicationPreparation(db, { runPublicationToReplace });
|
||||
const { prepareTelegramPublication } = createTelegramPublicationPreparation(db, { runPublicationToReplace });
|
||||
|
||||
|
||||
async function ensureDurablePublicationTransport(
|
||||
publication: typeof chatPublications.$inferSelect,
|
||||
): Promise<typeof chatPublications.$inferSelect> {
|
||||
const persisted = publication.payload as SafeChatPublicationPayload;
|
||||
if (
|
||||
!persisted ||
|
||||
typeof persisted !== "object" ||
|
||||
typeof persisted.text !== "string"
|
||||
) {
|
||||
throw Object.assign(
|
||||
new Error("Durable chat publication payload is malformed"),
|
||||
{ code: "VALIDATION_ERROR", name: "ValidationError" },
|
||||
);
|
||||
}
|
||||
// Validate closed persisted wrappers even on retries that are already split.
|
||||
renderPublicationTransportText(persisted);
|
||||
await options.publicationTransportPreparationBarrier?.({
|
||||
publicationId: publication.id,
|
||||
});
|
||||
const provider = await db
|
||||
.select({ provider: chatEndpoints.provider })
|
||||
.from(chatEndpoints)
|
||||
.where(eq(chatEndpoints.id, publication.endpointId))
|
||||
.then((rows) => rows[0]?.provider ?? null);
|
||||
if (
|
||||
provider === "slack" ||
|
||||
provider === "github" ||
|
||||
provider === "microsoft-teams"
|
||||
) {
|
||||
if (
|
||||
persisted.transportPart ||
|
||||
persisted.card ||
|
||||
persisted.interactionId ||
|
||||
persisted.attachmentIds?.length
|
||||
)
|
||||
return publication;
|
||||
const parts = splitNativePublicationText(provider, persisted.text);
|
||||
if (parts.length === 1) return publication;
|
||||
return db.transaction(async (tx) => {
|
||||
const current = await tx
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(eq(chatPublications.id, publication.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) return publication;
|
||||
const currentPayload = current.payload as SafeChatPublicationPayload;
|
||||
renderPublicationTransportText(currentPayload);
|
||||
if (
|
||||
currentPayload.transportPart ||
|
||||
currentPayload.card ||
|
||||
currentPayload.interactionId ||
|
||||
currentPayload.attachmentIds?.length ||
|
||||
!["pending", "retry"].includes(current.state)
|
||||
)
|
||||
return current;
|
||||
const currentParts =
|
||||
currentPayload.text === persisted.text
|
||||
? parts
|
||||
: splitNativePublicationText(provider, currentPayload.text);
|
||||
if (currentParts.length === 1) return current;
|
||||
const batchId = current.id;
|
||||
const updatedAt = new Date();
|
||||
const payloadFor = (index: number): SafeChatPublicationPayload => {
|
||||
const part = currentParts[index]!;
|
||||
const payload = {
|
||||
...currentPayload,
|
||||
text: part.text,
|
||||
transportPart: {
|
||||
batchId,
|
||||
index,
|
||||
count: currentParts.length,
|
||||
mode: "inline" as const,
|
||||
orderKey: `${batchId}:${String(index).padStart(4, "0")}`,
|
||||
...(part.prefix ? { prefix: part.prefix } : {}),
|
||||
...(part.suffix ? { suffix: part.suffix } : {}),
|
||||
},
|
||||
};
|
||||
if (index > 0) delete payload.progressState;
|
||||
return payload;
|
||||
};
|
||||
const rootPayload = payloadFor(0);
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: rootPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx.insert(chatPublications).values(
|
||||
currentParts.slice(1).map((_part, offset) => ({
|
||||
companyId: current.companyId,
|
||||
endpointId: current.endpointId,
|
||||
conversationId: current.conversationId,
|
||||
issueId: current.issueId,
|
||||
commentId: current.commentId,
|
||||
idempotencyKey: `${current.idempotencyKey}:${provider}-part:${offset + 1}`,
|
||||
payload: payloadFor(offset + 1),
|
||||
state: "pending" as const,
|
||||
createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`,
|
||||
updatedAt,
|
||||
})),
|
||||
);
|
||||
return { ...current, payload: rootPayload, updatedAt };
|
||||
});
|
||||
}
|
||||
if (provider === "discord") {
|
||||
return prepareDiscordPublication(publication, persisted);
|
||||
}
|
||||
|
||||
if (provider !== "telegram") return publication;
|
||||
return prepareTelegramPublication(publication, persisted);
|
||||
}
|
||||
return { ensureDurablePublicationTransport };
|
||||
}
|
||||
@@ -0,0 +1,389 @@
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import { unresolvedSlackPublicationCondition } from "../../connectors/slack-publication.js";
|
||||
|
||||
import { settleExternalConversation } from "../../slack-conversation-lifecycle.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
asc,
|
||||
eq,
|
||||
gt,
|
||||
inArray,
|
||||
isNotNull,
|
||||
isNull,
|
||||
lte,
|
||||
notExists,
|
||||
notInArray,
|
||||
or,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
import { alias } from "drizzle-orm/pg-core";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatActions,
|
||||
chatEndpointLeases,
|
||||
chatEndpoints,
|
||||
chatPublications,
|
||||
chatTeamsFileTransfers,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import { logger } from "../../../middleware/logger.js";
|
||||
|
||||
import { publicationTransportOrderKey, publicationPrecedes } from "./order.js";
|
||||
import type { createChatPublicationWorker } from "./worker.js";
|
||||
type PublicationQueueOptions = {
|
||||
PUBLICATION_ENDPOINT_CONCURRENCY: number;
|
||||
enqueueInboundWakeupPublications: () => Promise<unknown>;
|
||||
processPendingInteractionWakeups: (limit?: number) => Promise<number>;
|
||||
reconcileTerminalConfirmationActions: (limit?: number) => Promise<number>;
|
||||
processSelectedPublication: ReturnType<typeof createChatPublicationWorker>["processSelectedPublication"];
|
||||
publicationEndpointTasks: Map<string, Promise<void>>;
|
||||
teamsFileRetryAfter: Map<string, number>;
|
||||
redactError: (error: unknown) => string;
|
||||
isShuttingDown: () => boolean;
|
||||
wakePublications: () => void;
|
||||
};
|
||||
export function createChatPublicationQueue(db: Db, options: PublicationQueueOptions) {
|
||||
const { PUBLICATION_ENDPOINT_CONCURRENCY, enqueueInboundWakeupPublications, processPendingInteractionWakeups, reconcileTerminalConfirmationActions, processSelectedPublication, publicationEndpointTasks, teamsFileRetryAfter, redactError, wakePublications, isShuttingDown } = options;
|
||||
|
||||
|
||||
function publicationEligibility({ now, busyEndpointIds = [], coolingTeamsPublicationIds = [], attemptedIds = [] }: { now?: Date; busyEndpointIds?: string[]; coolingTeamsPublicationIds?: string[]; attemptedIds?: string[] } = {}) {
|
||||
const earlierPublication = alias(chatPublications, "earlier_chat_publications");
|
||||
return and(
|
||||
sql`not (${unresolvedSlackPublicationCondition()})`,
|
||||
// An explicit Board send promises both delivery and work. Keep
|
||||
// its text/files pending until the durable wake has been accepted.
|
||||
sql`not exists (select 1 from chat_actions a where a.company_id = ${chatPublications.companyId} and a.endpoint_id = ${chatPublications.endpointId} and a.conversation_id = ${chatPublications.conversationId} and a.kind = 'slack_board_message' and a.status = 'received' and a.payload->>'commentId' = ${chatPublications.commentId}::text)`,
|
||||
or(
|
||||
and(
|
||||
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`,
|
||||
inArray(chatPublications.state, ["pending", "retry"]),
|
||||
notExists(
|
||||
db
|
||||
.select({ id: chatTeamsFileTransfers.id })
|
||||
.from(chatTeamsFileTransfers)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
chatTeamsFileTransfers.companyId,
|
||||
chatPublications.companyId,
|
||||
),
|
||||
eq(
|
||||
chatTeamsFileTransfers.publicationId,
|
||||
chatPublications.id,
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
// Dedicated protocol work shares the endpoint concurrency cap,
|
||||
// never the ordinary publication claim or resend implementation.
|
||||
sql`exists (${db
|
||||
.select({ id: chatTeamsFileTransfers.id })
|
||||
.from(chatTeamsFileTransfers)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
chatTeamsFileTransfers.companyId,
|
||||
chatPublications.companyId,
|
||||
),
|
||||
eq(
|
||||
chatTeamsFileTransfers.endpointId,
|
||||
chatPublications.endpointId,
|
||||
),
|
||||
eq(
|
||||
chatTeamsFileTransfers.conversationId,
|
||||
chatPublications.conversationId,
|
||||
),
|
||||
eq(
|
||||
chatTeamsFileTransfers.publicationId,
|
||||
chatPublications.id,
|
||||
),
|
||||
sql`exists (${db
|
||||
.select({ id: chatEndpoints.id })
|
||||
.from(chatEndpoints)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, chatPublications.endpointId),
|
||||
eq(
|
||||
chatEndpoints.companyId,
|
||||
chatPublications.companyId,
|
||||
),
|
||||
eq(chatEndpoints.provider, "microsoft-teams"),
|
||||
eq(chatEndpoints.status, "active"),
|
||||
),
|
||||
)})`,
|
||||
or(
|
||||
inArray(chatTeamsFileTransfers.phase, [
|
||||
"consent_pending",
|
||||
"upload_pending",
|
||||
"file_info_pending",
|
||||
]),
|
||||
and(
|
||||
eq(chatTeamsFileTransfers.phase, "consent_unknown"),
|
||||
isNotNull(chatTeamsFileTransfers.responseActivityId),
|
||||
sql`${chatTeamsFileTransfers.privateState}->'response' is not null`,
|
||||
),
|
||||
),
|
||||
),
|
||||
)})`,
|
||||
),
|
||||
busyEndpointIds.length > 0
|
||||
? notInArray(chatPublications.endpointId, busyEndpointIds)
|
||||
: undefined,
|
||||
coolingTeamsPublicationIds.length > 0
|
||||
? notInArray(chatPublications.id, coolingTeamsPublicationIds)
|
||||
: undefined,
|
||||
// Inactive endpoints must not consume the eligibility page or
|
||||
// acquire an artificial retry deadline. When an operator resumes
|
||||
// them, due work is immediately eligible; genuine provider retry
|
||||
// deadlines below remain unchanged.
|
||||
notExists(
|
||||
db
|
||||
.select({ id: chatEndpoints.id })
|
||||
.from(chatEndpoints)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, chatPublications.endpointId),
|
||||
inArray(chatEndpoints.status, ["paused", "attention"]),
|
||||
),
|
||||
),
|
||||
),
|
||||
now ? or(isNull(chatPublications.nextAttemptAt), lte(chatPublications.nextAttemptAt, now)) : undefined,
|
||||
attemptedIds.length > 0
|
||||
? notInArray(chatPublications.id, attemptedIds)
|
||||
: undefined,
|
||||
notExists(
|
||||
db
|
||||
.select({ id: earlierPublication.id })
|
||||
.from(earlierPublication)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
earlierPublication.conversationId,
|
||||
chatPublications.conversationId,
|
||||
),
|
||||
publicationPrecedes(earlierPublication, chatPublications),
|
||||
inArray(earlierPublication.state, [
|
||||
"pending",
|
||||
"retry",
|
||||
"streaming",
|
||||
"delivery_unknown",
|
||||
]),
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async function processPendingPublications(
|
||||
limit = 25,
|
||||
{ waitForCompletion = true, maintenance = true }: { waitForCompletion?: boolean; maintenance?: boolean } = {},
|
||||
) {
|
||||
if (isShuttingDown()) return 0;
|
||||
if (maintenance) {
|
||||
await enqueueInboundWakeupPublications();
|
||||
await reconcileTerminalConfirmationActions(limit);
|
||||
}
|
||||
const now = new Date();
|
||||
const staleBefore = new Date(now.getTime() - 60_000);
|
||||
// A process that disappears after the provider accepted a post but before
|
||||
// Paperclip persisted its message id leaves an ambiguous delivery. Never
|
||||
// resend it automatically; an operator can explicitly replay after
|
||||
// checking the provider conversation.
|
||||
const quarantinedPublications = await db.transaction(async tx => {
|
||||
const condition = and(
|
||||
sql`not exists (select 1 from chat_endpoints e where e.id = ${chatPublications.endpointId} and e.publication_mode = 'explicit')`,
|
||||
eq(chatPublications.state, "streaming"),
|
||||
lte(chatPublications.updatedAt, staleBefore),
|
||||
// Teams owns separate staged I/O intents and a longer attempt lease.
|
||||
// Only that protocol may recover/quarantine its in-flight stages.
|
||||
notExists(
|
||||
db
|
||||
.select({ id: chatTeamsFileTransfers.id })
|
||||
.from(chatTeamsFileTransfers)
|
||||
.where(
|
||||
and(
|
||||
eq(
|
||||
chatTeamsFileTransfers.companyId,
|
||||
chatPublications.companyId,
|
||||
),
|
||||
eq(chatTeamsFileTransfers.publicationId, chatPublications.id),
|
||||
),
|
||||
),
|
||||
),
|
||||
notExists(
|
||||
db
|
||||
.select({ id: chatEndpointLeases.id })
|
||||
.from(chatEndpointLeases)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointLeases.companyId, chatPublications.companyId),
|
||||
eq(
|
||||
chatEndpointLeases.endpointId,
|
||||
chatPublications.endpointId,
|
||||
),
|
||||
sql`${chatEndpointLeases.leaseKey} = 'publication:' || ${chatPublications.id}::text || ':' || ${chatPublications.attempts}::text`,
|
||||
gt(chatEndpointLeases.expiresAt, now),
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
const candidates = await tx.select({ id: chatPublications.id }).from(chatPublications).where(condition).for("update", { skipLocked: true });
|
||||
if (!candidates.length) return [];
|
||||
await notifyChatPublicationWork(tx);
|
||||
return tx
|
||||
.update(chatPublications)
|
||||
.set({
|
||||
state: "delivery_unknown",
|
||||
nextAttemptAt: null,
|
||||
redactedError:
|
||||
"Provider delivery could not be confirmed after the worker stopped. Check the external conversation before replaying.",
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(
|
||||
and(condition, inArray(chatPublications.id, candidates.map(row => row.id))),
|
||||
)
|
||||
.returning({ id: chatPublications.id });
|
||||
});
|
||||
if (quarantinedPublications.length > 0) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "delivery_unknown",
|
||||
result: {
|
||||
code: "task_control_delivery_unknown",
|
||||
retryable: false,
|
||||
},
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.kind, "task_control_authorization"),
|
||||
eq(chatActions.status, "processing"),
|
||||
inArray(
|
||||
chatActions.providerActionId,
|
||||
quarantinedPublications.map(
|
||||
(publication) => `task-control-authorization:${publication.id}`,
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
const attemptedIds: string[] = [];
|
||||
const ownedTasks = new Set<Promise<void>>();
|
||||
let failed = false;
|
||||
let firstError: unknown;
|
||||
try {
|
||||
while (attemptedIds.length < limit && !failed && !isShuttingDown()) {
|
||||
if (publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY) {
|
||||
if (!waitForCompletion) break;
|
||||
await Promise.race(publicationEndpointTasks.values());
|
||||
continue;
|
||||
}
|
||||
const busyEndpointIds = [...publicationEndpointTasks.keys()];
|
||||
for (const [id, retryAfter] of teamsFileRetryAfter)
|
||||
if (retryAfter <= Date.now()) teamsFileRetryAfter.delete(id);
|
||||
const coolingTeamsPublicationIds = [...teamsFileRetryAfter.keys()];
|
||||
// Select only each conversation's current head before applying the
|
||||
// global limit. Re-query after every batch so one invocation can still
|
||||
// drain a conversation's newly unblocked milestones in FIFO order.
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(
|
||||
publicationEligibility({ now, busyEndpointIds, coolingTeamsPublicationIds, attemptedIds }),
|
||||
)
|
||||
.orderBy(
|
||||
asc(chatPublications.createdAt),
|
||||
asc(publicationTransportOrderKey(chatPublications)),
|
||||
)
|
||||
.limit(limit - attemptedIds.length);
|
||||
if (rows.length === 0) {
|
||||
// The query excluded the busy-endpoint snapshot above. A worker
|
||||
// may have completed while PostgreSQL selected this empty page,
|
||||
// exposing a new head that must be queried before declaring idle.
|
||||
if (busyEndpointIds.some((id) => !publicationEndpointTasks.has(id)))
|
||||
continue;
|
||||
if (!waitForCompletion || publicationEndpointTasks.size === 0) break;
|
||||
await Promise.race(publicationEndpointTasks.values());
|
||||
continue;
|
||||
}
|
||||
let started = false;
|
||||
for (const selectedPublication of rows) {
|
||||
if (
|
||||
publicationEndpointTasks.size >= PUBLICATION_ENDPOINT_CONCURRENCY ||
|
||||
attemptedIds.length >= limit ||
|
||||
failed ||
|
||||
isShuttingDown()
|
||||
)
|
||||
break;
|
||||
if (publicationEndpointTasks.has(selectedPublication.endpointId))
|
||||
continue;
|
||||
attemptedIds.push(selectedPublication.id);
|
||||
started = true;
|
||||
// Reserve synchronously before provider work; overlapping drains share
|
||||
// this cap. Different conversations for one bot keep the existing
|
||||
// endpoint-exclusive credential fence instead of timing out behind it.
|
||||
const task = Promise.resolve()
|
||||
.then(async () => {
|
||||
await processSelectedPublication(selectedPublication);
|
||||
await settleExternalConversation(db, selectedPublication.companyId, selectedPublication.issueId).catch((err) => {
|
||||
logger.warn({ err, publicationId: selectedPublication.id }, "Slack conversation settlement deferred to reconciliation");
|
||||
});
|
||||
})
|
||||
.catch((error: unknown) => {
|
||||
if (!failed) {
|
||||
failed = true;
|
||||
firstError = error;
|
||||
}
|
||||
if (!waitForCompletion) {
|
||||
logger.error(
|
||||
{
|
||||
endpointId: selectedPublication.endpointId,
|
||||
publicationId: selectedPublication.id,
|
||||
error: redactError(error),
|
||||
},
|
||||
"scheduled chat publication worker failed",
|
||||
);
|
||||
}
|
||||
})
|
||||
.finally(() => {
|
||||
if (
|
||||
publicationEndpointTasks.get(selectedPublication.endpointId) ===
|
||||
task
|
||||
) {
|
||||
publicationEndpointTasks.delete(selectedPublication.endpointId);
|
||||
}
|
||||
ownedTasks.delete(task);
|
||||
wakePublications();
|
||||
});
|
||||
publicationEndpointTasks.set(selectedPublication.endpointId, task);
|
||||
ownedTasks.add(task);
|
||||
}
|
||||
// Re-query while slots remain: one busy endpoint must not fill the
|
||||
// candidate page and hide later eligible endpoints behind its backlog.
|
||||
if (!started && publicationEndpointTasks.size > 0) {
|
||||
if (!waitForCompletion) break;
|
||||
await Promise.race(publicationEndpointTasks.values());
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (!failed) {
|
||||
failed = true;
|
||||
firstError = error;
|
||||
}
|
||||
} finally {
|
||||
// Observe rejection immediately above and join every started worker
|
||||
// before this invocation returns or propagates its original failure.
|
||||
if (waitForCompletion) await Promise.all(ownedTasks);
|
||||
}
|
||||
if (failed) throw firstError;
|
||||
if (maintenance) await processPendingInteractionWakeups(limit);
|
||||
return attemptedIds.length;
|
||||
}
|
||||
return { processPendingPublications, publicationEligibility };
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
import type { DbOrTransaction } from "../runtime-types.js";
|
||||
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
gt,
|
||||
inArray,
|
||||
like,
|
||||
or,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatConversations,
|
||||
chatPublications,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import { projectSafeChatPublication } from "../../chat-publication-projection.js";
|
||||
|
||||
import { publicationTransportOrderKey } from "./order.js";
|
||||
export function createChatPublicationTaskControl(db: Db) {
|
||||
|
||||
|
||||
async function currentTaskControlPayload(
|
||||
publication: typeof chatPublications.$inferSelect,
|
||||
): Promise<SafeChatPublicationPayload> {
|
||||
const persisted = publication.payload as SafeChatPublicationPayload;
|
||||
if (!publication.idempotencyKey.startsWith("control:status:")) {
|
||||
return persisted;
|
||||
}
|
||||
const issue = await db
|
||||
.select({
|
||||
identifier: issues.identifier,
|
||||
status: issues.status,
|
||||
title: issues.title,
|
||||
})
|
||||
.from(issues)
|
||||
.where(
|
||||
and(
|
||||
eq(issues.companyId, publication.companyId),
|
||||
eq(issues.id, publication.issueId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!issue) return persisted;
|
||||
return projectSafeChatPublication({
|
||||
classification: "external",
|
||||
source: "task_control",
|
||||
text: `${issue.identifier}: ${issue.title} — ${issue.status}`,
|
||||
});
|
||||
}
|
||||
|
||||
function taskControlCompletesConversation(
|
||||
publication: Pick<typeof chatPublications.$inferSelect, "idempotencyKey">,
|
||||
): boolean {
|
||||
return /^control:(?:new|close):/.test(publication.idempotencyKey);
|
||||
}
|
||||
|
||||
async function hasCommittedTaskControlCompletion(
|
||||
conversationId: string,
|
||||
database: DbOrTransaction = db,
|
||||
): Promise<boolean> {
|
||||
return database
|
||||
.select({ id: chatPublications.id })
|
||||
.from(chatPublications)
|
||||
.where(
|
||||
and(
|
||||
eq(chatPublications.conversationId, conversationId),
|
||||
eq(chatPublications.state, "published"),
|
||||
or(
|
||||
like(chatPublications.idempotencyKey, "control:new:%"),
|
||||
like(chatPublications.idempotencyKey, "control:close:%"),
|
||||
),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows.length > 0);
|
||||
}
|
||||
|
||||
async function commitTaskControlCompletion(
|
||||
tx: Db,
|
||||
publication: Pick<
|
||||
typeof chatPublications.$inferSelect,
|
||||
| "companyId"
|
||||
| "conversationId"
|
||||
| "createdAt"
|
||||
| "endpointId"
|
||||
| "id"
|
||||
| "idempotencyKey"
|
||||
| "payload"
|
||||
>,
|
||||
committedAt: Date,
|
||||
): Promise<void> {
|
||||
if (!taskControlCompletesConversation(publication)) return;
|
||||
await tx
|
||||
.update(chatConversations)
|
||||
.set({ state: "completed", updatedAt: committedAt })
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, publication.companyId),
|
||||
eq(chatConversations.endpointId, publication.endpointId),
|
||||
eq(chatConversations.id, publication.conversationId),
|
||||
inArray(chatConversations.state, ["active", "waiting"]),
|
||||
),
|
||||
);
|
||||
// The provider-visible completion is the external boundary. Anything
|
||||
// enqueued later for this old generation stays internal even if its run
|
||||
// finishes after /new or /close. A racing insert that commits after this
|
||||
// transaction is rejected by the completed-conversation send guard.
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({
|
||||
state: "cancelled",
|
||||
nextAttemptAt: null,
|
||||
redactedError:
|
||||
"Conversation was completed by a task control before delivery",
|
||||
updatedAt: committedAt,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatPublications.companyId, publication.companyId),
|
||||
eq(chatPublications.endpointId, publication.endpointId),
|
||||
eq(chatPublications.conversationId, publication.conversationId),
|
||||
or(
|
||||
gt(chatPublications.createdAt, publication.createdAt),
|
||||
and(
|
||||
eq(chatPublications.createdAt, publication.createdAt),
|
||||
gt(
|
||||
publicationTransportOrderKey(chatPublications),
|
||||
publication.payload.transportPart?.orderKey ?? publication.id,
|
||||
),
|
||||
),
|
||||
),
|
||||
inArray(chatPublications.state, ["pending", "retry", "streaming"]),
|
||||
sql`not (${chatPublications.idempotencyKey} like 'explicit:%' or ${chatPublications.idempotencyKey} like 'explicit-board:%')`,
|
||||
),
|
||||
);
|
||||
}
|
||||
return { currentTaskControlPayload, hasCommittedTaskControlCompletion, commitTaskControlCompletion };
|
||||
}
|
||||
@@ -0,0 +1,439 @@
|
||||
import type { PublicationTransportInput } from "./types.js";
|
||||
import { createPhotonPublicationDelivery } from "../imessage-photon/publications.js";
|
||||
import { createOutboundAttachments } from "../outbound-attachments.js";
|
||||
import { discordMarkdownAttachment } from "../discord/messages.js";
|
||||
import { telegramMarkdownAttachment } from "../telegram/messages.js";
|
||||
|
||||
import type { EndpointRow, DbOrTransaction } from "../runtime-types.js";
|
||||
|
||||
import { createChatPublicationReplacement } from "../publication-replacement.js";
|
||||
|
||||
import { voiceSessionService } from "../../voice/voice-session-service.js";
|
||||
|
||||
import { voiceSessionStore } from "../../voice/voice-session-store.js";
|
||||
import { type ChatRuntime } from "../../chat-runtime.js";
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
|
||||
import { chatGitHubConfigurations } from "@paperclipai/db";
|
||||
|
||||
import { githubResponseCommentService } from "../../chat-github-response-comments.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
isNull,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatMessageLinks,
|
||||
chatPublications,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
ChatAdapterCapabilities,
|
||||
ChatEndpointSetupState,
|
||||
ChatProvider,
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import { telegramAttachmentForUpload } from "../../chat-telegram-photo.js";
|
||||
|
||||
import {
|
||||
conflict,
|
||||
forbidden,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import {
|
||||
type ChatSdkRuntime,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
|
||||
import { safeChatTaskUrl } from "../../chat-task-url.js";
|
||||
|
||||
import {
|
||||
renderPublicationTransportText,
|
||||
} from "../../chat-publication-text-parts.js";
|
||||
|
||||
import { classifyChatPublicationError } from "../../chat-publication-errors.js";
|
||||
|
||||
import {
|
||||
prepareTeamsInlineImage,
|
||||
} from "../../chat-teams-inline-images.js";
|
||||
import {
|
||||
shouldStreamSafePublicationText,
|
||||
streamSafePublicationText,
|
||||
} from "../../chat-publication-stream.js";
|
||||
|
||||
import {
|
||||
isChatQuestionFormOpenActionId,
|
||||
} from "../../chat-question-forms.js";
|
||||
|
||||
import type {
|
||||
AdapterPostableMessage,
|
||||
Attachment,
|
||||
FileUpload,
|
||||
} from "chat";
|
||||
import { Actions, Button, Card, CardText, LinkButton } from "chat";
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
import { isExplicitOperatorPublication } from "./policy.js";
|
||||
type PublicationTransportOptions = {
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
getTaskBaseUrl: () => string | null | undefined;
|
||||
publicationFiles: ReturnType<typeof createOutboundAttachments>["publicationFiles"];
|
||||
receiptReactionCompletionRunId: ReturnType<typeof createChatPublicationReplacement>["receiptReactionCompletionRunId"];
|
||||
runIdFromMilestonePublication: ReturnType<typeof createChatPublicationReplacement>["runIdFromMilestonePublication"];
|
||||
runtime: Pick<ChatRuntime | ChatSdkRuntime, "get">;
|
||||
runtimeFor: (endpoint: EndpointRow) => Promise<ChatEndpointRuntime>;
|
||||
runtimeGeneration: (setup: ChatEndpointSetupState) => number;
|
||||
CAPABILITIES: Record<ChatProvider, ChatAdapterCapabilities>;
|
||||
voice: Pick<ReturnType<typeof voiceSessionService>, "pushReplies">;
|
||||
voiceStore: Pick<ReturnType<typeof voiceSessionStore>, "enqueuePublication">;
|
||||
scheduleMessageProcessing: (task: () => Promise<void>) => void;
|
||||
};
|
||||
export function createChatPublicationTransport(db: Db, options: PublicationTransportOptions) {
|
||||
const { endpointRecord, fetchImpl, getTaskBaseUrl, publicationFiles, receiptReactionCompletionRunId, runIdFromMilestonePublication, runtime, runtimeFor, runtimeGeneration, CAPABILITIES, voice, voiceStore, scheduleMessageProcessing } = options;
|
||||
const { deliverPhotonPublication } = createPhotonPublicationDelivery(db, { endpointRecord, runtimeGeneration, runtime, receiptReactionCompletionRunId, getTaskBaseUrl });
|
||||
|
||||
|
||||
async function attemptProviderPublication<T>(
|
||||
send: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
return await send();
|
||||
}
|
||||
|
||||
async function editOrPostProviderPublication<T>(
|
||||
edit: () => Promise<T>,
|
||||
post: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
try {
|
||||
return await edit();
|
||||
} catch (error) {
|
||||
// A definite 404 proves the old progress comment no longer exists. It is
|
||||
// therefore safe to create a replacement without risking a duplicate.
|
||||
// Ambiguous transport failures must still enter delivery_unknown.
|
||||
if (classifyChatPublicationError(error, 1).kind !== "resource_unavailable")
|
||||
throw error;
|
||||
return await post();
|
||||
}
|
||||
}
|
||||
|
||||
function safeCardForPublication(
|
||||
payload: SafeChatPublicationPayload,
|
||||
provider: ChatProvider,
|
||||
) {
|
||||
if (!payload.card) return null;
|
||||
const children = [];
|
||||
if (payload.card.body) children.push(CardText(payload.card.body));
|
||||
const actions = (payload.card.actions ?? []).map((action) =>
|
||||
action.type === "callback"
|
||||
? Button({
|
||||
id: action.actionId,
|
||||
label: action.label,
|
||||
style: action.style ?? "default",
|
||||
// Telegram encodes both id and value into callback_data, whose hard
|
||||
// limit is 64 bytes. Its opaque action id resolves through the
|
||||
// durable token ledger, so repeating the interaction UUID is both
|
||||
// unnecessary and too large. Other adapters retain the value as a
|
||||
// defense-in-depth binding supported by their larger envelopes.
|
||||
value: provider === "telegram" ? undefined : payload.interactionId,
|
||||
actionType: isChatQuestionFormOpenActionId(action.actionId)
|
||||
? "modal"
|
||||
: undefined,
|
||||
})
|
||||
: LinkButton({ label: action.label, url: action.url }),
|
||||
);
|
||||
if (actions.length) children.push(Actions(actions));
|
||||
return Card({ title: payload.card.title, children });
|
||||
}
|
||||
|
||||
async function recordExactOutboundPublicationLink(
|
||||
tx: DbOrTransaction,
|
||||
input: {
|
||||
publication: typeof chatPublications.$inferSelect;
|
||||
providerMessageId: string;
|
||||
},
|
||||
): Promise<boolean> {
|
||||
const inserted = await tx
|
||||
.insert(chatMessageLinks)
|
||||
.values({
|
||||
companyId: input.publication.companyId,
|
||||
endpointId: input.publication.endpointId,
|
||||
conversationId: input.publication.conversationId,
|
||||
publicationId: input.publication.id,
|
||||
commentId: input.publication.commentId,
|
||||
providerMessageId: input.providerMessageId,
|
||||
direction: "outbound",
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning({ id: chatMessageLinks.id });
|
||||
if (inserted.length) return true;
|
||||
return tx
|
||||
.select({ id: chatMessageLinks.id })
|
||||
.from(chatMessageLinks)
|
||||
.where(
|
||||
and(
|
||||
eq(chatMessageLinks.companyId, input.publication.companyId),
|
||||
eq(chatMessageLinks.endpointId, input.publication.endpointId),
|
||||
eq(chatMessageLinks.conversationId, input.publication.conversationId),
|
||||
eq(chatMessageLinks.publicationId, input.publication.id),
|
||||
input.publication.commentId
|
||||
? eq(chatMessageLinks.commentId, input.publication.commentId)
|
||||
: isNull(chatMessageLinks.commentId),
|
||||
eq(chatMessageLinks.providerMessageId, input.providerMessageId),
|
||||
eq(chatMessageLinks.direction, "outbound"),
|
||||
),
|
||||
)
|
||||
.limit(1)
|
||||
.then((rows) => rows.length > 0);
|
||||
}
|
||||
|
||||
async function postSafePublication(input: PublicationTransportInput) {
|
||||
if (input.endpoint.provider === "speko") {
|
||||
const receipt = await voiceStore.enqueuePublication(input.endpoint.companyId, input.publication.id);
|
||||
scheduleMessageProcessing(() => voice.pushReplies(25, input.endpoint.companyId, input.endpoint.id).then(() => undefined));
|
||||
return receipt;
|
||||
}
|
||||
if (input.endpoint.provider === "github" && ["failed", "completed"].includes(input.payload.progressState ?? "") &&
|
||||
!input.payload.interactionId && !isExplicitOperatorPublication(input.publication) &&
|
||||
(await db.select({ endpointId: chatGitHubConfigurations.endpointId }).from(chatGitHubConfigurations).where(and(
|
||||
eq(chatGitHubConfigurations.companyId, input.endpoint.companyId), eq(chatGitHubConfigurations.endpointId, input.endpoint.id))).limit(1)).length > 0) {
|
||||
const runId = runIdFromMilestonePublication(input.publication);
|
||||
if (!runId) throw forbidden("GitHub failure has no run binding");
|
||||
const receipt = await githubResponseCommentService(db, fetchImpl).failRun({
|
||||
companyId: input.publication.companyId, endpointId: input.endpoint.id,
|
||||
issueId: input.publication.issueId, runId, body: input.payload.progressState === "completed"
|
||||
? "This turn ended without publishing a final response. See the Paperclip task for details."
|
||||
: renderPublicationTransportText(input.payload),
|
||||
});
|
||||
if (!receipt) throw conflict("GitHub response publication is busy");
|
||||
return { id: receipt.id, threadId: input.conversation.externalThreadId, raw: {}, text: input.payload.text };
|
||||
}
|
||||
const endpointRuntime = await runtimeFor(input.endpoint);
|
||||
const thread = endpointRuntime.thread(input.conversation.externalThreadId);
|
||||
const card = safeCardForPublication(input.payload, input.endpoint.provider);
|
||||
let attachments: Attachment[] = [];
|
||||
let files: FileUpload[] = [];
|
||||
let text = renderPublicationTransportText(input.payload);
|
||||
if (input.payload.attachmentIds?.length) {
|
||||
if (
|
||||
input.endpoint.provider === "microsoft-teams" &&
|
||||
!input.conversation.isDirectMessage
|
||||
) {
|
||||
// The staged personal-file lane remains authoritative for consent.
|
||||
// Channel/group pictures need no OneDrive upload or public asset URL:
|
||||
// send only verified original picture bytes through the native adapter.
|
||||
const candidates = await publicationFiles(
|
||||
input.publication,
|
||||
input.payload,
|
||||
{ teamsInlineImagesOnly: true },
|
||||
);
|
||||
const pictures = await Promise.all(
|
||||
candidates.map(prepareTeamsInlineImage),
|
||||
);
|
||||
if (pictures.length && pictures.every((picture) => picture !== null)) {
|
||||
files = pictures;
|
||||
}
|
||||
}
|
||||
const nativeFileSurface =
|
||||
CAPABILITIES[input.endpoint.provider].files &&
|
||||
input.endpoint.provider !== "microsoft-teams";
|
||||
if (nativeFileSurface) {
|
||||
const uploads = await publicationFiles(
|
||||
input.publication,
|
||||
input.payload,
|
||||
);
|
||||
if (input.endpoint.provider === "telegram") {
|
||||
attachments = await Promise.all(
|
||||
uploads.map(telegramAttachmentForUpload),
|
||||
);
|
||||
} else {
|
||||
files = uploads;
|
||||
}
|
||||
} else if (files.length === 0) {
|
||||
const taskUrl = safeChatTaskUrl(
|
||||
getTaskBaseUrl(),
|
||||
input.publication.issueId,
|
||||
);
|
||||
if (
|
||||
input.endpoint.provider === "github" ||
|
||||
input.endpoint.provider === "microsoft-teams"
|
||||
) {
|
||||
// GitHub lacks a bot upload contract. Teams reaches this fallback
|
||||
// only without an admissible personal-consent destination. Replace
|
||||
// our generated caption, preserving any future custom text.
|
||||
const generatedFileLabel = /^Shared (.+)\.$/s.exec(text)?.[1] ?? null;
|
||||
const limitation =
|
||||
input.endpoint.provider === "github"
|
||||
? "This GitHub App connection cannot upload file bytes into comments."
|
||||
: "Direct file delivery isn't available for this Teams conversation.";
|
||||
if (generatedFileLabel) {
|
||||
const saved = taskUrl
|
||||
? `File saved on the Paperclip task: ${generatedFileLabel}.`
|
||||
: `File saved on the private Paperclip task: ${generatedFileLabel}.`;
|
||||
text = taskUrl
|
||||
? `${saved} ${limitation} Download it: ${taskUrl}`
|
||||
: `${saved} ${limitation}`;
|
||||
} else {
|
||||
const handoff = taskUrl
|
||||
? `Open the file on its Paperclip task: ${taskUrl}`
|
||||
: "The file remains available only on the private Paperclip task.";
|
||||
text = `${text}\n\n${limitation} ${handoff}`;
|
||||
}
|
||||
} else {
|
||||
const attachmentFallback = taskUrl
|
||||
? `Open the task in Paperclip: ${taskUrl}`
|
||||
: "Open the task in Paperclip to download the attachment.";
|
||||
text = `${text}\n\n${attachmentFallback}`;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (input.endpoint.provider === "imessage-photon") {
|
||||
return deliverPhotonPublication(input, endpointRuntime, thread, text, files);
|
||||
}
|
||||
if (
|
||||
input.endpoint.provider === "discord" &&
|
||||
input.payload.transportPart?.mode === "discord_markdown_attachment"
|
||||
) {
|
||||
files.push(discordMarkdownAttachment(text));
|
||||
text =
|
||||
input.payload.transportPart.count > 1
|
||||
? "Complete response attached."
|
||||
: "Paperclip attached the complete response because it exceeds Discord’s message limit.";
|
||||
}
|
||||
if (
|
||||
input.endpoint.provider === "telegram" &&
|
||||
input.payload.transportPart?.mode === "telegram_markdown_attachment"
|
||||
) {
|
||||
attachments.push(telegramMarkdownAttachment(text));
|
||||
text =
|
||||
input.payload.transportPart.count > 1
|
||||
? "Complete response attached."
|
||||
: "Paperclip attached the complete response to preserve its Markdown formatting.";
|
||||
}
|
||||
if (card && CAPABILITIES[input.endpoint.provider].cards) {
|
||||
return await attemptProviderPublication(async () =>
|
||||
input.replaceProviderMessageId
|
||||
? await editOrPostProviderPublication(
|
||||
() =>
|
||||
thread.adapter.editMessage(
|
||||
thread.id,
|
||||
input.replaceProviderMessageId!,
|
||||
{
|
||||
card,
|
||||
fallbackText: text,
|
||||
...(files.length ? { files } : {}),
|
||||
},
|
||||
),
|
||||
() =>
|
||||
thread.post({
|
||||
card,
|
||||
fallbackText: text,
|
||||
...(files.length ? { files } : {}),
|
||||
}),
|
||||
)
|
||||
: await thread.post({
|
||||
card,
|
||||
fallbackText: text,
|
||||
...(files.length ? { files } : {}),
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (attachments.length) {
|
||||
const attachmentMessage: AdapterPostableMessage = {
|
||||
attachments,
|
||||
markdown: text,
|
||||
};
|
||||
return await attemptProviderPublication(async () =>
|
||||
input.replaceProviderMessageId
|
||||
? await editOrPostProviderPublication(
|
||||
() =>
|
||||
thread.adapter.editMessage(
|
||||
thread.id,
|
||||
input.replaceProviderMessageId!,
|
||||
attachmentMessage,
|
||||
),
|
||||
() => thread.post(attachmentMessage),
|
||||
)
|
||||
: await thread.post(attachmentMessage),
|
||||
);
|
||||
}
|
||||
if (files.length) {
|
||||
// The pinned Slack adapter performs file upload and text publication as
|
||||
// two provider calls when both fields are present. A rate limit or
|
||||
// network failure on the second call would make an automatic retry
|
||||
// duplicate the already-shared file. File publications have their own
|
||||
// ordered outbox row, so Slack receives a single native upload here.
|
||||
const fileMessage: AdapterPostableMessage = {
|
||||
// Empty Slack markdown keeps the SDK call type-safe while its pinned
|
||||
// adapter treats this as a file-only upload and skips chat.postMessage.
|
||||
markdown: input.endpoint.provider === "slack" ? "" : text,
|
||||
files,
|
||||
};
|
||||
if (
|
||||
input.endpoint.provider === "slack" &&
|
||||
!input.replaceProviderMessageId &&
|
||||
!input.payload.card &&
|
||||
!input.payload.interactionId &&
|
||||
!input.publication.idempotencyKey.startsWith("control:") &&
|
||||
input.payload.attachmentIds?.length &&
|
||||
input.onSlackFileUploadAccepted
|
||||
) {
|
||||
return await attemptProviderPublication(
|
||||
async () =>
|
||||
await endpointRuntime.postSlackFilePublication(
|
||||
thread.id,
|
||||
fileMessage,
|
||||
input.onSlackFileUploadAccepted!,
|
||||
),
|
||||
);
|
||||
}
|
||||
return await attemptProviderPublication(async () =>
|
||||
input.replaceProviderMessageId
|
||||
? await editOrPostProviderPublication(
|
||||
() =>
|
||||
thread.adapter.editMessage(
|
||||
thread.id,
|
||||
input.replaceProviderMessageId!,
|
||||
fileMessage,
|
||||
),
|
||||
() => thread.post(fileMessage),
|
||||
)
|
||||
: await thread.post(fileMessage),
|
||||
);
|
||||
}
|
||||
if (
|
||||
!input.replaceProviderMessageId &&
|
||||
!input.payload.transportPart &&
|
||||
CAPABILITIES[input.endpoint.provider].nativeStreaming &&
|
||||
(input.endpoint.provider !== "telegram" ||
|
||||
!input.conversation.isDirectMessage ||
|
||||
input.telegramDraftControl !== undefined) &&
|
||||
shouldStreamSafePublicationText(text)
|
||||
) {
|
||||
return await attemptProviderPublication(async () =>
|
||||
input.telegramDraftControl
|
||||
? endpointRuntime.streamTelegramDraft(
|
||||
input.conversation.externalThreadId,
|
||||
streamSafePublicationText(text),
|
||||
input.telegramDraftControl,
|
||||
)
|
||||
: thread.post(streamSafePublicationText(text)),
|
||||
);
|
||||
}
|
||||
return await attemptProviderPublication(async () =>
|
||||
input.replaceProviderMessageId
|
||||
? await editOrPostProviderPublication(
|
||||
() =>
|
||||
thread.adapter.editMessage(
|
||||
thread.id,
|
||||
input.replaceProviderMessageId!,
|
||||
{ markdown: text },
|
||||
),
|
||||
() => thread.post({ markdown: text }),
|
||||
)
|
||||
: await thread.post({ markdown: text }),
|
||||
);
|
||||
}
|
||||
return { postSafePublication, recordExactOutboundPublicationLink };
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { chatConversations, chatPublications } from "@paperclipai/db";
|
||||
import type { SafeChatPublicationPayload } from "@paperclipai/shared";
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
import type { TelegramDraftControl } from "../../chat-telegram-draft-stop.js";
|
||||
import type { SlackFileUploadAcceptedReceipt } from "../../chat-sdk-runtime.js";
|
||||
type ConversationRow = typeof chatConversations.$inferSelect;
|
||||
export type PublicationTransportInput = {
|
||||
endpoint: EndpointRow;
|
||||
conversation: ConversationRow;
|
||||
publication: typeof chatPublications.$inferSelect;
|
||||
payload: SafeChatPublicationPayload;
|
||||
replaceProviderMessageId?: string | null;
|
||||
telegramDraftControl?: TelegramDraftControl;
|
||||
beforePhotonWrite?(): Promise<void>;
|
||||
onSlackFileUploadAccepted?: (
|
||||
receipt: SlackFileUploadAcceptedReceipt,
|
||||
) => Promise<void>;
|
||||
};
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,3 +1,5 @@
|
||||
import type { Message, Thread } from "chat";
|
||||
import type { ChatSdkMessageCallbackEvent } from "../chat-sdk-runtime.js";
|
||||
import type { Db, chatEndpoints } from "@paperclipai/db";
|
||||
import type { ToolCredentialSecretRef, ChatEndpointSetupState, ChatEndpointCallbackSurfaces } from "@paperclipai/shared";
|
||||
import type { ChatEndpointRuntime } from "../voice/voice-runtime.js";
|
||||
@@ -66,3 +68,26 @@ export type PreparedProviderInventory = {
|
||||
inventory: import("../chat-provider-inventory.js").ChatProviderInventoryResult | null;
|
||||
managementUrl?: string;
|
||||
};
|
||||
|
||||
export type LiveInboundMessage = {
|
||||
endpoint: EndpointRow;
|
||||
message: Message;
|
||||
receiptReactionSupported: boolean;
|
||||
providerUrl: string | null;
|
||||
runtimeContext?: InboundRuntimeContext;
|
||||
thread: Thread;
|
||||
trigger: ChatSdkMessageCallbackEvent["trigger"];
|
||||
};
|
||||
|
||||
export type DiscordGatewayOwnership = {
|
||||
companyId: string;
|
||||
context: RuntimeContext;
|
||||
endpointId: string;
|
||||
expiresAt: Date;
|
||||
leaseKey: "discord_gateway_runtime" | "photon_receiver_runtime";
|
||||
renewTimer: ReturnType<typeof setInterval> | null;
|
||||
renewal: Promise<void> | null;
|
||||
stopPromise: Promise<void> | null;
|
||||
stopping: boolean;
|
||||
token: string;
|
||||
};
|
||||
@@ -0,0 +1,88 @@
|
||||
import {
|
||||
and,
|
||||
eq,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatSdkState,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import type {
|
||||
ChatSdkStateCompareAndSetInput,
|
||||
ChatSdkStateDeleteInput,
|
||||
ChatSdkStatePersistence,
|
||||
ChatSdkStateScope,
|
||||
} from "../chat-sdk-state.js";
|
||||
|
||||
export function createChatSdkStatePersistence(db: Db): ChatSdkStatePersistence {
|
||||
return {
|
||||
async read(scope: ChatSdkStateScope, key: string) {
|
||||
return db
|
||||
.select({
|
||||
value: chatSdkState.value,
|
||||
version: chatSdkState.version,
|
||||
expiresAt: chatSdkState.expiresAt,
|
||||
})
|
||||
.from(chatSdkState)
|
||||
.where(
|
||||
and(
|
||||
eq(chatSdkState.companyId, scope.companyId),
|
||||
eq(chatSdkState.endpointId, scope.endpointId),
|
||||
eq(chatSdkState.stateKey, key),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
},
|
||||
async compareAndSet(input: ChatSdkStateCompareAndSetInput) {
|
||||
if (input.expectedVersion === null) {
|
||||
const inserted = await db
|
||||
.insert(chatSdkState)
|
||||
.values({
|
||||
companyId: input.companyId,
|
||||
endpointId: input.endpointId,
|
||||
stateKey: input.key,
|
||||
version: 1,
|
||||
value: input.value,
|
||||
expiresAt: input.expiresAt,
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning({ id: chatSdkState.id });
|
||||
return inserted.length === 1;
|
||||
}
|
||||
const updated = await db
|
||||
.update(chatSdkState)
|
||||
.set({
|
||||
version: input.expectedVersion + 1,
|
||||
value: input.value,
|
||||
expiresAt: input.expiresAt,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatSdkState.companyId, input.companyId),
|
||||
eq(chatSdkState.endpointId, input.endpointId),
|
||||
eq(chatSdkState.stateKey, input.key),
|
||||
eq(chatSdkState.version, input.expectedVersion),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatSdkState.id });
|
||||
return updated.length === 1;
|
||||
},
|
||||
async deleteIfVersion(input: ChatSdkStateDeleteInput) {
|
||||
const deleted = await db
|
||||
.delete(chatSdkState)
|
||||
.where(
|
||||
and(
|
||||
eq(chatSdkState.companyId, input.companyId),
|
||||
eq(chatSdkState.endpointId, input.endpointId),
|
||||
eq(chatSdkState.stateKey, input.key),
|
||||
eq(chatSdkState.version, input.expectedVersion),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatSdkState.id });
|
||||
return deleted.length === 1;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import { slackCommandForAgent } from "./commands.js";
|
||||
import { canonicalCallbackUrl, slackCallbackMatchesPublicUrl } from "./callback-verification.js";
|
||||
import type { EndpointRow, SlackCallbackSurface, InternalSetupState } from "../runtime-types.js";
|
||||
|
||||
import type {
|
||||
ChatEndpointCallbackSurfaces,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
export function slackSetupState(input: {
|
||||
endpoint: Pick<
|
||||
EndpointRow,
|
||||
| "provider"
|
||||
| "providerAccountId"
|
||||
| "publicId"
|
||||
| "status"
|
||||
| "setup"
|
||||
| "botUsername"
|
||||
>;
|
||||
webhookUrl: string | null;
|
||||
step: EndpointRow["setup"]["step"];
|
||||
assignedAgentName?: string | null;
|
||||
}) {
|
||||
const { endpoint, webhookUrl, step, assignedAgentName } = input;
|
||||
|
||||
const observations = (endpoint.setup as InternalSetupState)
|
||||
.slackCallbackSurfaces;
|
||||
const currentUrl = webhookUrl ? canonicalCallbackUrl(webhookUrl) : null;
|
||||
const surfaceState = (
|
||||
surface: SlackCallbackSurface,
|
||||
): ChatEndpointCallbackSurfaces[SlackCallbackSurface] => {
|
||||
const observation = observations?.[surface];
|
||||
if (!observation) return { status: "unverified" };
|
||||
return {
|
||||
status:
|
||||
currentUrl !== null &&
|
||||
slackCallbackMatchesPublicUrl(observation.url, currentUrl)
|
||||
? "current"
|
||||
: "stale",
|
||||
observedAt: observation.observedAt,
|
||||
};
|
||||
};
|
||||
const callbackSurfaces: ChatEndpointCallbackSurfaces = {
|
||||
events: surfaceState("events"),
|
||||
interactivity: surfaceState("interactivity"),
|
||||
slashCommands: surfaceState("slashCommands"),
|
||||
};
|
||||
return {
|
||||
step,
|
||||
testStartedAt: endpoint.setup.testStartedAt ?? null,
|
||||
testSkipped: endpoint.setup.testSkipped ?? false,
|
||||
authorizationUrl: "https://api.slack.com/apps?new_app=1",
|
||||
providerUrl: "https://app.slack.com/",
|
||||
webhookUrl,
|
||||
webhookVerifiedAt: endpoint.setup.webhookVerifiedAt ?? null,
|
||||
callbackSurfaces,
|
||||
callbacksNeedUpdate: Object.values(callbackSurfaces).some(
|
||||
(surface) => surface.status === "stale",
|
||||
),
|
||||
slackApp: endpoint.setup.slackApp,
|
||||
slackSetupMethod: endpoint.setup.slackSetupMethod,
|
||||
slackAvatar: endpoint.setup.slackAvatar,
|
||||
slackManagerError: endpoint.setup.slackManagerError,
|
||||
slackAccount: endpoint.setup.slackAccount,
|
||||
// Slack registers the slash command in the provider configuration.
|
||||
// Keep that identity immutable when the assigned agent is renamed;
|
||||
// deriving it remains only a compatibility path for older rows.
|
||||
command:
|
||||
typeof endpoint.setup.command === "string"
|
||||
? endpoint.setup.command
|
||||
: assignedAgentName
|
||||
? slackCommandForAgent(assignedAgentName, endpoint.publicId)
|
||||
: null,
|
||||
} as const;
|
||||
|
||||
}
|
||||
|
||||
import type { ResolvedChatSdkProviderConfig } from "../../chat-sdk-runtime.js";
|
||||
|
||||
export function slackRuntimeConfig(endpoint: EndpointRow, userName: string, credentials: Record<string, string>): ResolvedChatSdkProviderConfig {
|
||||
return {
|
||||
provider: "slack",
|
||||
userName,
|
||||
credentials: {
|
||||
botToken: credentials.botToken,
|
||||
signingSecret: credentials.signingSecret,
|
||||
botUserId: endpoint.botExternalId ?? undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import type { EndpointRow } from "../runtime-types.js";
|
||||
|
||||
import { syncSpekoVoiceTools } from "../../voice/speko-agent-tools.js";
|
||||
import { configureSpekoSessionTools } from "../../voice/speko-tool-setup.js";
|
||||
import { createSpekoProvider } from "../../voice/speko-provider.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
forbidden,
|
||||
unprocessable,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import type { DbTransaction } from "../runtime-types.js";
|
||||
type SpekoSetupOptions = {
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
getWebhookPublicBaseUrl: () => string | null;
|
||||
allowLocalVoiceBoard?: boolean;
|
||||
};
|
||||
export function createSpekoSetup(db: Db, options: SpekoSetupOptions) {
|
||||
const { fetchImpl, getWebhookPublicBaseUrl } = options;
|
||||
|
||||
async function installSpekoVoiceTools(tx: DbTransaction, endpoint: EndpointRow, actorUserId?: string | null) {
|
||||
if (!actorUserId) throw forbidden("An authenticated connection owner is required to install phone tools");
|
||||
await syncSpekoVoiceTools(tx, endpoint, actorUserId, true, options.allowLocalVoiceBoard === true);
|
||||
}
|
||||
|
||||
async function configureSpekoTools(endpoint: EndpointRow, credentials: Record<string, string>, credentialLease: CredentialMutationLeaseGuard, previousSpekoSigningSecret: string | undefined) {
|
||||
const webhookPublicBaseUrl = getWebhookPublicBaseUrl();
|
||||
if (!webhookPublicBaseUrl?.startsWith("https://")) throw unprocessable("Speko needs an explicitly configured public HTTPS callback origin");
|
||||
await configureSpekoSessionTools(db, { companyId: endpoint.companyId, endpointId: endpoint.id, agentId: credentials.agentId, callbackUrl: `${webhookPublicBaseUrl}/api/voice-webhooks/${endpoint.publicId}/tools`, signingSecret: credentials.signingSecret, previousSigningSecret: previousSpekoSigningSecret, client: createSpekoProvider(credentials.apiKey, fetchImpl), assertOwned: () => credentialLease.assertOwned() });
|
||||
}
|
||||
|
||||
return { installSpekoVoiceTools, configureSpekoTools };
|
||||
}
|
||||
@@ -0,0 +1,656 @@
|
||||
import type { DbTransaction, RuntimeContext, LifecycleRuntimeFence } from "../runtime-types.js";
|
||||
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
|
||||
import {
|
||||
and,
|
||||
asc,
|
||||
eq,
|
||||
inArray,
|
||||
ne,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
chatActions,
|
||||
chatConversations,
|
||||
chatEndpoints,
|
||||
chatExternalPrincipals,
|
||||
chatPublications,
|
||||
issueThreadInteractions,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
|
||||
import {
|
||||
type ChatSdkCallbackEvent,
|
||||
} from "../../chat-sdk-runtime.js";
|
||||
|
||||
import {
|
||||
logActivity,
|
||||
} from "../../activity-log.js";
|
||||
|
||||
import { projectSafeChatPublication } from "../../chat-publication-projection.js";
|
||||
|
||||
import { issueThreadInteractionService } from "../../issue-thread-interactions.js";
|
||||
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
import type { ActionEvent } from "chat";
|
||||
import { nativeTelegramConfirmation } from "../../chat-interaction-publications.js";
|
||||
type TelegramConfirmationInput = {
|
||||
record: EndpointRecord;
|
||||
event: ChatSdkCallbackEvent<ActionEvent>;
|
||||
runtimeContext: RuntimeContext;
|
||||
conversation: typeof chatConversations.$inferSelect;
|
||||
principal: { principal: typeof chatExternalPrincipals.$inferSelect; userId: string };
|
||||
issued: { publication: typeof chatPublications.$inferSelect };
|
||||
interaction: Awaited<ReturnType<ReturnType<typeof issueThreadInteractionService>["listForIssue"]>>[number];
|
||||
deny: (known?: { conversationId?: string | null; principalId?: string | null }) => Promise<void>;
|
||||
safelyKnown: { conversationId: string; principalId: string };
|
||||
};
|
||||
type TelegramConfirmationOptions = {
|
||||
requireCurrentExternalActionAuthorization: (tx: DbTransaction, input: { conversationId: string; endpointId: string; expectedUserId: string; principalId: string; runtimeContext: LifecycleRuntimeFence }) => Promise<void>;
|
||||
isExternalActionAuthorizationChange: (error: unknown) => boolean;
|
||||
scheduleMessageProcessing: (task: () => Promise<void>) => void;
|
||||
processPendingPublications: () => Promise<unknown>;
|
||||
confirmationResolutionPersistBarrier?: () => Promise<void>;
|
||||
};
|
||||
export function createTelegramConfirmationActions(db: Db, options: TelegramConfirmationOptions) {
|
||||
const { requireCurrentExternalActionAuthorization, isExternalActionAuthorizationChange, scheduleMessageProcessing, processPendingPublications } = options;
|
||||
async function handleTelegramConfirmation(input: TelegramConfirmationInput) {
|
||||
const { record, event, runtimeContext, conversation, principal, issued, interaction, deny, safelyKnown } = input;
|
||||
|
||||
if (
|
||||
event.provider !== "telegram" ||
|
||||
!nativeTelegramConfirmation(interaction)
|
||||
) {
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
const action = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, record.endpoint.companyId),
|
||||
eq(chatActions.endpointId, record.endpoint.id),
|
||||
eq(chatActions.conversationId, conversation.id),
|
||||
eq(chatActions.kind, "confirmation_response"),
|
||||
eq(chatActions.providerActionId, event.event.actionId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!action || action.status !== "issued") return deny(safelyKnown);
|
||||
const tokenPayload = action.payload;
|
||||
const tokenExpiresAt =
|
||||
typeof tokenPayload.expiresAt === "string"
|
||||
? Date.parse(tokenPayload.expiresAt)
|
||||
: Number.NaN;
|
||||
if (!Number.isFinite(tokenExpiresAt) || tokenExpiresAt <= Date.now()) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "expired",
|
||||
result: { code: "confirmation_action_token_expired" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
);
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
if (
|
||||
tokenPayload.version !== 1 ||
|
||||
tokenPayload.publicationId !== issued.publication.id ||
|
||||
tokenPayload.interactionId !== interaction.id ||
|
||||
(tokenPayload.decision !== "accept" &&
|
||||
tokenPayload.decision !== "reject")
|
||||
) {
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
|
||||
const issue = await db
|
||||
.select({
|
||||
id: issues.id,
|
||||
companyId: issues.companyId,
|
||||
projectId: issues.projectId,
|
||||
goalId: issues.goalId,
|
||||
status: issues.status,
|
||||
assigneeAgentId: issues.assigneeAgentId,
|
||||
})
|
||||
.from(issues)
|
||||
.where(
|
||||
and(
|
||||
eq(issues.companyId, record.endpoint.companyId),
|
||||
eq(issues.id, conversation.issueId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!issue) {
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
|
||||
const decision = tokenPayload.decision;
|
||||
try {
|
||||
if (decision === "accept") {
|
||||
await issueThreadInteractionService(db).acceptInteraction(
|
||||
issue,
|
||||
interaction.id,
|
||||
{},
|
||||
{ userId: principal.userId },
|
||||
{
|
||||
beforeResolveInTransaction: async (tx) => {
|
||||
await requireCurrentExternalActionAuthorization(tx, {
|
||||
conversationId: conversation.id,
|
||||
endpointId: record.endpoint.id,
|
||||
expectedUserId: principal.userId!,
|
||||
principalId: principal.principal.id,
|
||||
runtimeContext,
|
||||
});
|
||||
},
|
||||
afterResolveInTransaction: async (tx, resolved) => {
|
||||
const [owned] = await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
principalId: principal.principal.id,
|
||||
status: "processing",
|
||||
payload: {
|
||||
...tokenPayload,
|
||||
messageId: event.event.messageId,
|
||||
actionId: event.event.actionId,
|
||||
value: null,
|
||||
},
|
||||
result: {
|
||||
code: "confirmation_resolution_committed_by_provider",
|
||||
interactionId: resolved.id,
|
||||
interactionStatus: resolved.status,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatActions.id });
|
||||
if (!owned) {
|
||||
throw new Error(
|
||||
"External confirmation action ownership changed before interaction commit",
|
||||
);
|
||||
}
|
||||
},
|
||||
},
|
||||
);
|
||||
} else {
|
||||
await issueThreadInteractionService(db).rejectInteraction(
|
||||
issue,
|
||||
interaction.id,
|
||||
{},
|
||||
{ userId: principal.userId },
|
||||
{
|
||||
beforeResolveInTransaction: async (tx) => {
|
||||
await requireCurrentExternalActionAuthorization(tx, {
|
||||
conversationId: conversation.id,
|
||||
endpointId: record.endpoint.id,
|
||||
expectedUserId: principal.userId!,
|
||||
principalId: principal.principal.id,
|
||||
runtimeContext,
|
||||
});
|
||||
},
|
||||
afterResolveInTransaction: async (tx, resolved) => {
|
||||
const [owned] = await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
principalId: principal.principal.id,
|
||||
status: "processing",
|
||||
payload: {
|
||||
...tokenPayload,
|
||||
messageId: event.event.messageId,
|
||||
actionId: event.event.actionId,
|
||||
value: null,
|
||||
},
|
||||
result: {
|
||||
code: "confirmation_resolution_committed_by_provider",
|
||||
interactionId: resolved.id,
|
||||
interactionStatus: resolved.status,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "issued"),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatActions.id });
|
||||
if (!owned) {
|
||||
throw new Error(
|
||||
"External confirmation action ownership changed before interaction commit",
|
||||
);
|
||||
}
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
if (isExternalActionAuthorizationChange(error)) {
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
const currentStatus = await db
|
||||
.select({ status: issueThreadInteractions.status })
|
||||
.from(issueThreadInteractions)
|
||||
.where(eq(issueThreadInteractions.id, interaction.id))
|
||||
.then((rows) => rows[0]?.status ?? null);
|
||||
if (currentStatus !== null && currentStatus !== "pending") {
|
||||
await settleTerminalConfirmationAction(action.id);
|
||||
const completedAction = await db
|
||||
.select({ status: chatActions.status, result: chatActions.result })
|
||||
.from(chatActions)
|
||||
.where(eq(chatActions.id, action.id))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (
|
||||
completedAction?.status === "processed" &&
|
||||
completedAction.result?.code ===
|
||||
"confirmation_resolution_committed_by_provider"
|
||||
) {
|
||||
return;
|
||||
}
|
||||
return deny(safelyKnown);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
await options.confirmationResolutionPersistBarrier?.();
|
||||
if (!(await settleTerminalConfirmationAction(action.id))) {
|
||||
throw new Error(
|
||||
"External confirmation action ownership changed before settlement",
|
||||
);
|
||||
}
|
||||
|
||||
scheduleMessageProcessing(async () => {
|
||||
await processPendingPublications();
|
||||
});
|
||||
return;
|
||||
|
||||
}
|
||||
|
||||
|
||||
async function settleTerminalConfirmationAction(
|
||||
actionId: string,
|
||||
): Promise<boolean> {
|
||||
const action = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, actionId),
|
||||
eq(chatActions.kind, "confirmation_response"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (action?.status === "processed") return true;
|
||||
if (!action || action.status !== "processing" || !action.conversationId)
|
||||
return false;
|
||||
const token = action.payload;
|
||||
if (
|
||||
token.version !== 1 ||
|
||||
typeof token.publicationId !== "string" ||
|
||||
typeof token.interactionId !== "string" ||
|
||||
(token.decision !== "accept" && token.decision !== "reject")
|
||||
) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "failed",
|
||||
result: { code: "confirmation_action_payload_invalid" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
return false;
|
||||
}
|
||||
const [conversation, publication, endpoint] = await Promise.all([
|
||||
db
|
||||
.select()
|
||||
.from(chatConversations)
|
||||
.where(
|
||||
and(
|
||||
eq(chatConversations.companyId, action.companyId),
|
||||
eq(chatConversations.endpointId, action.endpointId),
|
||||
eq(chatConversations.id, action.conversationId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null),
|
||||
db
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(
|
||||
and(
|
||||
eq(chatPublications.companyId, action.companyId),
|
||||
eq(chatPublications.endpointId, action.endpointId),
|
||||
eq(chatPublications.id, token.publicationId as string),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null),
|
||||
db
|
||||
.select()
|
||||
.from(chatEndpoints)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.companyId, action.companyId),
|
||||
eq(chatEndpoints.id, action.endpointId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null),
|
||||
]);
|
||||
if (
|
||||
!conversation ||
|
||||
!publication ||
|
||||
!endpoint ||
|
||||
endpoint.provider !== "telegram" ||
|
||||
publication.conversationId !== conversation.id ||
|
||||
publication.payload.interactionId !== token.interactionId
|
||||
) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "failed",
|
||||
result: { code: "confirmation_action_binding_invalid" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
return false;
|
||||
}
|
||||
const interaction = (
|
||||
await issueThreadInteractionService(db).listForIssue(conversation.issueId)
|
||||
).find((candidate) => candidate.id === token.interactionId);
|
||||
if (
|
||||
!interaction ||
|
||||
interaction.kind !== "request_confirmation" ||
|
||||
!nativeTelegramConfirmation(interaction)
|
||||
) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "failed",
|
||||
result: { code: "confirmation_interaction_invalid" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
interaction.status !== "accepted" &&
|
||||
interaction.status !== "rejected"
|
||||
) {
|
||||
if (interaction.status === "pending") return false;
|
||||
const [retired] = await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "expired",
|
||||
result: {
|
||||
code: "confirmation_interaction_resolved_elsewhere",
|
||||
interactionId: interaction.id,
|
||||
interactionStatus: interaction.status,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatActions.id });
|
||||
if (retired) return true;
|
||||
const winner = await db
|
||||
.select({ status: chatActions.status })
|
||||
.from(chatActions)
|
||||
.where(eq(chatActions.id, action.id))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
return winner?.status === "expired" || winner?.status === "processed";
|
||||
}
|
||||
const decisionMatches =
|
||||
(token.decision === "accept" && interaction.status === "accepted") ||
|
||||
(token.decision === "reject" && interaction.status === "rejected");
|
||||
if (!decisionMatches) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "expired",
|
||||
result: { code: "interaction_resolved_by_sibling_action" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
return false;
|
||||
}
|
||||
const issue = await db
|
||||
.select({
|
||||
id: issues.id,
|
||||
companyId: issues.companyId,
|
||||
assigneeAgentId: issues.assigneeAgentId,
|
||||
status: issues.status,
|
||||
})
|
||||
.from(issues)
|
||||
.where(
|
||||
and(
|
||||
eq(issues.companyId, action.companyId),
|
||||
eq(issues.id, conversation.issueId),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!issue || !interaction.resolvedByUserId) {
|
||||
await db
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "failed",
|
||||
result: { code: "confirmation_resolution_context_missing" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
);
|
||||
return false;
|
||||
}
|
||||
const resolvedByUserId = interaction.resolvedByUserId;
|
||||
const shouldWake =
|
||||
Boolean(issue.assigneeAgentId) &&
|
||||
issue.status !== "done" &&
|
||||
issue.status !== "cancelled" &&
|
||||
(interaction.continuationPolicy === "wake_assignee" ||
|
||||
(interaction.continuationPolicy === "wake_assignee_on_accept" &&
|
||||
interaction.status === "accepted"));
|
||||
const resolutionLabel =
|
||||
interaction.status === "accepted" ? "Accepted" : "Rejected";
|
||||
const resolvedByThisProviderAction =
|
||||
action.result?.code === "confirmation_resolution_committed_by_provider";
|
||||
return db.transaction(async (tx) => {
|
||||
const [completed] = await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "processed",
|
||||
result: {
|
||||
code: resolvedByThisProviderAction
|
||||
? "confirmation_resolution_committed_by_provider"
|
||||
: "confirmation_resolution_reconciled",
|
||||
interactionId: interaction.id,
|
||||
interactionStatus: interaction.status,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.id, action.id),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
)
|
||||
.returning({ id: chatActions.id });
|
||||
if (!completed) {
|
||||
// A publication sweep may reconcile the durable action while the
|
||||
// original provider callback is still in flight. Treat that CAS loss
|
||||
// as success once the winner has processed the same action so the
|
||||
// provider does not receive a spurious 5xx and redeliver it.
|
||||
const winner = await tx
|
||||
.select({ status: chatActions.status })
|
||||
.from(chatActions)
|
||||
.where(eq(chatActions.id, action.id))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
return winner?.status === "processed";
|
||||
}
|
||||
await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "expired",
|
||||
result: { code: "interaction_resolved_by_sibling_action" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.companyId, issue.companyId),
|
||||
eq(chatActions.endpointId, action.endpointId),
|
||||
eq(chatActions.conversationId, conversation.id),
|
||||
eq(chatActions.kind, "confirmation_response"),
|
||||
inArray(chatActions.status, ["issued", "processing"]),
|
||||
ne(chatActions.id, action.id),
|
||||
eq(
|
||||
sql<string>`${chatActions.payload}->>'interactionId'`,
|
||||
interaction.id,
|
||||
),
|
||||
),
|
||||
);
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.insert(chatPublications)
|
||||
.values({
|
||||
companyId: issue.companyId,
|
||||
endpointId: action.endpointId,
|
||||
conversationId: conversation.id,
|
||||
issueId: issue.id,
|
||||
idempotencyKey: `interaction-resolution:${interaction.id}:${action.endpointId}`,
|
||||
payload: projectSafeChatPublication({
|
||||
classification: "external",
|
||||
source: "issue_interaction",
|
||||
text: `${resolutionLabel}: ${interaction.payload.prompt}`,
|
||||
interaction: {
|
||||
id: interaction.id,
|
||||
card: {
|
||||
kind: "confirmation",
|
||||
title: interaction.payload.prompt,
|
||||
body: resolutionLabel,
|
||||
actions: [],
|
||||
},
|
||||
},
|
||||
}),
|
||||
state: "pending",
|
||||
})
|
||||
.onConflictDoNothing();
|
||||
if (shouldWake && issue.assigneeAgentId) {
|
||||
await tx
|
||||
.insert(chatActions)
|
||||
.values({
|
||||
companyId: issue.companyId,
|
||||
endpointId: action.endpointId,
|
||||
conversationId: conversation.id,
|
||||
principalId: action.principalId,
|
||||
kind: "interaction_wakeup",
|
||||
providerActionId: `interaction_wakeup:${interaction.id}`,
|
||||
payload: {
|
||||
version: 1,
|
||||
interactionId: interaction.id,
|
||||
interactionKind: interaction.kind,
|
||||
interactionStatus: interaction.status,
|
||||
issueId: issue.id,
|
||||
agentId: issue.assigneeAgentId,
|
||||
sourceCommentId: interaction.sourceCommentId ?? null,
|
||||
sourceRunId: interaction.sourceRunId ?? null,
|
||||
requestedByUserId: resolvedByUserId,
|
||||
requestedByActorType: "user",
|
||||
requestedByActorId: resolvedByUserId,
|
||||
},
|
||||
status: "issued",
|
||||
})
|
||||
.onConflictDoNothing();
|
||||
}
|
||||
// A provider callback can lose the pending -> terminal interaction race
|
||||
// to the Paperclip UI. In that case this transaction only retires the
|
||||
// now-redundant provider action and must not fabricate a second external
|
||||
// resolution event attributed to the board winner.
|
||||
if (resolvedByThisProviderAction) {
|
||||
await logActivity(tx as unknown as Db, {
|
||||
companyId: issue.companyId,
|
||||
actorType: "user",
|
||||
actorId: resolvedByUserId,
|
||||
action:
|
||||
interaction.status === "accepted"
|
||||
? "issue.thread_interaction_accepted"
|
||||
: "issue.thread_interaction_rejected",
|
||||
entityType: "issue",
|
||||
entityId: issue.id,
|
||||
details: {
|
||||
source: "external_chat",
|
||||
endpointId: action.endpointId,
|
||||
provider: endpoint.provider,
|
||||
conversationId: conversation.id,
|
||||
publicationId: publication.id,
|
||||
providerMessageId: publication.providerMessageId,
|
||||
interactionId: interaction.id,
|
||||
interactionKind: interaction.kind,
|
||||
interactionStatus: interaction.status,
|
||||
resolutionActorKind: "user",
|
||||
requestedResolverPolicy: interaction.requestedResolverPolicy,
|
||||
effectiveResolverPolicy: interaction.effectiveResolverPolicy,
|
||||
},
|
||||
});
|
||||
}
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
async function reconcileTerminalConfirmationActions(
|
||||
limit = 100,
|
||||
): Promise<number> {
|
||||
const candidates = await db
|
||||
.select({ id: chatActions.id })
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.kind, "confirmation_response"),
|
||||
eq(chatActions.status, "processing"),
|
||||
),
|
||||
)
|
||||
.orderBy(asc(chatActions.updatedAt), asc(chatActions.id))
|
||||
.limit(limit);
|
||||
let settled = 0;
|
||||
for (const candidate of candidates) {
|
||||
if (await settleTerminalConfirmationAction(candidate.id)) settled += 1;
|
||||
}
|
||||
return settled;
|
||||
}
|
||||
return { settleTerminalConfirmationAction, reconcileTerminalConfirmationActions, handleTelegramConfirmation };
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import type { Thread } from "chat";
|
||||
import { and, eq, ne, sql } from "drizzle-orm";
|
||||
import { chatEndpointResources } from "@paperclipai/db";
|
||||
import type { EndpointRow, DbTransaction, DbOrTransaction } from "../runtime-types.js";
|
||||
import { nonDirectDestinationAllowed } from "../inbound/surfaces.js";
|
||||
|
||||
type DestinationAdmissionInput = {
|
||||
endpoint: EndpointRow;
|
||||
currentEndpoint: Pick<EndpointRow, "status" | "allowGroupChats">;
|
||||
thread: Thread;
|
||||
addressed: boolean;
|
||||
};
|
||||
type DestinationAdmissionOptions = {
|
||||
ensureResource: (endpoint: EndpointRow, thread: Thread, enableForSetup: boolean, database?: DbOrTransaction) => Promise<typeof chatEndpointResources.$inferSelect>;
|
||||
};
|
||||
export function createTelegramDestinationAdmission(options: DestinationAdmissionOptions) {
|
||||
const { ensureResource } = options;
|
||||
async function admitTelegramDestination(input: DestinationAdmissionInput, tx: DbTransaction) {
|
||||
const { endpoint, currentEndpoint, thread, addressed } = input;
|
||||
let destinationAccepting = true;
|
||||
|
||||
let resource = await ensureResource(endpoint, thread, false, tx);
|
||||
const enabledResourceCount = await tx
|
||||
.select({ count: sql<number>`count(*)::int` })
|
||||
.from(chatEndpointResources)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpointResources.endpointId, endpoint.id),
|
||||
eq(chatEndpointResources.enabled, true),
|
||||
eq(chatEndpointResources.availability, "available"),
|
||||
ne(chatEndpointResources.type, "direct_message"),
|
||||
ne(chatEndpointResources.type, "group_chat"),
|
||||
),
|
||||
)
|
||||
.then((rows) => rows[0]?.count ?? 0);
|
||||
const canActivateFirstSetupGroup =
|
||||
currentEndpoint.status === "verifying" &&
|
||||
resource.availability === "available" &&
|
||||
enabledResourceCount === 0 &&
|
||||
addressed;
|
||||
if (canActivateFirstSetupGroup && !resource.enabled) {
|
||||
resource = await tx
|
||||
.update(chatEndpointResources)
|
||||
.set({ enabled: true, updatedAt: new Date() })
|
||||
.where(eq(chatEndpointResources.id, resource.id))
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? resource);
|
||||
}
|
||||
// Telegram's privacy-mode contract is explicit: in a group or forum
|
||||
// topic, every admitted turn must address the bot. A direct reply to a
|
||||
// bot message is normalized as a mention by the pinned adapter, while
|
||||
// unrelated subscribed traffic remains unaddressed and is filtered.
|
||||
destinationAccepting =
|
||||
addressed &&
|
||||
nonDirectDestinationAllowed(
|
||||
{ ...endpoint, allowGroupChats: currentEndpoint.allowGroupChats },
|
||||
resource,
|
||||
);
|
||||
return destinationAccepting;
|
||||
}
|
||||
return { admitTelegramDestination };
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
import { createChatPublicationReplacement } from "../publication-replacement.js";
|
||||
import { notifyChatPublicationWork } from "../../chat-work-notifications.js";
|
||||
import {
|
||||
eq,
|
||||
sql,
|
||||
} from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
chatPublications,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
SafeChatPublicationPayload,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
splitTelegramPublicationText,
|
||||
telegramMarkdownRequiresAttachment,
|
||||
} from "../../chat-publication-stream.js";
|
||||
type PublicationPreparationOptions = {
|
||||
runPublicationToReplace: ReturnType<typeof createChatPublicationReplacement>["runPublicationToReplace"];
|
||||
};
|
||||
export function createTelegramPublicationPreparation(db: Db, options: PublicationPreparationOptions) {
|
||||
const { runPublicationToReplace } = options;
|
||||
async function prepareTelegramPublication(publication: typeof chatPublications.$inferSelect, persisted: SafeChatPublicationPayload): Promise<typeof chatPublications.$inferSelect> {if (
|
||||
persisted.transportPart ||
|
||||
persisted.card ||
|
||||
persisted.interactionId ||
|
||||
persisted.attachmentIds?.length
|
||||
) {
|
||||
return publication;
|
||||
}
|
||||
if (telegramMarkdownRequiresAttachment(persisted.text)) {
|
||||
// Telegram parses each message independently. A fixed-size split can
|
||||
// turn the second half of a code fence, link, or list into unrelated
|
||||
// plain text even though concatenating the source parts is lossless.
|
||||
// Preserve structured long Markdown as one document. When this is a run
|
||||
// final, first complete its existing progress placeholder, then send the
|
||||
// file through a separate ordered outbox row because Telegram cannot add
|
||||
// an attachment while editing that placeholder.
|
||||
const replacesProgressMessage = Boolean(
|
||||
await runPublicationToReplace(publication, persisted),
|
||||
);
|
||||
return db.transaction(async (tx) => {
|
||||
const current = await tx
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(eq(chatPublications.id, publication.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) return publication;
|
||||
const currentPayload = current.payload as SafeChatPublicationPayload;
|
||||
if (
|
||||
currentPayload.transportPart ||
|
||||
currentPayload.card ||
|
||||
currentPayload.interactionId ||
|
||||
currentPayload.attachmentIds?.length ||
|
||||
!["pending", "retry"].includes(current.state) ||
|
||||
!telegramMarkdownRequiresAttachment(currentPayload.text)
|
||||
) {
|
||||
return current;
|
||||
}
|
||||
|
||||
const batchId = current.id;
|
||||
const updatedAt = new Date();
|
||||
const transportPart = (
|
||||
index: number,
|
||||
count: number,
|
||||
mode: "inline" | "telegram_markdown_attachment",
|
||||
) => ({
|
||||
batchId,
|
||||
count,
|
||||
index,
|
||||
mode,
|
||||
orderKey: `${batchId}:${String(index).padStart(4, "0")}`,
|
||||
});
|
||||
if (!replacesProgressMessage) {
|
||||
const attachmentPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
transportPart: transportPart(0, 1, "telegram_markdown_attachment"),
|
||||
};
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: attachmentPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
return { ...current, payload: attachmentPayload, updatedAt };
|
||||
}
|
||||
|
||||
const handoffPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
text: "This response needs a separate attachment because it exceeds the message limit.",
|
||||
transportPart: transportPart(0, 2, "inline"),
|
||||
};
|
||||
delete handoffPayload.attachmentIds;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: handoffPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
|
||||
const attachmentPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
transportPart: transportPart(1, 2, "telegram_markdown_attachment"),
|
||||
};
|
||||
delete attachmentPayload.progressState;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx.insert(chatPublications).values({
|
||||
companyId: current.companyId,
|
||||
endpointId: current.endpointId,
|
||||
conversationId: current.conversationId,
|
||||
issueId: current.issueId,
|
||||
commentId: current.commentId,
|
||||
idempotencyKey: `telegram-markdown-attachment:${current.id}`,
|
||||
payload: attachmentPayload,
|
||||
state: "pending",
|
||||
createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`,
|
||||
updatedAt,
|
||||
});
|
||||
return { ...current, payload: handoffPayload, updatedAt };
|
||||
});
|
||||
}
|
||||
const parts = splitTelegramPublicationText(persisted.text);
|
||||
if (parts.length === 1) return publication;
|
||||
|
||||
return db.transaction(async (tx) => {
|
||||
const current = await tx
|
||||
.select()
|
||||
.from(chatPublications)
|
||||
.where(eq(chatPublications.id, publication.id))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) return publication;
|
||||
const currentPayload = current.payload as SafeChatPublicationPayload;
|
||||
if (
|
||||
currentPayload.transportPart ||
|
||||
currentPayload.card ||
|
||||
currentPayload.interactionId ||
|
||||
currentPayload.attachmentIds?.length ||
|
||||
!["pending", "retry"].includes(current.state)
|
||||
) {
|
||||
return current;
|
||||
}
|
||||
const currentParts = splitTelegramPublicationText(currentPayload.text);
|
||||
if (currentParts.length === 1) return current;
|
||||
|
||||
const batchId = current.id;
|
||||
const updatedAt = new Date();
|
||||
const transportPart = (index: number) => ({
|
||||
batchId,
|
||||
count: currentParts.length,
|
||||
index,
|
||||
orderKey: `${batchId}:${String(index).padStart(4, "0")}`,
|
||||
});
|
||||
const rootPayload: SafeChatPublicationPayload = {
|
||||
...currentPayload,
|
||||
text: currentParts[0]!,
|
||||
transportPart: transportPart(0),
|
||||
};
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx
|
||||
.update(chatPublications)
|
||||
.set({ payload: rootPayload, updatedAt })
|
||||
.where(eq(chatPublications.id, current.id));
|
||||
|
||||
const { progressState: _progressState, ...nonProgressPayload } =
|
||||
currentPayload;
|
||||
await notifyChatPublicationWork(tx);
|
||||
await tx.insert(chatPublications).values(
|
||||
currentParts.slice(1).map((text, offset) => {
|
||||
const index = offset + 1;
|
||||
return {
|
||||
companyId: current.companyId,
|
||||
endpointId: current.endpointId,
|
||||
conversationId: current.conversationId,
|
||||
issueId: current.issueId,
|
||||
commentId: current.commentId,
|
||||
idempotencyKey: `${current.idempotencyKey}:telegram-part:${index}`,
|
||||
payload: {
|
||||
...nonProgressPayload,
|
||||
text,
|
||||
transportPart: transportPart(index),
|
||||
},
|
||||
state: "pending" as const,
|
||||
createdAt: sql`(select ${chatPublications.createdAt} from ${chatPublications} where ${chatPublications.id} = ${current.id})`,
|
||||
updatedAt,
|
||||
};
|
||||
}),
|
||||
);
|
||||
return { ...current, payload: rootPayload, updatedAt };
|
||||
});}
|
||||
return { prepareTelegramPublication };
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
import { createTelegramMaintenance } from "./maintenance.js";
|
||||
import type { EndpointRow, RuntimeContext } from "../runtime-types.js";
|
||||
|
||||
import { type CredentialMutationLeaseGuard } from "../../chat-credential-mutation-lease.js";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
|
||||
import type {
|
||||
ConfigureChatEndpointInput,
|
||||
} from "@paperclipai/shared";
|
||||
|
||||
import {
|
||||
unprocessable,
|
||||
} from "../../../errors.js";
|
||||
|
||||
import type { EndpointRecord } from "../runtime-types.js";
|
||||
|
||||
type TelegramSetupOptions = {
|
||||
fetchImpl: typeof globalThis.fetch;
|
||||
getWebhookPublicBaseUrl: () => string | null;
|
||||
PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS: number;
|
||||
runtimeContextForRecord: (record: EndpointRecord) => RuntimeContext;
|
||||
recordTelegramStopSubscription: ReturnType<typeof createTelegramMaintenance>["recordTelegramStopSubscription"];
|
||||
stageTelegramMaintenance: ReturnType<typeof createTelegramMaintenance>["stageTelegramMaintenance"];
|
||||
processTelegramMaintenance: ReturnType<typeof createTelegramMaintenance>["processTelegramMaintenance"];
|
||||
};
|
||||
export function createTelegramSetup(db: Db, options: TelegramSetupOptions) {
|
||||
const { fetchImpl, getWebhookPublicBaseUrl, PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS, runtimeContextForRecord, recordTelegramStopSubscription, stageTelegramMaintenance, processTelegramMaintenance } = options;
|
||||
|
||||
async function registerTelegramWebhook(endpoint: EndpointRow, input: ConfigureChatEndpointInput, credentials: Record<string, string>, credentialLease: CredentialMutationLeaseGuard, next: EndpointRecord) {
|
||||
const webhookUrl = `${getWebhookPublicBaseUrl()}/api/chat-webhooks/${endpoint.publicId}/telegram`;
|
||||
const infoResponse = await fetchImpl(
|
||||
`https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/getWebhookInfo`,
|
||||
{ signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS) },
|
||||
);
|
||||
const infoResult = (await infoResponse.json()) as {
|
||||
ok?: boolean;
|
||||
description?: string;
|
||||
result?: { url?: string };
|
||||
};
|
||||
if (!infoResponse.ok || !infoResult.ok) {
|
||||
throw unprocessable(
|
||||
`Telegram could not inspect the existing webhook: ${infoResult.description ?? infoResponse.status}`,
|
||||
);
|
||||
}
|
||||
await credentialLease.assertOwned();
|
||||
const response = await fetchImpl(
|
||||
`https://api.telegram.org/bot${encodeURIComponent(credentials.botToken)}/setWebhook`,
|
||||
{
|
||||
method: "POST",
|
||||
signal: AbortSignal.timeout(PROVIDER_CREDENTIAL_CHECK_TIMEOUT_MS),
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
url: webhookUrl,
|
||||
secret_token: credentials.webhookSecret,
|
||||
allowed_updates: [
|
||||
"message",
|
||||
"edited_message",
|
||||
"callback_query",
|
||||
"stopped_message_generation",
|
||||
"message_reaction",
|
||||
"my_chat_member",
|
||||
],
|
||||
// A first-time connection may inherit stale updates from a bot
|
||||
// that was used before Paperclip owned it. Reconnect is
|
||||
// different: Telegram may be holding legitimate updates while
|
||||
// the old callback URL is unavailable. Preserve that backlog
|
||||
// even when an operator is rotating the public origin.
|
||||
drop_pending_updates: input.action === "configure",
|
||||
}),
|
||||
},
|
||||
);
|
||||
const result = (await response.json()) as {
|
||||
ok?: boolean;
|
||||
description?: string;
|
||||
result?: unknown;
|
||||
};
|
||||
if (!response.ok || !result.ok)
|
||||
throw unprocessable(
|
||||
`Telegram could not register the webhook: ${result.description ?? response.status}`,
|
||||
);
|
||||
await credentialLease.assertOwned();
|
||||
if (result.ok === true && result.result === true) {
|
||||
// Telegram's successful boolean receipt acknowledges the exact URL,
|
||||
// secret and allowed_updates request above. GET alone cannot prove
|
||||
// secret_token, which Telegram deliberately does not return.
|
||||
await recordTelegramStopSubscription(
|
||||
next.endpoint,
|
||||
runtimeContextForRecord(next),
|
||||
credentialLease,
|
||||
);
|
||||
}
|
||||
// Command discovery is optional for ingress, but its provider mutation
|
||||
// is still a durable, auditable action. A transient or rate-limited
|
||||
// failure is retried by the maintenance worker without blocking the
|
||||
// otherwise-valid webhook connection.
|
||||
const commandAction = await stageTelegramMaintenance(
|
||||
endpoint,
|
||||
"register_commands",
|
||||
);
|
||||
if (commandAction)
|
||||
await processTelegramMaintenance(
|
||||
commandAction.id,
|
||||
false,
|
||||
credentialLease,
|
||||
);
|
||||
}
|
||||
|
||||
return { registerTelegramWebhook };
|
||||
}
|
||||
Reference in new issue
Block a user