mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-29 04:25:52 +01:00
fix(server): schedule Discord command registration from saved deadlines (#17557)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat connections register and verify provider commands in the background. > - Discord command recovery ran inside the recurring Gateway runtime scan. > - Its existing durable records already store verification and retry deadlines. > - This pull request gives command registration its own shared-coordinator worker and scoped commit notifications. > - Idle command work stops querying between real deadlines. Existing command authority and recovery remain intact. ## Linked Issues or Issue Description Refs #17554 and #17561. Both foundations have merged. This slice targets master. Runtime discovery and the other chat maintenance duties remain separate work. **What happened?** The chat runtime maintenance loop called Discord command registration on every reconciliation. It repeatedly read durable command state even when the next remote verification or retry was minutes away. **Expected behavior** Use committed producer changes and saved deadlines. Keep the existing five-minute verification policy, Retry-After, uncertain-write quarantine, namespace ownership, and credential fences. Do not add a database table for the timer. **Steps to reproduce** Configure a Discord connection and let command registration settle. In the parent revision, repeated runtime reconciliation still invokes command recovery. This revision reconstructs command deadlines at startup and removes that call from runtime reconciliation. ## What Changed - Expose normalized deadlines from the existing registration store. Park indefinite retries and invalid or foreign-owned records. Signal before store-owned claims, receipts, and retry-result writes. - Discover eligible endpoints in bounded pages. Refresh only affected endpoint, connection, or company scopes after commits. Dispatch at most 25 due commands per pass with fair company rotation. - Wire configuration, credentials, provider lifecycle/failure paths, inbound activation, generic connection mutation/removal/health, access revocation, and company deletion. - Register the worker through the chat service and shared coordinator. Preserve direct foreground setup calls. Refresh an existing local Gateway through its current owner after command work. - Preserve dirty work during drain and concurrent changes, back off unresolved attempts, and join admitted provider work during shutdown. - Document producer coverage, retained deadlines, supported writers, verification failures, and remaining runtime/Cloud obligations in `doc/idle-discord-command-registration.md` and its dated audit. ## Verification Published head: `547fdf2139144e1e6484198278bf4bb0eade8794`. It normally integrates master at `f8563d25077b17681ab6ef5949f7b07415a58ede`, including landed health #17554 and company deletion #17561. Squash ancestry caused merge conflicts; resolution applied this slice's exact 26-file owned patch from its recorded parent to pinned master. Every other path matches master. Migration 0394 and the current lockfile are preserved. - **141 distinct targeted cases pass** on reconciled production source: 78 complete command-store, batching, coordinator and startup-timing cases; 8 real automatic Discord and held-transaction Slack cases; and 55 complete startup/export and company-deletion cases. The 1,581 other chat integration cases were deselected. - Server typecheck and build pass. The company-deletion fixture self-migrates an initially empty isolated database. The two Discord fixtures use separate migrated clones; this is not a fresh-schema claim for those fixtures. Signal-listener warnings remain recorded without assertion failures. - The parent integration preserves health and command notifications at shared producer sites. It changes no provider behavior, schema or coordinator contract. Prior producer, backlog, drain/resume, malformed-hint, transaction, namespace, shutdown and retry regressions remain in the complete command suite. - [Landing compatibility acceptance](https://github.com/paperclipai/paperclip/blob/547fdf2139144e1e6484198278bf4bb0eade8794/doc/idle-discord-command-registration.md#acceptance-after-the-company-deletion-parent-landed-october-27) records current scope and the earlier failures. Older full-matrix or combined-stack runs apply only to their stated revisions. - Fresh exact-head Greptile, Ponytail if present and all CI are required. No full local repository suite, live provider, Cloud alarm, host suspension or savings claim is made. ## Risks - Discord still has real five-minute verification and failure-retry obligations. This does not prove host sleep or external alarm coverage. - Runtime discovery, receipt-to-runtime projection, Gateway lease renewal and persistent sockets remain separate. The remaining runtime poll can still read command receipts; it no longer invokes the command protocol. - Process-local signals cover integrated writers. Independent SQL writers and other replicas require an explicit wake contract. Existing authorization and durable ownership checks remain authoritative. - Refreshes coalesce with a 30-second minimum spacing. Saved work is never discarded because a call returned without progress. - No schema migration, new table, workflow or lockfile change is introduced by this slice. Landed parent repairs are retained. The receiver child must receive this integration before landing. ## Model Used OpenAI Codex, GPT-6 family, with repository tools, code execution, and reasoning. The exact serving version and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
4c314117d2
commit
5bf0142d45
26 files changed
+1242
-32
No files matched your search
@@ -1,3 +1,4 @@
|
||||
import { deleteCompany } from "../services/company-deletion.js";
|
||||
import { adapterExecutionControls, createAdapterExecutionControl } from "../services/adapter-execution-control.js";
|
||||
import { chatRunSourceMessageId } from "../services/chat-channels/run-origin.js";
|
||||
import * as durableChatWakeup from "../services/durable-chat-wakeup.js";
|
||||
@@ -798,11 +799,13 @@ class FakeChatSdkRuntime {
|
||||
|
||||
const TEST_SLACK_BOT_SCOPES =
|
||||
"app_mentions:read,assistant:write,channels:history,channels:read,chat:write,commands,files:read,files:write,groups:history,groups:read,im:history,im:read,mpim:history,mpim:read,reactions:read,reactions:write,users:read";
|
||||
const discordApplicationPrefix = 1_400_000_000_000_000_000n +
|
||||
BigInt(`0x${randomUUID().replaceAll("-", "").slice(0, 12)}`) * 1000n;
|
||||
let discordApplicationSequence = 0n;
|
||||
|
||||
function uniqueDiscordApplicationId() {
|
||||
discordApplicationSequence += 1n;
|
||||
return (123_456_789_012_345_678n + discordApplicationSequence).toString();
|
||||
return (discordApplicationPrefix + discordApplicationSequence).toString();
|
||||
}
|
||||
|
||||
function fakeSlackFetch(botId = `U-BOT-${randomUUID()}`) {
|
||||
@@ -7090,6 +7093,17 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
return { ...context, endpoint, callbacks };
|
||||
}
|
||||
|
||||
async function recoverDiscordCommands(service: ChatChannelService) {
|
||||
const onError = vi.fn();
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError });
|
||||
try {
|
||||
await service.registerCommandWork(coordinator, () => true).ready;
|
||||
expect(onError).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
await coordinator.stop();
|
||||
}
|
||||
}
|
||||
|
||||
describe("Discord automatic command registration", () => {
|
||||
async function retireRegistrationFixture(
|
||||
service: ChatChannelService,
|
||||
@@ -7100,7 +7114,9 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
.select()
|
||||
.from(chatDiscordCommandOwners)
|
||||
.where(eq(chatDiscordCommandOwners.endpointId, endpointId));
|
||||
await service.configure(endpointId, { action: "remove" }, "owner-user");
|
||||
if ((await service.get(endpointId))?.status !== "archived") {
|
||||
await service.configure(endpointId, { action: "remove" }, "owner-user");
|
||||
}
|
||||
expect((await service.get(endpointId))?.status).toBe("archived");
|
||||
// Removal retires only this fixture's runtime; immutable application
|
||||
// ownership stays available to the no-adoption/tombstone checks.
|
||||
@@ -7227,6 +7243,98 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
};
|
||||
}
|
||||
|
||||
it("learns the first configured command and cancels its deadline when the endpoint is removed", async () => {
|
||||
const f = await registrationFixture();
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const onError = vi.fn();
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError });
|
||||
try {
|
||||
await f.service.registerCommandWork(coordinator, () => true).ready;
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
await f.configure();
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(() => expect(coordinator.nextWakeAt()).toBeGreaterThan(Date.now()));
|
||||
const deadline = coordinator.nextWakeAt();
|
||||
expect(deadline).toBeGreaterThan(Date.now() + 200_000);
|
||||
await f.service.configure(f.endpoint.id, { action: "remove" }, "owner-user");
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
});
|
||||
const reads = vi.spyOn(db, "select"), writes = vi.spyOn(db, "transaction");
|
||||
try {
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(reads).not.toHaveBeenCalled(); expect(writes).not.toHaveBeenCalled();
|
||||
} finally { reads.mockRestore(); writes.mockRestore(); }
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
expect(onError).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
await coordinator.stop(); vi.useRealTimers();
|
||||
await retireRegistrationFixture(f.service, f.endpoint.id);
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["credential rotation", "company deletion"] as const)("updates scheduled registration after real %s", async (mutation) => {
|
||||
const f = await registrationFixture();
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const onError = vi.fn();
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError });
|
||||
let deleted = false;
|
||||
let lifecycle: ReturnType<typeof configureAgentLifecycle> | undefined;
|
||||
try {
|
||||
await f.service.registerCommandWork(coordinator, () => true).ready;
|
||||
await f.configure();
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(() => expect(coordinator.nextWakeAt()).toBeGreaterThan(Date.now()));
|
||||
const before = coordinator.nextWakeAt()!;
|
||||
const owners = await db.select().from(chatDiscordCommandOwners)
|
||||
.where(eq(chatDiscordCommandOwners.endpointId, f.endpoint.id));
|
||||
if (mutation === "credential rotation") {
|
||||
await f.service.configure(f.endpoint.id, {
|
||||
action: "reconnect", credentials: { ...f.credentials, botToken: "discord-rotated-secret" },
|
||||
}, "owner-user");
|
||||
const [saved] = await db.select().from(chatActions).where(and(
|
||||
eq(chatActions.endpointId, f.endpoint.id), eq(chatActions.kind, "discord_command_registration"),
|
||||
));
|
||||
const expected = Date.parse(String(saved.result!.retryAt));
|
||||
expect(expected).toBeGreaterThan(before);
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(() => expect(coordinator.nextWakeAt()).toBe(expected));
|
||||
expect(f.calls).toEqual(["GET", "POST", "GET"]);
|
||||
} else {
|
||||
lifecycle = configureAgentLifecycle(db, {
|
||||
requiredPluginIds: async () => [], runHost: async () => "complete", runPlugin: async () => "complete",
|
||||
});
|
||||
await deleteCompany(db, f.fixture.companyId);
|
||||
deleted = true;
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
});
|
||||
expect(await db.select().from(chatDiscordCommandOwners)
|
||||
.where(eq(chatDiscordCommandOwners.endpointId, f.endpoint.id))).toEqual(owners);
|
||||
await f.service.shutdown();
|
||||
const reads = vi.spyOn(db, "select"), writes = vi.spyOn(db, "transaction");
|
||||
try {
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(reads).not.toHaveBeenCalled(); expect(writes).not.toHaveBeenCalled();
|
||||
} finally { reads.mockRestore(); writes.mockRestore(); }
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
}
|
||||
expect(onError).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
await lifecycle?.stop();
|
||||
await coordinator.stop(); vi.useRealTimers();
|
||||
if (deleted) await f.service.shutdown();
|
||||
else await retireRegistrationFixture(f.service, f.endpoint.id);
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps mention setup usable on registration failure and upgrades automatically when due", async () => {
|
||||
const f = await registrationFixture("unavailable");
|
||||
try {
|
||||
@@ -7240,7 +7348,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
).toBeTypeOf("function");
|
||||
f.available();
|
||||
await f.makeDue();
|
||||
await f.service.reconcileProviderRuntimes();
|
||||
await recoverDiscordCommands(f.service);
|
||||
expect(
|
||||
(await f.service.get(f.endpoint.id))?.capabilities.slashCommands,
|
||||
).toBe(true);
|
||||
@@ -7268,6 +7376,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
await f.makeDue();
|
||||
restarted = createService(new FakeChatSdkRuntime(), f.providerFetch);
|
||||
await restarted.service.reconcileProviderRuntimes();
|
||||
await recoverDiscordCommands(restarted.service);
|
||||
expect(
|
||||
(await restarted.service.get(f.endpoint.id))?.capabilities
|
||||
.slashCommands,
|
||||
@@ -7342,7 +7451,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
};
|
||||
f.available(false);
|
||||
await f.makeDue();
|
||||
await f.service.reconcileProviderRuntimes();
|
||||
await recoverDiscordCommands(f.service);
|
||||
expect(f.runtime.endpoints.get(f.endpoint.id)).toBe(original);
|
||||
expect(f.runtime.configurations.get(f.endpoint.id)?.callbacks).toBe(
|
||||
originalCallbacks,
|
||||
@@ -7383,7 +7492,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
),
|
||||
);
|
||||
await f.makeDue();
|
||||
await f.service.reconcileProviderRuntimes();
|
||||
await recoverDiscordCommands(f.service);
|
||||
expect(
|
||||
(await f.service.get(f.endpoint.id))?.capabilities.slashCommands,
|
||||
).toBe(false);
|
||||
@@ -51809,6 +51918,7 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
});
|
||||
coldContexts.push(cold);
|
||||
await cold.service.reconcileProviderRuntimes();
|
||||
await recoverDiscordCommands(cold.service);
|
||||
return {
|
||||
...cold,
|
||||
adapter: await parser(
|
||||
|
||||
@@ -1298,6 +1298,7 @@ export async function createApp(
|
||||
}
|
||||
registerChatDeliveryWork(deliveryWork, chatChannels, () => !isIdleTaskDrainActive());
|
||||
registerChatActionWork(deliveryWork, chatChannels, () => !isIdleTaskDrainActive());
|
||||
chatChannels.registerCommandWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
emailChannels.start();
|
||||
const reconcileChatPublicationMaintenance = async () => {
|
||||
await chatChannels.processPublicationMaintenance();
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatEndpointWork } from "./chat-work-notifications.js";
|
||||
import { notifyToolConnectionHealth } from "./tool-connection-health-notifications.js";
|
||||
import { and, eq, inArray, ne, notInArray, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
@@ -130,6 +131,7 @@ export function accessService(db: Db) {
|
||||
}> = [];
|
||||
let connectionRefs: Array<{
|
||||
id: string;
|
||||
transport: typeof toolConnections.$inferSelect.transport;
|
||||
credentialRefs: typeof toolConnections.$inferSelect.credentialRefs;
|
||||
credentialSecretRefs: typeof toolConnections.$inferSelect.credentialSecretRefs;
|
||||
}> = [];
|
||||
@@ -168,6 +170,7 @@ export function accessService(db: Db) {
|
||||
}).from(connectionGrants).where(eq(connectionGrants.companyId, companyId)),
|
||||
tx.select({
|
||||
id: toolConnections.id,
|
||||
transport: toolConnections.transport,
|
||||
credentialRefs: toolConnections.credentialRefs,
|
||||
credentialSecretRefs: toolConnections.credentialSecretRefs,
|
||||
}).from(toolConnections).where(eq(toolConnections.companyId, companyId)),
|
||||
@@ -324,6 +327,7 @@ export function accessService(db: Db) {
|
||||
&& grant.credentialSecretRefs.length > 0
|
||||
&& grant.credentialSecretRefs.every((ref) => !connectionSecretIdsToRemove.has(ref.secretId)),
|
||||
);
|
||||
if (connection.transport === "chat_sdk") await notifyChatEndpointWork(tx, connection.id);
|
||||
if (!hasUnaffectedActiveGrant) await notifyToolConnectionHealth(tx, { companyId, id: connection.id });
|
||||
await tx.update(toolConnections).set({
|
||||
credentialRefs,
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
import { notifyChatCommandRegistration } from "./chat-channels/command-work-notifications.js";
|
||||
import { registerDiscordCommandWork } from "./chat-channels/discord/command-work.js";
|
||||
import { discordCommandSchedule } from "./chat-channels/discord/command-schedule.js";
|
||||
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
|
||||
import { notifyChatConnectionHealth } from "./chat-channels/connection-health-notifications.js";
|
||||
import { credentialFingerprint } from "./chat-channels/credential-fingerprint.js";
|
||||
import { createChatSdkMessageReceiver } from "./chat-channels/runtime/sdk-messages.js";
|
||||
@@ -884,7 +888,15 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
|
||||
const { authenticateGitHubWebhook } = createGitHubWebhookAuthentication(db, { endpointRecord, runtimeContextForRecord, withCredentialMutationLease, resolveCredentials, reconcileGitHubWebhookRepository, stageGitHubWebhookIngress, processGitHubWebhookIngress, githubIngressAcceptedResponse, scheduleMessageProcessing, backgroundMessageTasks, redactError, get githubWebhookAuthenticationBarrier() { return options.githubWebhookAuthenticationBarrier; }, get deferWebhookProcessing() { return options.deferWebhookProcessing; } });
|
||||
const { inspectSlackWebhookScope, recordSlackWebhookCallback } = createSlackWebhookCallbacks(db, { resolveCredentials, runtimeCallbackEndpoint, getWebhookPublicBaseUrl, scheduleMessageProcessing, slackRegistration: { notifyVerified: (endpointId) => slackRegistration.notifyVerified(endpointId) } });
|
||||
|
||||
const { reconcileDiscordRuntime } = createDiscordRuntimeReconciliation(db, { reconcileDiscordCommands, endpointRecord, runtimeContexts, invalidateRuntime, runtimeFor, runtime, runtimeCallbackEndpoint, credentialFingerprint, discordGatewayRuntimeIsCurrent });
|
||||
const { reconcileDiscordRuntime } = createDiscordRuntimeReconciliation(db, { endpointRecord, runtimeContexts, invalidateRuntime, runtimeFor, runtime, runtimeCallbackEndpoint, credentialFingerprint, discordGatewayRuntimeIsCurrent });
|
||||
|
||||
function registerCommandWork(coordinator: ReturnType<typeof createDeliveryWorkCoordinator>, canRun: () => boolean) {
|
||||
return registerDiscordCommandWork(coordinator, discordCommandSchedule(db, async endpointId => {
|
||||
await reconcileDiscordCommands(endpointId);
|
||||
const instance = runtime.get(endpointId);
|
||||
if (instance) await reconcileDiscordRuntime(endpointId, instance);
|
||||
}), () => !shuttingDown && canRun());
|
||||
}
|
||||
|
||||
async function runtimeFor(
|
||||
endpoint: EndpointRow,
|
||||
@@ -2199,6 +2211,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
|
||||
? "Started a conversation by mentioning the agent."
|
||||
: "Sent an empty message.");
|
||||
let comment!: Awaited<ReturnType<typeof issuesSvc.addComment>>;
|
||||
await notifyChatCommandRegistration(taskTx, taskEndpoint);
|
||||
await taskTx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
@@ -3458,6 +3471,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
|
||||
const { prepareTeamsBoardSend } = createTeamsBoardSendPreparation(db, { endpointRecord, runtimeFor });
|
||||
const { getPublicationBatchStatus, publishComment, publishBoardMessage } = createChatPublicationRequests(db, { endpointRecord, stageTeamsBoardFileIntent, processPendingSlackBoardMessages, processPendingPublications, prepareTeamsBoardSend });
|
||||
return {
|
||||
registerCommandWork,
|
||||
voice,
|
||||
authorizeFastResponse,
|
||||
slackRegistration,
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { ChatProvider, ToolConnectionTransport } from "@paperclipai/shared";
|
||||
import { notifyDiscordCommandRegistration } from "./discord/command-work-notifications.js";
|
||||
|
||||
export async function notifyChatCommandRegistration(transaction: object, endpoint: { id: string; provider: ChatProvider }) {
|
||||
if (endpoint.provider === "discord") {
|
||||
await notifyDiscordCommandRegistration(transaction, { kind: "endpoint", id: endpoint.id });
|
||||
}
|
||||
}
|
||||
|
||||
export async function notifyChatConnectionCommandRegistration(transaction: object, connectionId: string, transport: ToolConnectionTransport = "chat_sdk") {
|
||||
if (transport === "chat_sdk") {
|
||||
await notifyDiscordCommandRegistration(transaction, { kind: "connection", id: connectionId });
|
||||
}
|
||||
}
|
||||
|
||||
export async function notifyChatCompanyCommandRegistration(transaction: object, companyId: string) {
|
||||
await notifyDiscordCommandRegistration(transaction, { kind: "company", id: companyId });
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCommandRegistration } from "./command-work-notifications.js";
|
||||
import { prepareSlackInventory } from "./slack/inventory.js";
|
||||
import { prepareGitHubInventory } from "./github/inventory.js";
|
||||
import { prepareDiscordInventory } from "./discord/inventory.js";
|
||||
@@ -351,6 +352,7 @@ export function createChatCredentials(db: Db, options: ChatCredentialsOptions) {
|
||||
code: "chat_endpoint_credentials_changed",
|
||||
});
|
||||
}
|
||||
await notifyChatCommandRegistration(tx, endpoint);
|
||||
await tx
|
||||
.delete(companySecretBindings)
|
||||
.where(
|
||||
@@ -462,6 +464,7 @@ export function createChatCredentials(db: Db, options: ChatCredentialsOptions) {
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!current) throw notFound("Chat connection not found");
|
||||
refs = current.refs;
|
||||
await notifyChatCommandRegistration(tx, endpoint);
|
||||
await tx
|
||||
.delete(companySecretBindings)
|
||||
.where(
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import { and, eq, gt, inArray, isNotNull, ne, or } from "drizzle-orm";
|
||||
import { chatEndpoints, toolConnections, type Db } from "@paperclipai/db";
|
||||
import { readDiscordCommandRegistrationDeadline } from "../../chat-discord-command-registration-store.js";
|
||||
import type { CommandRegistrationScope } from "./command-work-notifications.js";
|
||||
|
||||
export type CommandRegistrationDeadline = {
|
||||
id: string;
|
||||
companyId: string;
|
||||
connectionId: string;
|
||||
dueAt: number;
|
||||
};
|
||||
|
||||
export function discordCommandSchedule(db: Db, reconcile: (endpointId: string) => Promise<void>) {
|
||||
async function load(scope?: CommandRegistrationScope, signal?: AbortSignal): Promise<CommandRegistrationDeadline[]> {
|
||||
const deadlines: CommandRegistrationDeadline[] = [];
|
||||
let cursor: string | undefined;
|
||||
while (true) {
|
||||
signal?.throwIfAborted();
|
||||
const rows = await db.select({
|
||||
id: chatEndpoints.id,
|
||||
companyId: chatEndpoints.companyId,
|
||||
connectionId: chatEndpoints.connectionId,
|
||||
applicationId: chatEndpoints.botExternalId,
|
||||
guildId: chatEndpoints.providerAccountId,
|
||||
}).from(chatEndpoints).innerJoin(toolConnections, and(
|
||||
eq(toolConnections.id, chatEndpoints.connectionId),
|
||||
eq(toolConnections.companyId, chatEndpoints.companyId),
|
||||
)).where(and(
|
||||
eq(chatEndpoints.provider, "discord"),
|
||||
inArray(chatEndpoints.status, ["verifying", "active", "attention"]),
|
||||
eq(toolConnections.status, "active"), eq(toolConnections.enabled, true),
|
||||
or(ne(chatEndpoints.status, "attention"), ne(toolConnections.healthStatus, "failed")),
|
||||
isNotNull(chatEndpoints.botExternalId), isNotNull(chatEndpoints.providerAccountId),
|
||||
scope?.kind === "endpoint" ? eq(chatEndpoints.id, scope.id) : undefined,
|
||||
scope?.kind === "connection" ? eq(chatEndpoints.connectionId, scope.id) : undefined,
|
||||
scope?.kind === "company" ? eq(chatEndpoints.companyId, scope.id) : undefined,
|
||||
cursor ? gt(chatEndpoints.id, cursor) : undefined,
|
||||
)).orderBy(chatEndpoints.id).limit(100);
|
||||
for (const row of rows) {
|
||||
signal?.throwIfAborted();
|
||||
if (!row.applicationId || !row.guildId) continue;
|
||||
const dueAt = await readDiscordCommandRegistrationDeadline(db, {
|
||||
companyId: row.companyId, endpointId: row.id,
|
||||
applicationId: row.applicationId, guildId: row.guildId,
|
||||
});
|
||||
if (dueAt !== null) deadlines.push({ id: row.id, companyId: row.companyId, connectionId: row.connectionId, dueAt });
|
||||
}
|
||||
if (rows.length < 100) return deadlines;
|
||||
cursor = rows[rows.length - 1].id;
|
||||
}
|
||||
}
|
||||
async function check(id: string, signal?: AbortSignal): Promise<CommandRegistrationDeadline | null> {
|
||||
const scope = { kind: "endpoint", id } as const;
|
||||
const [before] = await load(scope, signal);
|
||||
if (!before || before.dueAt > Date.now()) return before ?? null;
|
||||
signal?.throwIfAborted();
|
||||
await reconcile(id);
|
||||
const [after] = await load(scope, signal);
|
||||
return after ?? null;
|
||||
}
|
||||
return { load, check };
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../delivery-work-notifications.js";
|
||||
|
||||
export type CommandRegistrationScope =
|
||||
| { kind: "endpoint"; id: string }
|
||||
| { kind: "connection"; id: string }
|
||||
| { kind: "company"; id: string };
|
||||
|
||||
export async function notifyDiscordCommandRegistration(transaction: object, scope: CommandRegistrationScope) {
|
||||
await notifyDeliveryWork(transaction, DELIVERY_QUEUES.chatCommandRegistration, `${scope.kind}:${scope.id}`);
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
import { takeCompanyWorkBatch } from "../../company-work-batch.js";
|
||||
import type { createDeliveryWorkCoordinator } from "../../delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES } from "../../delivery-work-notifications.js";
|
||||
import type { CommandRegistrationDeadline, discordCommandSchedule } from "./command-schedule.js";
|
||||
import type { CommandRegistrationScope } from "./command-work-notifications.js";
|
||||
|
||||
function decodeScope(hint: string): CommandRegistrationScope | undefined {
|
||||
if (hint === "*") return undefined;
|
||||
const separator = hint.indexOf(":");
|
||||
const kind = hint.slice(0, separator), id = hint.slice(separator + 1);
|
||||
if (kind !== "endpoint" && kind !== "connection" && kind !== "company")
|
||||
throw new Error("Unknown command registration scope");
|
||||
return { kind, id };
|
||||
}
|
||||
|
||||
function matches(entry: CommandRegistrationDeadline, scope?: CommandRegistrationScope) {
|
||||
if (!scope) return true;
|
||||
return (scope.kind === "endpoint" ? entry.id : scope.kind === "connection" ? entry.connectionId : entry.companyId) === scope.id;
|
||||
}
|
||||
|
||||
export function registerDiscordCommandWork(
|
||||
coordinator: ReturnType<typeof createDeliveryWorkCoordinator>,
|
||||
service: ReturnType<typeof discordCommandSchedule>,
|
||||
canRun: () => boolean,
|
||||
) {
|
||||
const dirty = new Set<string>(["*"]);
|
||||
const deadlines = new Map<string, CommandRegistrationDeadline>();
|
||||
const failures = new Map<string, { count: number; retryAt: number }>();
|
||||
const retryMs = 30_000;
|
||||
let earliest = 0;
|
||||
let lastCompany: string | null = null;
|
||||
function next() {
|
||||
let at: number | null = dirty.size ? earliest : null;
|
||||
for (const entry of deadlines.values()) {
|
||||
const due = Math.max(earliest, entry.dueAt, failures.get(entry.id)?.retryAt ?? 0);
|
||||
at = at === null ? due : Math.min(at, due);
|
||||
}
|
||||
return at;
|
||||
}
|
||||
return coordinator.register(DELIVERY_QUEUES.chatCommandRegistration, {
|
||||
retryMs,
|
||||
pausePassiveMaintenance: true,
|
||||
onSettled: hints => { for (const hint of hints ?? ["*"]) dirty.add(hint); },
|
||||
run: async signal => {
|
||||
if (!canRun() || Date.now() < earliest) return;
|
||||
earliest = Date.now() + retryMs;
|
||||
const requested = dirty.has("*") ? ["*"] : [...dirty];
|
||||
dirty.clear();
|
||||
const errors: unknown[] = [];
|
||||
const unavailable: Array<CommandRegistrationScope | undefined> = [];
|
||||
for (let index = 0; index < requested.length; index++) {
|
||||
if (signal.aborted || !canRun()) {
|
||||
for (const remaining of requested.slice(index)) dirty.add(remaining);
|
||||
return;
|
||||
}
|
||||
const hint = requested[index];
|
||||
let scope: CommandRegistrationScope | undefined;
|
||||
try {
|
||||
scope = decodeScope(hint);
|
||||
const rows = await service.load(scope, signal);
|
||||
for (const entry of deadlines.values()) if (matches(entry, scope)) deadlines.delete(entry.id);
|
||||
for (const entry of rows) deadlines.set(entry.id, entry);
|
||||
for (const id of failures.keys()) {
|
||||
const entry = deadlines.get(id);
|
||||
if (!entry || entry.dueAt > Date.now()) failures.delete(id);
|
||||
}
|
||||
} catch (error) { dirty.add(scope ? hint : "*"); unavailable.push(scope); errors.push(error); }
|
||||
}
|
||||
const due = [...deadlines.values()]
|
||||
.filter(entry => !unavailable.some(scope => matches(entry, scope)) &&
|
||||
Math.max(entry.dueAt, failures.get(entry.id)?.retryAt ?? 0) <= Date.now())
|
||||
.sort((a, b) => a.dueAt - b.dueAt || a.id.localeCompare(b.id));
|
||||
const batch = takeCompanyWorkBatch(due, 25, lastCompany);
|
||||
lastCompany = batch.lastCompany;
|
||||
for (const entry of batch.entries) {
|
||||
if (signal.aborted || !canRun()) break;
|
||||
try {
|
||||
const current = await service.check(entry.id, signal);
|
||||
if (current) deadlines.set(entry.id, current); else deadlines.delete(entry.id);
|
||||
if (current && current.dueAt <= Date.now()) throw new Error("Discord command registration remains due");
|
||||
failures.delete(entry.id);
|
||||
} catch (error) {
|
||||
const count = Math.min((failures.get(entry.id)?.count ?? 0) + 1, 5);
|
||||
failures.set(entry.id, { count, retryAt: Date.now() + Math.min(300_000, retryMs * 2 ** (count - 1)) });
|
||||
errors.push(error);
|
||||
}
|
||||
}
|
||||
if (due.length > batch.entries.length) earliest = Date.now() + 1_000;
|
||||
if (errors.length) throw new AggregateError(errors, "Discord command registration could not settle");
|
||||
},
|
||||
// Preserve hints/deadlines for resume without holding settled intent during drain.
|
||||
// The coordinator independently fences unresolved database transactions.
|
||||
hasPending: async () => canRun() && dirty.size > 0,
|
||||
nextRunAt: next,
|
||||
});
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyDiscordCommandRegistration } from "./command-work-notifications.js";
|
||||
import type { EndpointRow, DbTransaction, RuntimeContext, LifecycleRuntimeFence, InternalSetupState } from "../runtime-types.js";
|
||||
|
||||
import { notifyChatEndpointWork } from "../../chat-work-notifications.js";
|
||||
@@ -129,6 +130,7 @@ export function createDiscordGatewayEvents(db: Db, options: DiscordGatewayEvents
|
||||
}
|
||||
const now = new Date();
|
||||
if (event.type === "connecting") {
|
||||
if (current.status === "attention") await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: current.id });
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
@@ -157,6 +159,7 @@ export function createDiscordGatewayEvents(db: Db, options: DiscordGatewayEvents
|
||||
if (current.status === "attention" && !gatewayOwnedAttention) return;
|
||||
const status =
|
||||
current.setup.step === "complete" ? "active" : "verifying";
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: current.id });
|
||||
if (status !== current.status) await notifyChatEndpointWork(tx, current.connectionId);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
@@ -186,6 +189,7 @@ export function createDiscordGatewayEvents(db: Db, options: DiscordGatewayEvents
|
||||
return;
|
||||
}
|
||||
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: current.id });
|
||||
const detail = discordGatewayFailureDetail(event);
|
||||
const transientGuildOutage = event.type === "guild_unavailable";
|
||||
const terminal =
|
||||
|
||||
@@ -7,11 +7,9 @@ import type { Db } from "@paperclipai/db";
|
||||
import type { EndpointRow, EndpointRecord, RuntimeContext } from "../runtime-types.js";
|
||||
import type { ChatRuntime } from "../../chat-runtime.js";
|
||||
import type { ChatEndpointRuntime } from "../../voice/voice-runtime.js";
|
||||
import type { createDiscordCommands } from "./commands.js";
|
||||
import type { createChatMessageLifecycle } from "../inbound/lifecycle.js";
|
||||
import type { ToolCredentialSecretRef } from "@paperclipai/shared";
|
||||
type ReconciliationOptions = {
|
||||
reconcileDiscordCommands: ReturnType<typeof createDiscordCommands>["reconcileDiscordCommands"];
|
||||
endpointRecord: (endpointId: string) => Promise<EndpointRecord | null>;
|
||||
runtimeContexts: WeakMap<object, RuntimeContext>;
|
||||
invalidateRuntime: (endpointId: string) => Promise<boolean>;
|
||||
@@ -22,9 +20,8 @@ type ReconciliationOptions = {
|
||||
discordGatewayRuntimeIsCurrent: (endpointId: string, context: RuntimeContext) => boolean;
|
||||
};
|
||||
export function createDiscordRuntimeReconciliation(db: Db, options: ReconciliationOptions) {
|
||||
const { reconcileDiscordCommands, endpointRecord, runtimeContexts, invalidateRuntime, runtimeFor, runtime, runtimeCallbackEndpoint, credentialFingerprint, discordGatewayRuntimeIsCurrent } = options;
|
||||
const { endpointRecord, runtimeContexts, invalidateRuntime, runtimeFor, runtime, runtimeCallbackEndpoint, credentialFingerprint, discordGatewayRuntimeIsCurrent } = options;
|
||||
async function reconcileDiscordRuntime(endpointId: string, instance: ChatEndpointRuntime) {
|
||||
await reconcileDiscordCommands(endpointId);
|
||||
const afterCommands = await endpointRecord(endpointId);
|
||||
const desiredCommands =
|
||||
afterCommands?.endpoint.capabilities.slashCommands &&
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCommandRegistration } from "../command-work-notifications.js";
|
||||
import { notifyChatConnectionHealth } from "../connection-health-notifications.js";
|
||||
import { slackQuestionFormValidationError } from "../slack/question-forms.js";
|
||||
import { chatInteractionForm, chatItemVerdicts } from "../../chat-interaction-forms.js";
|
||||
@@ -417,6 +418,7 @@ export function createChatInteractionCallbacks(db: Db, options: InteractionCallb
|
||||
["active"],
|
||||
);
|
||||
if (currentEndpoint) {
|
||||
await notifyChatCommandRegistration(tx, currentEndpoint);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCommandRegistration } from "./command-work-notifications.js";
|
||||
import { notifyChatConnectionHealth } from "./connection-health-notifications.js";
|
||||
import { createTelegramCallbackNoticeAuthority } from "./telegram/callback-notices.js";
|
||||
|
||||
@@ -317,6 +318,7 @@ export function createProviderEffects(db: Db, options: ProviderEffectsOptions) {
|
||||
);
|
||||
}
|
||||
if (input.disposition.kind === "endpoint_attention" && currentEndpoint) {
|
||||
await notifyChatCommandRegistration(tx, currentEndpoint);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCommandRegistration } from "../command-work-notifications.js";
|
||||
import { notifyChatConnectionHealth } from "../connection-health-notifications.js";
|
||||
import { isOutboundAttachmentHydrationError, isOutboundAttachmentValidationError } from "../outbound-attachments.js";
|
||||
|
||||
@@ -189,6 +190,7 @@ export function createChatPublicationFailures(db: Db, options: PublicationFailur
|
||||
: null;
|
||||
|
||||
if (disposition.kind === "endpoint_attention" && currentEndpoint) {
|
||||
await notifyChatCommandRegistration(tx, currentEndpoint);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCommandRegistration } from "./command-work-notifications.js";
|
||||
import { notifyChatConnectionHealth } from "./connection-health-notifications.js";
|
||||
import type { ConnectionManagementContext } from "../connection-management.js";
|
||||
import { imessageResourceParticipants } from "./imessage-photon/resources.js";
|
||||
@@ -510,6 +511,7 @@ export function createChatResourceReconciliation(db: Db, options: ResourceReconc
|
||||
: currentEndpoint.status === "attention"
|
||||
? "verifying"
|
||||
: currentEndpoint.status;
|
||||
await notifyChatCommandRegistration(tx, currentEndpoint);
|
||||
if (status !== currentEndpoint.status) await notifyChatEndpointWork(tx, currentEndpoint.connectionId);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
@@ -539,6 +541,7 @@ export function createChatResourceReconciliation(db: Db, options: ResourceReconc
|
||||
.where(eq(toolConnections.id, currentEndpoint.connectionId));
|
||||
}
|
||||
} else {
|
||||
await notifyChatCommandRegistration(tx, currentEndpoint);
|
||||
await tx
|
||||
.update(chatEndpoints)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
import { reconcileDatabaseWork } from "@paperclipai/db";
|
||||
import { idleWorkSnapshot } from "./task-admission.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
import { discordCommandSchedule } from "./chat-channels/discord/command-schedule.js";
|
||||
import { registerDiscordCommandWork } from "./chat-channels/discord/command-work.js";
|
||||
import { notifyDiscordCommandRegistration } from "./chat-channels/discord/command-work-notifications.js";
|
||||
import { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import {
|
||||
afterAll,
|
||||
@@ -23,6 +30,7 @@ import {
|
||||
} from "@paperclipai/db";
|
||||
import { discordPaperclipCommandDefinition } from "./chat-discord-command-registration.js";
|
||||
import {
|
||||
readDiscordCommandRegistrationDeadline,
|
||||
readRegisteredDiscordCommandRegistration,
|
||||
reconcileStoredDiscordCommandRegistration,
|
||||
type StoredDiscordCommandRegistrationOptions,
|
||||
@@ -55,7 +63,33 @@ suite("Discord command ownership store (real PostgreSQL, no network)", () => {
|
||||
await db?.$client.end();
|
||||
await temporary?.cleanup();
|
||||
});
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
let coordinator: ReturnType<typeof createDeliveryWorkCoordinator> | undefined;
|
||||
afterEach(async () => {
|
||||
await coordinator?.stop(); coordinator = undefined;
|
||||
vi.useRealTimers(); vi.restoreAllMocks();
|
||||
});
|
||||
async function scheduled(companyId: string, reconcile: (id: string) => Promise<void>, canRun = () => true) {
|
||||
await Promise.all(Array.from({ length: 4 }, () => db.execute(sql`select 1`)));
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
await reconcileDatabaseWork(db, DELIVERY_QUEUES.chatCommandRegistration);
|
||||
const service = discordCommandSchedule(db, reconcile);
|
||||
const errors: unknown[] = [];
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun, canReconcile: () => true, onError: error => errors.push(error) });
|
||||
const worker = registerDiscordCommandWork(coordinator, {
|
||||
...service,
|
||||
load: (scope, signal) => service.load(scope ?? { kind: "company", id: companyId }, signal),
|
||||
}, canRun);
|
||||
await worker.ready;
|
||||
return { service, errors };
|
||||
}
|
||||
async function sqlQuiet(milliseconds: number) {
|
||||
const reads = vi.spyOn(db, "select"), writes = vi.spyOn(db, "transaction"), execute = vi.spyOn(db, "execute");
|
||||
await vi.advanceTimersByTimeAsync(milliseconds);
|
||||
expect(reads).not.toHaveBeenCalled();
|
||||
expect(writes).not.toHaveBeenCalled();
|
||||
expect(execute).not.toHaveBeenCalled();
|
||||
reads.mockRestore(); writes.mockRestore(); execute.mockRestore();
|
||||
}
|
||||
|
||||
async function fixture(applicationId = appId()) {
|
||||
const companyId = randomUUID(),
|
||||
@@ -628,4 +662,256 @@ suite("Discord command ownership store (real PostgreSQL, no network)", () => {
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
},
|
||||
);
|
||||
it("exposes saved verification and retry deadlines without contacting the provider", async () => {
|
||||
const f = await fixture();
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBe(0);
|
||||
expect(f.calls).toEqual([]);
|
||||
expect(await f.reconcile()).toMatchObject({ kind: "registered" });
|
||||
const row = await f.read();
|
||||
const due = Date.parse(String(row!.result!.retryAt));
|
||||
expect(due).toBeGreaterThan(Date.now() + 290_000);
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBe(due);
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
const retriedAt = new Date(Date.now() + 601_125).toISOString();
|
||||
await db.update(chatActions).set({ result: { ...row!.result, outcome: "unavailable", retryAt: retriedAt } })
|
||||
.where(eq(chatActions.id, row!.id));
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBe(Date.parse(retriedAt));
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
});
|
||||
it("parks indefinite and malformed retry records without inventing an earlier deadline", async () => {
|
||||
const f = await fixture(); await f.reconcile(); const row = (await f.read())!;
|
||||
for (const result of [
|
||||
{ ...row.result, retryAt: "+275760-09-13T00:00:00.000Z", retryIndefinite: true },
|
||||
{ ...row.result, retryAt: "invalid" },
|
||||
{ ...row.result, schema: "unknown" },
|
||||
]) {
|
||||
await db.update(chatActions).set({ result }).where(eq(chatActions.id, row.id));
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBeNull();
|
||||
}
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
});
|
||||
it("does not schedule ownership tombstones or another endpoint's registration", async () => {
|
||||
const f = await fixture(); await f.reconcile();
|
||||
await db.update(chatEndpoints).set({ status: "archived" }).where(eq(chatEndpoints.id, f.scope.endpointId));
|
||||
const other = await fixture(f.scope.applicationId);
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, other.scope)).toBeNull();
|
||||
await db.delete(chatActions).where(eq(chatActions.id, (await f.read())!.id));
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBeNull();
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, other.scope)).toBeNull();
|
||||
expect(other.calls).toEqual([]);
|
||||
});
|
||||
it("retains the stored retry deadline for an uncertain provider write", async () => {
|
||||
const f = await fixture();
|
||||
f.beforeResponse(async method => { if (method === "POST") throw new Error("lost response"); });
|
||||
expect(await f.reconcile()).toMatchObject({ kind: "unknown" });
|
||||
const row = (await f.read())!;
|
||||
expect(row.payload.registration).toMatchObject({ phase: "attempted" });
|
||||
expect(await readDiscordCommandRegistrationDeadline(db, f.scope)).toBe(Date.parse(String(row.result!.retryAt)));
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
});
|
||||
|
||||
it("leaves no deadline or SQL after an empty registration inventory", async () => {
|
||||
const reconcile = vi.fn();
|
||||
const { errors } = await scheduled(randomUUID(), reconcile);
|
||||
expect(coordinator!.nextWakeAt()).toBeNull();
|
||||
await sqlQuiet(180_000);
|
||||
expect(reconcile).not.toHaveBeenCalled(); expect(errors).toEqual([]);
|
||||
});
|
||||
it("reconstructs a registered verification deadline and stays quiet until it is due", async () => {
|
||||
const f = await fixture(); await f.reconcile();
|
||||
const due = Date.parse(String((await f.read())!.result!.retryAt));
|
||||
const { errors } = await scheduled(f.scope.companyId, async () => { await f.reconcile(); });
|
||||
expect(coordinator!.nextWakeAt()).toBe(due);
|
||||
await sqlQuiet(120_000);
|
||||
expect(f.calls).toEqual(["GET", "POST"]);
|
||||
await vi.advanceTimersByTimeAsync(due - Date.now());
|
||||
await vi.waitFor(() => expect(f.calls).toEqual(["GET", "POST", "GET"]));
|
||||
await vi.waitFor(async () => expect(Date.parse(String((await f.read())!.result!.retryAt))).toBeGreaterThan(due));
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
it("keeps indefinite holds quiet and respects disabled endpoint eligibility", async () => {
|
||||
const f = await fixture(); await f.reconcile(); const row = (await f.read())!;
|
||||
await db.update(chatActions).set({ result: { ...row.result, retryIndefinite: true } }).where(eq(chatActions.id, row.id));
|
||||
const reconcile = vi.fn();
|
||||
const { service, errors } = await scheduled(f.scope.companyId, reconcile);
|
||||
await sqlQuiet(600_000); expect(coordinator!.nextWakeAt()).toBeNull();
|
||||
await db.update(chatActions).set({ result: { ...row.result, retryAt: new Date(0).toISOString() } }).where(eq(chatActions.id, row.id));
|
||||
await db.update(chatEndpoints).set({ status: "paused" }).where(eq(chatEndpoints.id, f.scope.endpointId));
|
||||
expect(await service.check(f.scope.endpointId)).toBeNull();
|
||||
expect(reconcile).not.toHaveBeenCalled(); expect(errors).toEqual([]);
|
||||
});
|
||||
it("honors outer rollback and wakes from committed endpoint changes after drain", async () => {
|
||||
const f = await fixture();
|
||||
await db.update(chatEndpoints).set({ status: "paused" }).where(eq(chatEndpoints.id, f.scope.endpointId));
|
||||
let draining = false;
|
||||
const { errors } = await scheduled(f.scope.companyId, async () => { await f.reconcile(); }, () => !draining);
|
||||
const activate = async (tx: typeof db) => {
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: f.scope.endpointId });
|
||||
await tx.update(chatEndpoints).set({ status: "active" }).where(eq(chatEndpoints.id, f.scope.endpointId));
|
||||
};
|
||||
await expect(db.transaction(async tx => {
|
||||
await tx.transaction(async nested => activate(nested as unknown as typeof db));
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
expect(f.calls).toEqual([]);
|
||||
throw new Error("abort");
|
||||
})).rejects.toThrow("abort");
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
// A running sweep has already consumed its timer. Wait for its SQL and
|
||||
// transaction reconciliation to settle before measuring idle traffic.
|
||||
await vi.waitFor(() => {
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(coordinator!.nextWakeAt()).toBeNull();
|
||||
});
|
||||
await sqlQuiet(60_000);
|
||||
draining = true;
|
||||
await db.transaction(async tx => activate(tx as unknown as typeof db));
|
||||
await vi.waitFor(async () => { await vi.advanceTimersByTimeAsync(1000); expect(idleWorkSnapshot().active).toBe(0); });
|
||||
const drained = idleWorkSnapshot();
|
||||
await sqlQuiet(90_000); expect(idleWorkSnapshot()).toEqual(drained); expect(f.calls).toEqual([]);
|
||||
draining = false;
|
||||
await vi.advanceTimersByTimeAsync(30_000);
|
||||
await vi.waitFor(() => expect(f.calls).toEqual(["GET", "POST"]));
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
it("rechecks persisted eligibility before a due provider call", async () => {
|
||||
const f = await fixture(); await f.reconcile();
|
||||
const { service, errors } = await scheduled(f.scope.companyId, async () => { await f.reconcile(); });
|
||||
await f.due();
|
||||
await db.update(toolConnections).set({ enabled: false }).where(eq(toolConnections.id,
|
||||
(await db.select().from(chatEndpoints).where(eq(chatEndpoints.id, f.scope.endpointId)))[0].connectionId));
|
||||
expect(await service.check(f.scope.endpointId)).toBeNull();
|
||||
expect(f.calls).toEqual(["GET", "POST"]); expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("backs off an attempt that cannot advance its durable state", async () => {
|
||||
const f = await fixture();
|
||||
const reconcile = vi.fn(async () => {});
|
||||
const { errors } = await scheduled(f.scope.companyId, reconcile);
|
||||
expect(reconcile).toHaveBeenCalledTimes(1);
|
||||
await vi.advanceTimersByTimeAsync(30_000);
|
||||
await vi.waitFor(() => expect(reconcile).toHaveBeenCalledTimes(2));
|
||||
await vi.waitFor(() => expect(errors).toHaveLength(2));
|
||||
await sqlQuiet(30_000);
|
||||
expect(reconcile).toHaveBeenCalledTimes(2);
|
||||
await vi.advanceTimersByTimeAsync(30_000);
|
||||
await vi.waitFor(() => expect(reconcile).toHaveBeenCalledTimes(3));
|
||||
expect(await f.read()).toBeUndefined();
|
||||
});
|
||||
it("joins an admitted provider attempt during shutdown", async () => {
|
||||
const f = await fixture();
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>(resolve => { release = resolve; });
|
||||
f.beforeResponse(async method => { if (method === "GET") await held; });
|
||||
const starting = scheduled(f.scope.companyId, async () => { await f.reconcile(); });
|
||||
await vi.waitFor(() => expect(f.calls).toEqual(["GET"]));
|
||||
let stopped = false;
|
||||
const stopping = coordinator!.stop().then(() => { stopped = true; });
|
||||
await Promise.resolve(); expect(stopped).toBe(false);
|
||||
release(); await stopping; const { errors } = await starting;
|
||||
expect(stopped).toBe(true);
|
||||
expect(await f.read()).toMatchObject({ status: "processed" });
|
||||
expect(coordinator!.nextWakeAt()).toBeNull(); expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("retains a committed eligibility change while the provider attempt is in flight", async () => {
|
||||
const f = await fixture(); await f.reconcile(); await f.due();
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>(resolve => { release = resolve; });
|
||||
f.beforeResponse(async method => { if (method === "GET") await held; });
|
||||
const starting = scheduled(f.scope.companyId, async () => { await f.reconcile(); });
|
||||
try {
|
||||
await vi.waitFor(() => expect(f.calls).toEqual(["GET", "POST", "GET"]));
|
||||
await db.transaction(async tx => {
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: f.scope.endpointId });
|
||||
await tx.update(chatEndpoints).set({ status: "paused" }).where(eq(chatEndpoints.id, f.scope.endpointId));
|
||||
});
|
||||
} finally { release(); }
|
||||
const { errors } = await starting;
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(() => expect(coordinator!.nextWakeAt()).toBeNull());
|
||||
await sqlQuiet(120_000);
|
||||
expect(f.calls).toEqual(["GET", "POST", "GET"]); expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("checks a quiet company's due command in the first batch despite another company's backlog", async () => {
|
||||
await reconcileDatabaseWork(db, DELIVERY_QUEUES.chatCommandRegistration);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const entries = Array.from({ length: 30 }, (_, index) => ({
|
||||
id: `busy-${index.toString().padStart(2, "0")}`, companyId: "busy", connectionId: "busy-connection", dueAt: 0,
|
||||
}));
|
||||
entries.push({ id: "quiet", companyId: "quiet", connectionId: "quiet-connection", dueAt: 1 });
|
||||
const check = vi.fn(async (id: string) => ({ ...entries.find(entry => entry.id === id)!, dueAt: Date.now() + 300_000 }));
|
||||
const errors: unknown[] = [];
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
await registerDiscordCommandWork(coordinator, { load: async () => entries, check }, () => true).ready;
|
||||
expect(check).toHaveBeenCalledTimes(25);
|
||||
expect(check.mock.calls.map(call => call[0])).toContain("quiet");
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("drains a large due backlog without a thirty-second delay between bounded batches", async () => {
|
||||
await reconcileDatabaseWork(db, DELIVERY_QUEUES.chatCommandRegistration);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const entries = Array.from({ length: 300 }, (_, index) => ({
|
||||
id: `command-${index}`, companyId: `company-${index % 10}`, connectionId: `connection-${index}`, dueAt: 0,
|
||||
}));
|
||||
const check = vi.fn(async (id: string) => ({ ...entries.find(entry => entry.id === id)!, dueAt: Date.now() + 300_000 }));
|
||||
const errors: unknown[] = [];
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
await registerDiscordCommandWork(coordinator, { load: async () => entries, check }, () => true).ready;
|
||||
expect(check).toHaveBeenCalledTimes(25);
|
||||
await vi.advanceTimersByTimeAsync(15_000);
|
||||
expect(check).toHaveBeenCalledTimes(300);
|
||||
expect(new Set(check.mock.calls.map(([id]) => id)).size).toBe(300);
|
||||
await sqlQuiet(120_000);
|
||||
expect(check).toHaveBeenCalledTimes(300);
|
||||
expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("drains healthy commands promptly while one entry remains in failure backoff", async () => {
|
||||
await reconcileDatabaseWork(db, DELIVERY_QUEUES.chatCommandRegistration);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const entries = Array.from({ length: 300 }, (_, index) => ({
|
||||
id: `command-${index}`, companyId: `company-${index % 10}`, connectionId: `connection-${index}`, dueAt: 0,
|
||||
}));
|
||||
const check = vi.fn(async (id: string) => {
|
||||
if (id === "command-0") throw new Error("Unavailable credential");
|
||||
return { ...entries.find(entry => entry.id === id)!, dueAt: Date.now() + 300_000 };
|
||||
});
|
||||
const onError = vi.fn();
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError });
|
||||
await registerDiscordCommandWork(coordinator, { load: async () => entries, check }, () => true).ready;
|
||||
await vi.advanceTimersByTimeAsync(15_000);
|
||||
expect(new Set(check.mock.calls.map(([id]) => id)).size).toBe(300);
|
||||
expect(check.mock.calls.filter(([id]) => id === "command-0")).toHaveLength(1);
|
||||
expect(onError).toHaveBeenCalledOnce();
|
||||
await vi.advanceTimersByTimeAsync(45_000);
|
||||
expect(check.mock.calls.filter(([id]) => id === "command-0")).toHaveLength(2);
|
||||
expect(check.mock.calls.filter(([id]) => id !== "command-0")).toHaveLength(299);
|
||||
});
|
||||
it("recovers queued scopes after an unknown hint instead of dropping the wake", async () => {
|
||||
await reconcileDatabaseWork(db, DELIVERY_QUEUES.chatCommandRegistration);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const entry = { id: "endpoint", companyId: "company", connectionId: "connection", dueAt: 0 };
|
||||
let owed = false;
|
||||
const check = vi.fn(async () => { owed = false; return null; });
|
||||
const errors: unknown[] = [];
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
await registerDiscordCommandWork(coordinator, { load: async () => owed ? [entry] : [], check }, () => true).ready;
|
||||
await db.transaction(async tx => {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatCommandRegistration, "unknown:legacy");
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: entry.id });
|
||||
owed = true;
|
||||
});
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1_000);
|
||||
expect(errors).toHaveLength(1);
|
||||
}, { timeout: 5_000 });
|
||||
await vi.advanceTimersByTimeAsync(31_000);
|
||||
await vi.waitFor(() => expect(check).toHaveBeenCalledTimes(1));
|
||||
await vi.waitFor(() => expect(coordinator!.nextWakeAt()).toBeNull());
|
||||
await sqlQuiet(120_000);
|
||||
expect(errors).toHaveLength(1);
|
||||
});
|
||||
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyDiscordCommandRegistration } from "./chat-channels/discord/command-work-notifications.js";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import {
|
||||
@@ -66,6 +67,19 @@ function safeResult(outcome: string, seconds?: number) {
|
||||
return { schema: resultSchema, outcome, ...retrySchedule(outcome, seconds) };
|
||||
}
|
||||
|
||||
function registrationRetryAt(result: typeof chatActions.$inferSelect["result"]): number | null {
|
||||
if (result?.schema !== resultSchema || typeof result.retryAt !== "string") return null;
|
||||
const at = Date.parse(result.retryAt);
|
||||
return Number.isFinite(at) ? at : null;
|
||||
}
|
||||
|
||||
async function hasRegistrationHistory(database: Database, applicationId: string): Promise<boolean> {
|
||||
const [history] = await database.select({ id: chatActions.id }).from(chatActions).where(and(
|
||||
eq(chatActions.kind, kind), eq(chatActions.providerActionId, actionKey(applicationId)),
|
||||
)).limit(1);
|
||||
return history !== undefined;
|
||||
}
|
||||
|
||||
async function stored(
|
||||
database: Database,
|
||||
scope: DiscordCommandRegistrationScope,
|
||||
@@ -136,6 +150,21 @@ export async function readRegisteredDiscordCommandRegistration(
|
||||
: null;
|
||||
}
|
||||
|
||||
export async function readDiscordCommandRegistrationDeadline(
|
||||
database: Database,
|
||||
scope: DiscordCommandRegistrationScope,
|
||||
): Promise<number | null> {
|
||||
const current = await stored(database, scope, false, true);
|
||||
if (current) {
|
||||
if (current.row.result?.retryIndefinite === true) return null;
|
||||
return registrationRetryAt(current.row.result);
|
||||
}
|
||||
const [owner] = await database.select({ applicationId: chatDiscordCommandOwners.applicationId })
|
||||
.from(chatDiscordCommandOwners).where(eq(chatDiscordCommandOwners.applicationId, scope.applicationId));
|
||||
if (owner || await hasRegistrationHistory(database, scope.applicationId)) return null;
|
||||
return 0;
|
||||
}
|
||||
|
||||
export async function reconcileStoredDiscordCommandRegistration(
|
||||
db: Db,
|
||||
options: StoredDiscordCommandRegistrationOptions,
|
||||
@@ -169,17 +198,8 @@ export async function reconcileStoredDiscordCommandRegistration(
|
||||
.where(eq(chatDiscordCommandOwners.applicationId, scope.applicationId))
|
||||
.for("update");
|
||||
if (!priorOwner) {
|
||||
const [history] = await tx
|
||||
.select({ id: chatActions.id })
|
||||
.from(chatActions)
|
||||
.where(
|
||||
and(
|
||||
eq(chatActions.kind, kind),
|
||||
eq(chatActions.providerActionId, actionKey(scope.applicationId)),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (history) deny(); // No adoption of history lacking the global claim.
|
||||
if (await hasRegistrationHistory(tx, scope.applicationId)) deny();
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: scope.endpointId });
|
||||
const actionId = randomUUID();
|
||||
const inserted = await tx
|
||||
.insert(chatDiscordCommandOwners)
|
||||
@@ -209,11 +229,8 @@ export async function reconcileStoredDiscordCommandRegistration(
|
||||
const current = await stored(tx, scope, true, true);
|
||||
if (!current) deny(); // In particular, never recreate an orphaned action.
|
||||
const result = current.row.result;
|
||||
const due =
|
||||
result?.schema === resultSchema && typeof result.retryAt === "string"
|
||||
? Date.parse(result.retryAt)
|
||||
: NaN;
|
||||
if (!Number.isFinite(due)) deny();
|
||||
const due = registrationRetryAt(result);
|
||||
if (due === null) deny();
|
||||
return {
|
||||
state: current.state,
|
||||
deferred:
|
||||
@@ -253,6 +270,7 @@ export async function reconcileStoredDiscordCommandRegistration(
|
||||
await db.transaction(async (tx) => {
|
||||
const current = await check(tx);
|
||||
if (!same(current.state, expected)) deny();
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: scope.endpointId });
|
||||
const result = safeResult(
|
||||
next.phase === "registered" ? "registered" : "unknown",
|
||||
);
|
||||
@@ -272,6 +290,7 @@ export async function reconcileStoredDiscordCommandRegistration(
|
||||
});
|
||||
await db.transaction(async (tx) => {
|
||||
const current = await check(tx);
|
||||
await notifyDiscordCommandRegistration(tx, { kind: "endpoint", id: scope.endpointId });
|
||||
await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatConnectionCommandRegistration } from "./chat-channels/command-work-notifications.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export async function notifyChatDeliveryWork(tx: object): Promise<void> {
|
||||
@@ -37,4 +38,5 @@ export async function notifyChatEndpointWork(tx: object, connectionId: string):
|
||||
for (const queue of Object.values(CHAT_ACTION_QUEUES)) await notifyDeliveryWork(tx, queue);
|
||||
await notifyChatVerificationWork(tx);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatTelegramSubscriptions, connectionId);
|
||||
await notifyChatConnectionCommandRegistration(tx, connectionId);
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatCompanyCommandRegistration } from "./chat-channels/command-work-notifications.js";
|
||||
import { chatCompanyDeletion } from "./chat-channels/company-deletion.js";
|
||||
import { notifyToolConnectionHealth } from "./tool-connection-health-notifications.js";
|
||||
import { notifyCompanyRetentionWork } from "./retention-work-notifications.js";
|
||||
@@ -64,6 +65,7 @@ export async function deleteCompany(db: Db, id: string) {
|
||||
const [existing] = await tx.select({ id: companies.id }).from(companies)
|
||||
.where(eq(companies.id, id)).for("no key update");
|
||||
if (!existing) return null;
|
||||
await notifyChatCompanyCommandRegistration(tx, id);
|
||||
await chatCompanyDeletion.deleteCompanyData(tx, id);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.skillSourceSync);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.secretProposalExpiry, id);
|
||||
|
||||
@@ -43,6 +43,7 @@ export const DELIVERY_QUEUES = {
|
||||
chatGitHubPublications: "chat-github-publications",
|
||||
chatFailedRunRetries: "chat-failed-run-retries",
|
||||
chatTelegramMaintenance: "chat-telegram-maintenance",
|
||||
chatCommandRegistration: "chat-command-registration",
|
||||
chatTelegramSubscriptions: "chat-telegram-subscriptions",
|
||||
chatSessionSyncs: "chat-session-syncs",
|
||||
chatVerification: "chat-verification",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatConnectionCommandRegistration } from "../../chat-channels/command-work-notifications.js";
|
||||
import { writeToolConnectionHealthChange } from "../../tool-connection-health-notifications.js";
|
||||
import { connectionActorUserId } from "../actor.js";
|
||||
import { assertSupportedConnection } from "../connections/setup-policy.js";
|
||||
@@ -48,8 +49,9 @@ export function createConnectionHealth(db: Db, { now, audit, getConnectionRow, d
|
||||
message: string | null,
|
||||
) {
|
||||
const now = new Date();
|
||||
const [updated] = await writeToolConnectionHealthChange(db, connection, tx => tx
|
||||
.update(toolConnections)
|
||||
const [updated] = await writeToolConnectionHealthChange(db, connection, async tx => {
|
||||
await notifyChatConnectionCommandRegistration(tx, connection.id, connection.transport);
|
||||
return tx.update(toolConnections)
|
||||
.set({
|
||||
healthStatus: status,
|
||||
healthMessage: message,
|
||||
@@ -59,7 +61,8 @@ export function createConnectionHealth(db: Db, { now, audit, getConnectionRow, d
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(eq(toolConnections.id, connection.id))
|
||||
.returning());
|
||||
.returning();
|
||||
});
|
||||
if (connection.transport === "local_stdio") {
|
||||
await db
|
||||
.update(toolRuntimeSlots)
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyChatEndpointWork } from "../../chat-work-notifications.js";
|
||||
import { notifyToolConnectionHealth } from "../../tool-connection-health-notifications.js";
|
||||
import { toConnection } from "./projection.js";
|
||||
import { actorBinding } from "../actor.js";
|
||||
@@ -539,6 +540,7 @@ async function removeConnection(
|
||||
.returning({ id: connectionTokenIssuances.id });
|
||||
|
||||
const archived = await db.transaction(async (tx) => {
|
||||
if (connection.transport === "chat_sdk") await notifyChatEndpointWork(tx, connection.id);
|
||||
await notifyToolConnectionHealth(tx, connection);
|
||||
await tx.select({ id: toolApplications.id }).from(toolApplications)
|
||||
.where(and(eq(toolApplications.id, connection.applicationId), eq(toolApplications.companyId, connection.companyId)))
|
||||
|
||||
Reference in new issue
Block a user