mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 21:49:18 +01:00
fix(server): schedule governed review response recovery (#17702)
## Thinking Path > - Paperclip manages agents, tasks and their external chat responses. > - A completed chat turn can owe a response while an earlier review gate remains pending. > - Periodic recovery revisits that response history even when no work can advance. > - This change uses the existing delivery coordinator and saved retry deadlines. > - Approval, interaction and authority changes wake the affected issue or run after commit. > - The existing presentation owner verifies the durable result and current authority. > - Settled and dormant history becomes quiet without losing future eligible responses. ## Linked Issues or Issue Description Refs #17696 (merged) and #15625. This is slice 49, now based on master. It owns governed-review response presentation. Accepted responses and pending decision effects keep their existing owners. **What happened?** Native finalization recovery repeatedly inspects committed review responses. Removing the scan without covering approval, interaction, task-policy and authority changes would strand responses that become eligible later. **Expected behavior** Recover an owed response once from existing durable facts. Enforce authority at publication, wake after relevant outer transactions commit, retain real retry deadlines and restart/drain recovery, and stop recurring SQL when no response can advance. **Steps to reproduce** Complete an external chat turn while an earlier completion review remains pending. Block its response behind an approval or interaction, then resolve that blocker. The response must appear once without a periodic scan or resolving the original review gate. ## What Changed - Add a scoped governed-review worker using the existing coordinator, finalizations, decisions and comment receipts. No queue table or transaction wrapper is added. - Keep recovery and publication policy in the native finalizer. Preserve binding, source/session authority, ownership, deadline and deleted-comment fences; match coordinator-before-issue lock ordering. - Notify through actual approval, interaction, OAuth, task-policy and lifecycle producers before their owned writes. Serialize approval resolution with task linking so a concurrent link retains its wake. Nested transactions still settle at outer commit. - Scope ordinary milestones, finalization and coalesced input to their issue/run. Use the explicit company authority boundary for genuine global restoration, including membership changes. Both response workers share this boundary. - Register under maintenance and drain gates, disable the two periodic repair entrances, and retain fresh finalization and explicit recovery. Add the concurrent 0413 comment-receipt index and migration registration; retain five snapshots. - Preserve the producer inventory, source qualifications and earlier failures in the owner audit. Retain a real Photon regression proving routine ownership assertions stay quiet and concurrent authority restoration wakes each worker once. ## Verification - Scoped repair source `2341714720b5c273f0ce011b48345af0ef39227f` passes **2053 distinct cases**: 1703 in 15 complete producer/authority/recovery files, including all 273 governed-response cases and 95 wake-queue adapter cases, plus 350 selected chat cases. Another 1463 chat cases are outside that selection. Thirteen fresh PostgreSQL 17 fixtures migrate through unchanged 0413; the one temporary queue-fixture import restores exactly. All sixteen captured hashes match. - Two actual repeated coarse-hint SQL regressions fail before repair and pass afterward. Actual membership restoration passes before and after. The same assertions and time limits prove dormant work stays quiet and real authority restoration still publishes once. Real coalesced input covers both queues after commit. - Combined source `586f5674cd9798b33cd22437255e0480b5726d20`, including independent master #18129, passes **1297 distinct cases**: 1093 in five complete server files, 185 in five runner files, 14 Node protocol cases and five UI registry cases. This includes the complete 22-case Photon file for both response workers. The external Photon transport is mocked; Paperclip callbacks, transaction fences and observed SQL are real. The earlier and combined case counts are separate, with no cumulative double-counting. - A normal merge retains both complete owned and independent master binary patches. All twelve new-context hashes and sixteen earlier repair hashes match. Three dedicated fresh 0413 fixtures plus the live-event helper's own fresh database use PostgreSQL 17 without source adaptations. Database build, server types/build, boundaries, full-SHA migration order and generated TS/Rust profiles pass. The new inherited Runner vendor inputs rebuild successfully. - Database bytes remain identical to the previously qualified 0413 source. The recorded populated 0412-to0413 upgrade retains all 411 prior receipts and 38 companies, appends one receipt and none on replay; the index is valid and ready. The earlier 2828-case acceptance, CLI journey, setup failures and their repairs retain their recorded source scope in `doc/plans/2026-10-26-governed-review-response-sleep-audit.md`. - Parent #17696 merged with 58 clean checks, Greptile 5/5 and Ponytail 5/5. Its Photon warning was disproved by real callback/SQL evidence and the security check became successful. This PR still requires its own fresh exact-head CI and review gates. No deployed sleep, external alarm or fleet savings claim is made. ## Risks - Notifications use the existing process-local writer contract. Independent SQL writers and uncoordinated replicas are outside its coverage. - Genuine startup and company authority reconstruction remain proportional to the owed backlog. Queries page 100 rows, but discovery accumulates the candidates; this is not a constant-cost global operation. - Current publication authorization, durable binding, ownership and lock ordering remain mandatory. The worker does not resolve the original review gate. - The concurrent index uses existing migration replay handling. Other recovery duties and Cloud sleep eligibility remain separate. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, shell execution and PostgreSQL tests. The runtime does not expose a more specific serving snapshot or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
40 files changed
+1953
-145
No files matched your search
@@ -1,6 +1,6 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { approvalService } from "../services/approvals.ts";
|
||||
import { companies } from "@paperclipai/db";
|
||||
import { approvals, companies, issueApprovals } from "@paperclipai/db";
|
||||
|
||||
vi.mock("../services/budgets.js", async importOriginal => ({
|
||||
...await importOriginal<typeof import("../services/budgets.js")>(),
|
||||
@@ -50,10 +50,14 @@ function createDbStub(selectResults: ApprovalRecord[][], updateResults: Approval
|
||||
const pendingSelectResults = [...selectResults];
|
||||
let inTransaction = false;
|
||||
const selectWhere = vi.fn(async () => (inTransaction ? pendingSelectResults.shift() : pendingSelectResults[0]) ?? []);
|
||||
const from = vi.fn((table) => table === companies
|
||||
const from = vi.fn((table) => table === issueApprovals
|
||||
? { where: async () => [] }
|
||||
: table === companies
|
||||
? { where: () => ({ for: async () => [{ id: "company-1" }] }) }
|
||||
: { where: selectWhere });
|
||||
const select = vi.fn(() => ({ from }));
|
||||
const select = vi.fn((fields?: { id?: unknown }) => fields?.id === approvals.id
|
||||
? { from: () => ({ where: () => ({ for: async () => [{ id: "approval-1" }] }) }) }
|
||||
: { from });
|
||||
|
||||
const returning = vi.fn(async () => updateResults);
|
||||
const updateWhere = vi.fn(() => ({ returning }));
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
@@ -29,7 +30,8 @@ vi.mock("../telemetry.js", () => ({
|
||||
getTelemetryClient: telemetryMocks.getTelemetryClient,
|
||||
}));
|
||||
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_INTERACTION_TELEMETRY_TEST_DATABASE_URL");
|
||||
const embeddedPostgresSupport = externalUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||||
|
||||
describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
@@ -38,8 +40,8 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-issue-interaction-telemetry-");
|
||||
db = createDb(tempDb.connectionString);
|
||||
if (!externalUrl) tempDb = await startEmbeddedPostgresTestDatabase("paperclip-issue-interaction-telemetry-");
|
||||
db = createDb(externalUrl ?? tempDb!.connectionString);
|
||||
interactionsSvc = issueThreadInteractionService(db);
|
||||
}, 20_000);
|
||||
|
||||
@@ -65,6 +67,7 @@ describeEmbeddedPostgres("issueThreadInteractionService telemetry", () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await db?.$client.end({ timeout: 1 });
|
||||
await tempDb?.cleanup();
|
||||
});
|
||||
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
import { nativeReviewResponseWork } from "../../services/native-runtime/native-review-response-work.js";
|
||||
import { nativeResponsePresentationWork } from "../../services/native-runtime/native-response-presentation-work.js";
|
||||
import { DELIVERY_QUEUES } from "../../services/delivery-work-notifications.js";
|
||||
import { idleWorkSnapshot } from "../../services/task-admission.js";
|
||||
import { createDeliveryWorkCoordinator } from "../../services/delivery-work-coordinator.js";
|
||||
import { resolveExternalTestDatabaseUrl } from "../helpers/external-test-database.js";
|
||||
import { Readable } from "node:stream";
|
||||
@@ -24,6 +28,7 @@ import {
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import {
|
||||
applyPendingMigrations,
|
||||
subscribeDatabaseWork,
|
||||
createDb,
|
||||
companies,
|
||||
agents,
|
||||
@@ -372,6 +377,53 @@ describe("iMessage Photon channel control plane", () => {
|
||||
},
|
||||
};
|
||||
}
|
||||
it.each([DELIVERY_QUEUES.nativeResponsePresentation, DELIVERY_QUEUES.nativeReviewResponse])(
|
||||
"keeps repeated active Photon ownership assertions out of %s reconstruction and wakes once on recovery", async queue => {
|
||||
const t = await setup();
|
||||
await t.start();
|
||||
await t.qualify();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
coordinators.push(coordinator);
|
||||
const work = queue === DELIVERY_QUEUES.nativeResponsePresentation ? nativeResponsePresentationWork(db) : nativeReviewResponseWork(db);
|
||||
const worker = work.register(coordinator, () => true);
|
||||
const debug = db.$client.options.debug;
|
||||
const hints: unknown[] = [];
|
||||
const queries: string[] = [];
|
||||
const unsubscribe = subscribeDatabaseWork(db, queue, (event, values) => {
|
||||
if (event === "settled") for (const value of values ?? []) {
|
||||
const scope = JSON.parse(value);
|
||||
if (scope.companyId === t.companyId) hints.push(scope);
|
||||
}
|
||||
});
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 5000 })
|
||||
.toEqual({ active: 0, next: null });
|
||||
db.$client.options.debug = (_connection, query) => {
|
||||
if (query.startsWith("select") && query.includes("native_run_finalizations")) queries.push(query);
|
||||
};
|
||||
const assertOwned = t.callbacks().onPhotonAssertOwned!;
|
||||
for (let i = 0; i < 5; i++) await assertOwned();
|
||||
expect(hints).toEqual([]);
|
||||
expect(queries).toEqual([]);
|
||||
await db.update(chatEndpoints).set({ status: "attention", healthMessage: "Recovering transport" })
|
||||
.where(eq(chatEndpoints.id, t.endpoint.id));
|
||||
await Promise.all([assertOwned(), assertOwned(), assertOwned()]);
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 5000 })
|
||||
.toEqual({ active: 0, next: null });
|
||||
expect(hints).toEqual([{ companyId: t.companyId }]);
|
||||
expect(queries).toHaveLength(1);
|
||||
expect((await t.service.get(t.endpoint.id)).status).toBe("active");
|
||||
hints.length = 0;
|
||||
queries.length = 0;
|
||||
for (let i = 0; i < 5; i++) await assertOwned();
|
||||
expect(hints).toEqual([]);
|
||||
expect(queries).toEqual([]);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { unsubscribe(); db.$client.options.debug = debug; if ("stop" in worker) worker.stop(); }
|
||||
});
|
||||
|
||||
it("recovers a Photon transport loss through the receiver worker and cancels fatal recovery", async () => {
|
||||
const t = await setup();
|
||||
const onError = vi.fn();
|
||||
|
||||
@@ -21,19 +21,16 @@ import { idleWorkSnapshot } from "../services/task-admission.js";
|
||||
import { readIdleSleepSafety, type IdleSleepDrainStatus } from "../services/idle-sleep-safety.js";
|
||||
import { reserveRunBudget } from "../services/budget-reservations.js";
|
||||
|
||||
const externalTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_SATISFIED_CONNECTION_TEST_DATABASE_URL");
|
||||
const support = externalTestDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_SATISFIED_CONNECTION_TEST_DATABASE_URL");
|
||||
const support = externalUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
const describePostgres = support.supported ? describe : describe.skip;
|
||||
|
||||
describePostgres("already available connection requests", () => {
|
||||
let db!: ReturnType<typeof createDb>;
|
||||
let temp!: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
beforeAll(async () => {
|
||||
if (externalTestDatabaseUrl) db = createDb(externalTestDatabaseUrl);
|
||||
else {
|
||||
temp = await startEmbeddedPostgresTestDatabase("paperclip-satisfied-connections-");
|
||||
db = createDb(temp.connectionString);
|
||||
}
|
||||
if (!externalUrl) temp = await startEmbeddedPostgresTestDatabase("paperclip-satisfied-connections-");
|
||||
db = createDb(externalUrl ?? temp.connectionString);
|
||||
await db.insert(authUsers).values({ id: "connection-owner", name: "Owner", email: "owner@example.test", createdAt: new Date(), updatedAt: new Date() }).onConflictDoNothing();
|
||||
}, 20_000);
|
||||
beforeEach(async () => { await db.execute(sql`truncate table companies cascade`); });
|
||||
@@ -65,6 +62,19 @@ describePostgres("already available connection requests", () => {
|
||||
return run!;
|
||||
}
|
||||
|
||||
it("wakes governed response recovery only when automatic cleanup retires a connection card", async () => {
|
||||
const f = await seed();
|
||||
const notified = vi.fn();
|
||||
const unsubscribe = subscribeDeliveryWork(db, DELIVERY_QUEUES.nativeReviewResponse, notified);
|
||||
try {
|
||||
expect(await satisfiedConnectionIntentService(db).sweepPending()).toMatchObject({ satisfied: 1 });
|
||||
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, f.intent.id)))[0].status).toBe("expired");
|
||||
expect(notified).toHaveBeenCalledOnce();
|
||||
expect(await satisfiedConnectionIntentService(db).sweepPending()).toMatchObject({ satisfied: 0 });
|
||||
expect(notified).toHaveBeenCalledOnce();
|
||||
} finally { unsubscribe(); }
|
||||
});
|
||||
|
||||
it("notifies terminal confirmation recovery after existing-access retirement commits", async () => {
|
||||
const f = await seed();
|
||||
const notified = vi.fn();
|
||||
@@ -398,7 +408,7 @@ describePostgres("already available connection requests", () => {
|
||||
|
||||
it("records a system completion and one durable continuation with a one-connection pool", async () => {
|
||||
const f = await seed();
|
||||
const single = createDb(externalTestDatabaseUrl ?? temp.connectionString, { maxConnections: 1 });
|
||||
const single = createDb(externalUrl ?? temp.connectionString, { maxConnections: 1 });
|
||||
try {
|
||||
const outcomes = await Promise.all([satisfiedConnectionIntentService(single).sweepPending(), satisfiedConnectionIntentService(db).sweepPending()]);
|
||||
expect(outcomes.reduce((sum, outcome) => sum + outcome.satisfied, 0)).toBe(1);
|
||||
|
||||
@@ -116,6 +116,7 @@ const {
|
||||
reapOrphanedRuns: vi.fn(async () => ({ reaped: 0, runIds: [] })),
|
||||
registerScheduledRetryCancellationWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve() })),
|
||||
registerStoppedInstructionCollectionWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve() })),
|
||||
registerNativeReviewResponseWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve() })),
|
||||
registerNativeResponsePresentationWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve(), stop: vi.fn() })),
|
||||
registerNativePendingEffectWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve() })),
|
||||
registerUnsafeWorkspaceExportWork: vi.fn((_coordinator: unknown, _canRun: () => boolean) => ({ ready: Promise.resolve() })),
|
||||
@@ -570,6 +571,13 @@ describe("startServer feedback export wiring", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each([false, true])("registers governed-response recovery only when maintenance is enabled (%s)", async enabled => {
|
||||
loadConfigMock.mockReturnValue(buildTestConfig({ heartbeatSchedulerEnabled: enabled }));
|
||||
await startServer(); await settleStartupHousekeeping();
|
||||
expect(heartbeatServiceMock.registerNativeReviewResponseWork).toHaveBeenCalledTimes(enabled ? 1 : 0);
|
||||
if (enabled) expect(heartbeatServiceMock.registerNativeReviewResponseWork.mock.calls[0][1]()).toBe(true);
|
||||
});
|
||||
|
||||
it.each([false, true])("registers native pending effects only when maintenance is enabled (%s)", async enabled => {
|
||||
loadConfigMock.mockReturnValue(buildTestConfig({ heartbeatSchedulerEnabled: enabled }));
|
||||
await startServer(); await settleStartupHousekeeping();
|
||||
@@ -1017,7 +1025,7 @@ describe("startServer feedback export wiring", () => {
|
||||
intervalCallback?.();
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
expect(heartbeatServiceMock.sweepStaleIssueLocks).toHaveBeenCalledTimes(2);
|
||||
expect(heartbeatServiceMock.reapOrphanedRuns).toHaveBeenLastCalledWith({ staleThresholdMs: 300_000, skipScheduledRetryCancellation: true, skipCapturedInstructionCommit: true, skipInstructionDirectoryCleanup: true, skipStoppedInstructionCollection: true, skipCancelledWorkspacePreservation: true, skipObsoletePolicyReviews: true, skipAutomaticCompletionReviews: true, skipUnsafeWorkspaceExports: true, skipNativePendingEffects: true, skipNativeResponsePresentation: true });
|
||||
expect(heartbeatServiceMock.reapOrphanedRuns).toHaveBeenLastCalledWith({ staleThresholdMs: 300_000, skipScheduledRetryCancellation: true, skipCapturedInstructionCommit: true, skipInstructionDirectoryCleanup: true, skipStoppedInstructionCollection: true, skipCancelledWorkspacePreservation: true, skipObsoletePolicyReviews: true, skipAutomaticCompletionReviews: true, skipUnsafeWorkspaceExports: true, skipNativePendingEffects: true, skipNativeResponsePresentation: true, skipNativeReviewResponse: true });
|
||||
expect(heartbeatServiceMock.reconcileCostAccounting).toHaveBeenCalledTimes(1);
|
||||
expect(retiredDetector).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
|
||||
+3
-1
@@ -1825,6 +1825,8 @@ async function startServerWithDatabaseTeardown(
|
||||
() => !isIdleTaskDrainActive() && !heartbeatSchedulerStopped && !isWarmStandby()).ready;
|
||||
if (config.heartbeatSchedulerEnabled) await heartbeat.registerInstructionDirectoryCleanupWork(deliveryWork,
|
||||
() => !isIdleTaskDrainActive() && !heartbeatSchedulerStopped && !isWarmStandby()).ready;
|
||||
if (config.heartbeatSchedulerEnabled) await heartbeat.registerNativeReviewResponseWork(deliveryWork,
|
||||
() => !isIdleTaskDrainActive() && !heartbeatSchedulerStopped && !isWarmStandby()).ready;
|
||||
if (config.heartbeatSchedulerEnabled) {
|
||||
const worker = heartbeat.registerNativeResponsePresentationWork(deliveryWork,
|
||||
() => !isIdleTaskDrainActive() && !heartbeatSchedulerStopped && !isWarmStandby());
|
||||
@@ -1972,7 +1974,7 @@ async function startServerWithDatabaseTeardown(
|
||||
// Periodically reap orphaned runs (5-min staleness threshold) and make sure
|
||||
// persisted queued work is still being driven forward.
|
||||
runPeriodicRecovery(() => heartbeat
|
||||
.reapOrphanedRuns({ staleThresholdMs: 5 * 60 * 1000, skipScheduledRetryCancellation: true, skipCapturedInstructionCommit: true, skipInstructionDirectoryCleanup: true, skipStoppedInstructionCollection: true, skipCancelledWorkspacePreservation: true, skipObsoletePolicyReviews: true, skipAutomaticCompletionReviews: true, skipUnsafeWorkspaceExports: true, skipNativePendingEffects: true, skipNativeResponsePresentation: true })
|
||||
.reapOrphanedRuns({ staleThresholdMs: 5 * 60 * 1000, skipScheduledRetryCancellation: true, skipCapturedInstructionCommit: true, skipInstructionDirectoryCleanup: true, skipStoppedInstructionCollection: true, skipCancelledWorkspacePreservation: true, skipObsoletePolicyReviews: true, skipAutomaticCompletionReviews: true, skipUnsafeWorkspaceExports: true, skipNativePendingEffects: true, skipNativeResponsePresentation: true, skipNativeReviewResponse: true })
|
||||
.then(() => heartbeat.promoteDueScheduledRetries())
|
||||
.then(async (promotion) => {
|
||||
await heartbeat.resumeQueuedRuns();
|
||||
|
||||
@@ -52,7 +52,7 @@ export function approvalResolutionRecords(db: Db) {
|
||||
.where(and(eq(approvals.id, id), inArray(approvals.status, resolvableStatuses)))
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? null);
|
||||
});
|
||||
}, { approvalId: id });
|
||||
|
||||
if (updated) {
|
||||
return { approval: updated, applied: true };
|
||||
|
||||
@@ -1224,12 +1224,13 @@ describeEmbeddedPostgres("wake-queue postgres adapter", () => {
|
||||
expect(wakeRow?.payload).not.toHaveProperty("foo");
|
||||
});
|
||||
|
||||
it("scopes a real coalesced-input response wake to its active run after commit", async () => {
|
||||
it.each([DELIVERY_QUEUES.nativeResponsePresentation, DELIVERY_QUEUES.nativeReviewResponse])(
|
||||
"scopes a real coalesced-input %s response wake to its active run after commit", async queue => {
|
||||
const companyId = await seedCompany();
|
||||
const agentId = await seedAgent({ companyId });
|
||||
const runId = await seedRun({ companyId, agentId, status: "running" });
|
||||
const hints: Array<Record<string, unknown>> = [];
|
||||
const unsubscribe = subscribeDatabaseWork(db, DELIVERY_QUEUES.nativeResponsePresentation, (event, values) => {
|
||||
const unsubscribe = subscribeDatabaseWork(db, queue, (event, values) => {
|
||||
if (event === "settled") for (const value of values ?? []) hints.push(JSON.parse(value));
|
||||
});
|
||||
try {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { isAgentStatusInvokable } from "@paperclipai/shared";
|
||||
import { notifyNativeResponsePresentation } from "./native-response-presentation-notifications.js";
|
||||
import { notifyChatInboundWakeNotices } from "./chat-inbound-wake-notice-notifications.js";
|
||||
@@ -39,6 +40,7 @@ export function createAgentLifecycleEffects(_hooks: BudgetServiceHooks = {}): li
|
||||
async beforeStatusChange(tx, companyId, agentId, before, after) {
|
||||
if (!isAgentStatusInvokable(before) && isAgentStatusInvokable(after)) {
|
||||
await notifyNativeResponsePresentation(tx, { companyId, agentId });
|
||||
await notifyNativeReviewResponse(tx, { companyId, agentId });
|
||||
await notifyChatInboundWakeNotices(tx, { companyId });
|
||||
}
|
||||
},
|
||||
|
||||
@@ -1,16 +1,32 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import { approvals, issueApprovals, type Db } from "@paperclipai/db";
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export const notifyApprovalQueueWork = (transaction: object, companyId: string) => notifyDeliveryWork(transaction, DELIVERY_QUEUES.approvalQueueSeeding, companyId);
|
||||
|
||||
export async function notifyApprovalWork(transaction: object, companyId: string) {
|
||||
type ApprovalChangeScope = { approvalId?: string; issueIds?: string[] };
|
||||
|
||||
export async function notifyApprovalWork(transaction: object, companyId: string, scope?: ApprovalChangeScope) {
|
||||
await notifyDeliveryWork(transaction, DELIVERY_QUEUES.approvalRetention);
|
||||
await notifyApprovalQueueWork(transaction, companyId);
|
||||
const issueIds = new Set(scope?.issueIds);
|
||||
if (scope?.approvalId) {
|
||||
|
||||
|
||||
|
||||
await (transaction as Db).select({ id: approvals.id }).from(approvals)
|
||||
.where(and(eq(approvals.companyId, companyId), eq(approvals.id, scope.approvalId))).for("update");
|
||||
const links = await (transaction as Db).select({ issueId: issueApprovals.issueId }).from(issueApprovals)
|
||||
.where(and(eq(issueApprovals.companyId, companyId), eq(issueApprovals.approvalId, scope.approvalId)));
|
||||
for (const link of links) issueIds.add(link.issueId);
|
||||
}
|
||||
for (const issueId of issueIds) await notifyNativeReviewResponse(transaction, { companyId, issueId });
|
||||
}
|
||||
|
||||
export function writeApprovalChange<T>(db: Db, companyId: string, write: (transaction: Db) => Promise<T>): Promise<T> {
|
||||
export function writeApprovalChange<T>(db: Db, companyId: string, write: (transaction: Db) => Promise<T>, scope?: ApprovalChangeScope): Promise<T> {
|
||||
return db.transaction(async tx => {
|
||||
await notifyApprovalWork(tx, companyId);
|
||||
await notifyApprovalWork(tx, companyId, scope);
|
||||
return write(tx as unknown as Db);
|
||||
});
|
||||
}
|
||||
@@ -80,7 +80,7 @@ export function approvalRecords(db: Db) {
|
||||
.where(and(eq(approvals.id, id), inArray(approvals.status, resolvableStatuses)))
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? null);
|
||||
});
|
||||
}, { approvalId: id });
|
||||
return updated;
|
||||
},
|
||||
|
||||
@@ -102,7 +102,7 @@ export function approvalRecords(db: Db) {
|
||||
})
|
||||
.where(eq(approvals.id, id))
|
||||
.returning()
|
||||
.then((rows) => rows[0]));
|
||||
.then((rows) => rows[0]), { approvalId: id });
|
||||
},
|
||||
|
||||
resubmit: async (id: string, payload?: Record<string, unknown>) => {
|
||||
@@ -124,7 +124,7 @@ export function approvalRecords(db: Db) {
|
||||
})
|
||||
.where(eq(approvals.id, id))
|
||||
.returning()
|
||||
.then((rows) => rows[0]));
|
||||
.then((rows) => rows[0]), { approvalId: id });
|
||||
},
|
||||
|
||||
listComments: async (approvalId: string) => {
|
||||
|
||||
@@ -262,7 +262,7 @@ async function markApprovalStatus(
|
||||
if (!approvalId) return;
|
||||
const [approval] = await db.select({ companyId: approvals.companyId }).from(approvals).where(eq(approvals.id, approvalId));
|
||||
if (!approval) return;
|
||||
await notifyApprovalWork(db, approval.companyId);
|
||||
await notifyApprovalWork(db, approval.companyId, { approvalId });
|
||||
await db
|
||||
.update(approvals)
|
||||
.set({
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export type ChatResponseScope = { companyId: string; issueId?: string; deliveryId?: string; conversationId?: string };
|
||||
|
||||
export async function notifyChatResponseAvailability(tx: object, scope: ChatResponseScope) {
|
||||
if (scope.issueId) await notifyNativeReviewResponse(tx, { companyId: scope.companyId, issueId: scope.issueId });
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubTerminalResponses, JSON.stringify({ companyId: scope.companyId, issueId: scope.issueId, deliveryId: scope.deliveryId, conversationId: scope.conversationId }));
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubAcknowledgements, JSON.stringify({ companyId: scope.companyId, issueId: scope.issueId, deliveryId: scope.deliveryId, conversationId: scope.conversationId }));
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { notifyNativeResponsePresentation } from "./native-response-presentation-notifications.js";
|
||||
import { notifyChatResponseAvailability } from "./chat-response-availability.js";
|
||||
import { notifyChatConfirmationWork } from "./chat-confirmation-notifications.js";
|
||||
@@ -11,6 +12,8 @@ export type ChatRunMilestoneScope = { companyId: string; issueId?: string; runId
|
||||
|
||||
export async function notifyChatRunMilestones(tx: object, scope: ChatRunMilestoneScope) {
|
||||
if (scope.issueId || scope.runId) {
|
||||
await notifyNativeReviewResponse(tx, { companyId: scope.companyId, issueId: scope.issueId,
|
||||
runId: scope.issueId ? undefined : scope.runId });
|
||||
await notifyNativeResponsePresentation(tx, { companyId: scope.companyId, issueId: scope.issueId,
|
||||
runId: scope.issueId ? undefined : scope.runId });
|
||||
}
|
||||
@@ -30,4 +33,5 @@ export async function notifyChatRunMilestones(tx: object, scope: ChatRunMileston
|
||||
export async function notifyChatCompanyAuthorityChange(tx: object, scope: { companyId: string }) {
|
||||
await notifyChatRunMilestones(tx, scope);
|
||||
await notifyNativeResponsePresentation(tx, scope);
|
||||
await notifyNativeReviewResponse(tx, scope);
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db";
|
||||
|
||||
/** Topics for existing durable queues; register intent before their writes. */
|
||||
export const DELIVERY_QUEUES = {
|
||||
nativeReviewResponse: "native-review-response",
|
||||
nativeResponsePresentation: "native-response-presentation",
|
||||
nativePendingEffects: "native-pending-effects",
|
||||
unsafeWorkspaceExportRecovery: "unsafe-workspace-export-recovery",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { registerNativeReviewResponseWork } from "./native-runtime/native-review-response-work.js";
|
||||
import { registerNativeResponsePresentationWork } from "./native-runtime/native-response-presentation-work.js";
|
||||
import { registerNativePendingEffectWork } from "./native-runtime/native-pending-effect-work.js";
|
||||
import { registerUnsafeWorkspaceExportWork } from "./native-runtime/native-workspace-export-recovery.js";
|
||||
@@ -4090,6 +4091,8 @@ export function heartbeatService(
|
||||
triggerIssueMonitor,
|
||||
|
||||
reportRunActivity: clearDetachedRunWarning,
|
||||
registerNativeReviewResponseWork: (coordinator: Parameters<typeof registerNativeReviewResponseWork>[1], canRun: () => boolean) =>
|
||||
registerNativeReviewResponseWork(db, coordinator, () => canRun() && readTaskDrain(new Date()) === null),
|
||||
registerNativeResponsePresentationWork: (coordinator: Parameters<typeof registerNativeResponsePresentationWork>[1], canRun: () => boolean) =>
|
||||
registerNativeResponsePresentationWork(db, coordinator, () => canRun() && readTaskDrain(new Date()) === null),
|
||||
registerNativePendingEffectWork: (coordinator: Parameters<typeof registerNativePendingEffectWork>[1], canRun: () => boolean) =>
|
||||
|
||||
@@ -1947,7 +1947,7 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
|
||||
|
||||
let settledProcessesDiscovered = false;
|
||||
|
||||
async function reapOrphanedRuns(opts?: { staleThresholdMs?: number; skipScheduledRetryCancellation?: boolean; skipCapturedInstructionCommit?: boolean; skipInstructionDirectoryCleanup?: boolean; skipStoppedInstructionCollection?: boolean; skipCancelledWorkspacePreservation?: boolean; skipObsoletePolicyReviews?: boolean; skipAutomaticCompletionReviews?: boolean; skipUnsafeWorkspaceExports?: boolean; skipNativePendingEffects?: boolean; skipNativeResponsePresentation?: boolean }) {
|
||||
async function reapOrphanedRuns(opts?: { staleThresholdMs?: number; skipScheduledRetryCancellation?: boolean; skipCapturedInstructionCommit?: boolean; skipInstructionDirectoryCleanup?: boolean; skipStoppedInstructionCollection?: boolean; skipCancelledWorkspacePreservation?: boolean; skipObsoletePolicyReviews?: boolean; skipAutomaticCompletionReviews?: boolean; skipUnsafeWorkspaceExports?: boolean; skipNativePendingEffects?: boolean; skipNativeResponsePresentation?: boolean; skipNativeReviewResponse?: boolean }) {
|
||||
if (!opts?.skipCancelledWorkspacePreservation) await preserveCancelledNativeWorkspaces(db);
|
||||
const staleThresholdMs = opts?.staleThresholdMs ?? 0;
|
||||
const now = new Date();
|
||||
@@ -1970,6 +1970,7 @@ export function createHeartbeatRecovery(db: Db, dependencies: HeartbeatRecoveryD
|
||||
skipUnsafeWorkspaceExports: opts?.skipUnsafeWorkspaceExports,
|
||||
skipNativePendingEffects: opts?.skipNativePendingEffects,
|
||||
skipNativeResponsePresentation: opts?.skipNativeResponsePresentation,
|
||||
skipNativeReviewResponse: opts?.skipNativeReviewResponse,
|
||||
environmentRuntime,
|
||||
pluginWorkerManager: dependencies.pluginWorkerManager,
|
||||
onWorkspaceSettled: settleRecoveredNativeWorkspace,
|
||||
|
||||
@@ -117,7 +117,7 @@ export function issueApprovalService(db: Db) {
|
||||
linkedByAgentId: actor?.agentId ?? null,
|
||||
linkedByUserId: actor?.userId ?? null,
|
||||
})
|
||||
.onConflictDoNothing());
|
||||
.onConflictDoNothing(), { issueIds: [issueId] });
|
||||
|
||||
return db
|
||||
.select()
|
||||
@@ -130,7 +130,7 @@ export function issueApprovalService(db: Db) {
|
||||
const { issue } = await assertIssueAndApprovalSameCompany(issueId, approvalId);
|
||||
await writeApprovalChange(db, issue.companyId, tx => tx
|
||||
.delete(issueApprovals)
|
||||
.where(and(eq(issueApprovals.issueId, issueId), eq(issueApprovals.approvalId, approvalId))));
|
||||
.where(and(eq(issueApprovals.issueId, issueId), eq(issueApprovals.approvalId, approvalId))), { issueIds: [issueId] });
|
||||
},
|
||||
|
||||
linkManyForApproval: async (approvalId: string, issueIds: string[], actor?: LinkActor) => {
|
||||
@@ -169,7 +169,7 @@ export function issueApprovalService(db: Db) {
|
||||
linkedByUserId: actor?.userId ?? null,
|
||||
})),
|
||||
)
|
||||
.onConflictDoNothing());
|
||||
.onConflictDoNothing(), { issueIds: uniqueIssueIds });
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { notifyChatConfirmationResolution } from "./chat-confirmation-notifications.js";
|
||||
import { notifyChatRunMilestones } from "./chat-run-milestone-notifications.js";
|
||||
import { notifyToolReviewChange } from "./tool-review-work-notifications.js";
|
||||
@@ -1944,6 +1945,7 @@ async function expireStaleRequestConfirmationTarget(
|
||||
const expireInTransaction = async (tx: Db | any) => {
|
||||
await notifyChatConfirmationResolution(tx, args.row);
|
||||
if (args.row.kind === "request_confirmation") await notifyChatRunMilestones(tx, args.row);
|
||||
else await notifyNativeReviewResponse(tx, args.row);
|
||||
const [updated] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
@@ -2223,6 +2225,7 @@ export function issueThreadInteractionService(
|
||||
await assertInteractionRunWriteAllowed(tx as unknown as Db, args.issue, args.actor);
|
||||
await notifyChatConfirmationResolution(tx, args.current);
|
||||
if (args.current.kind === "request_confirmation") await notifyChatRunMilestones(tx, args.current);
|
||||
else await notifyNativeReviewResponse(tx, args.current);
|
||||
await args.mutationOptions?.beforeResolveInTransaction?.(tx);
|
||||
// Policy mutations and review transitions use the same issue-row lock,
|
||||
// so the authoritative review policy and requester are stable through
|
||||
@@ -2513,6 +2516,7 @@ export function issueThreadInteractionService(
|
||||
await assertInteractionRunWriteAllowed(tx as unknown as Db, args.issue, args.actor);
|
||||
await notifyChatConfirmationResolution(tx, args.current);
|
||||
if (args.current.kind === "request_confirmation") await notifyChatRunMilestones(tx, args.current);
|
||||
else await notifyNativeReviewResponse(tx, args.current);
|
||||
await args.mutationOptions?.beforeResolveInTransaction?.(tx);
|
||||
const issueContext = await tx
|
||||
.select({
|
||||
@@ -2905,6 +2909,7 @@ export function issueThreadInteractionService(
|
||||
const updated = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, current);
|
||||
await mutationOptions.beforeResolveInTransaction?.(tx);
|
||||
await notifyNativeReviewResponse(tx, current);
|
||||
const [row] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
@@ -3340,6 +3345,7 @@ export function issueThreadInteractionService(
|
||||
const updated = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(row.kind)) await notifyChatRunMilestones(tx, row);
|
||||
else await notifyNativeReviewResponse(tx, row);
|
||||
if (row.kind === "request_confirmation") await notifyToolProjectionForInteractions(tx, row.companyId, [row.id]);
|
||||
const [resolved] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
@@ -3427,6 +3433,7 @@ export function issueThreadInteractionService(
|
||||
const updated = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(row.kind)) await notifyChatRunMilestones(tx, row);
|
||||
else await notifyNativeReviewResponse(tx, row);
|
||||
if (row.kind === "request_confirmation") await notifyToolProjectionForInteractions(tx, row.companyId, [row.id]);
|
||||
const [updatedRow] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
@@ -4208,6 +4215,7 @@ export function issueThreadInteractionService(
|
||||
}
|
||||
}
|
||||
}
|
||||
await notifyNativeReviewResponse(tx, { companyId: issue.companyId, issueId: issue.id });
|
||||
const resolvedAt = new Date();
|
||||
const [claimed] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
@@ -4471,6 +4479,7 @@ export function issueThreadInteractionService(
|
||||
complete,
|
||||
items,
|
||||
} satisfies RequestItemVerdictsResult;
|
||||
if (complete) await notifyNativeReviewResponse(tx, { companyId: issue.companyId, issueId: issue.id });
|
||||
const [updated] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
@@ -4537,6 +4546,7 @@ export function issueThreadInteractionService(
|
||||
await notifyChatConfirmationResolution(tx, current);
|
||||
await mutationOptions.beforeResolveInTransaction?.(tx);
|
||||
await assertInteractionRunWriteAllowed(tx as unknown as Db, issue, actor);
|
||||
await notifyNativeReviewResponse(tx, { companyId: issue.companyId, issueId: issue.id });
|
||||
const rows = await tx.update(issueThreadInteractions)
|
||||
.set({
|
||||
status: "rejected",
|
||||
@@ -4628,6 +4638,7 @@ export function issueThreadInteractionService(
|
||||
const updated = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(row.kind)) await notifyChatRunMilestones(tx, row);
|
||||
else await notifyNativeReviewResponse(tx, row);
|
||||
if (row.kind === "connection_intent") {
|
||||
await tx
|
||||
.delete(toolOauthStates)
|
||||
@@ -4805,6 +4816,7 @@ export function issueThreadInteractionService(
|
||||
const updatedInteractions = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, sampleConfirmationRow);
|
||||
if (commentRows.some(row => row.kind === "request_confirmation")) await notifyChatRunMilestones(tx, sampleConfirmationRow);
|
||||
else await notifyNativeReviewResponse(tx, sampleConfirmationRow);
|
||||
await notifyToolProjectionForInteractions(tx, issue.companyId, confirmationRowIds);
|
||||
const updatedRows = await tx
|
||||
.update(issueThreadInteractions)
|
||||
@@ -4840,6 +4852,7 @@ export function issueThreadInteractionService(
|
||||
|
||||
for (const row of itemVerdictRows) {
|
||||
const updated = await db.transaction(async tx => {
|
||||
await notifyNativeReviewResponse(tx, row);
|
||||
const [retired] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
@@ -4865,6 +4878,7 @@ export function issueThreadInteractionService(
|
||||
|
||||
for (const row of connectionIntentRows) {
|
||||
const [updated] = await db.transaction(async (tx) => {
|
||||
await notifyNativeReviewResponse(tx, row);
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
await tx
|
||||
.delete(toolOauthStates)
|
||||
@@ -4953,6 +4967,7 @@ export function issueThreadInteractionService(
|
||||
const updated = await db.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
if (row.kind === "request_confirmation") await notifyChatRunMilestones(tx, row);
|
||||
else await notifyNativeReviewResponse(tx, row);
|
||||
const [resolved] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
@@ -5016,6 +5031,7 @@ export function issueThreadInteractionService(
|
||||
? "the AI account"
|
||||
: `the ${payload?.serviceName?.trim() || "requested"} connection`;
|
||||
const updated = await db.transaction(async tx => {
|
||||
await notifyNativeReviewResponse(tx, { companyId: issue.companyId, issueId: issue.id });
|
||||
await notifyChatConfirmationResolution(tx, { companyId: issue.companyId, issueId: issue.id, kind: "connection_intent" });
|
||||
const [resolved] = await tx
|
||||
.update(issueThreadInteractions)
|
||||
@@ -5077,6 +5093,7 @@ export function issueThreadInteractionService(
|
||||
.transaction(async (tx) => {
|
||||
await notifyChatConfirmationResolution(tx, row);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(row.kind)) await notifyChatRunMilestones(tx, row);
|
||||
else await notifyNativeReviewResponse(tx, row);
|
||||
if (row.kind === "connection_intent") {
|
||||
await tx
|
||||
.delete(toolOauthStates)
|
||||
@@ -5169,6 +5186,7 @@ export function issueThreadInteractionService(
|
||||
await assertInteractionRunWriteAllowed(tx as unknown as Db, issue, actor);
|
||||
await notifyChatConfirmationResolution(tx, current);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(current.kind)) await notifyChatRunMilestones(tx, current);
|
||||
else await notifyNativeReviewResponse(tx, current);
|
||||
await resolveLinkedToolActionRequests(tx, current, {
|
||||
status: "cancelled",
|
||||
fromStatuses: ["pending", "approved"],
|
||||
@@ -5431,6 +5449,7 @@ export function issueThreadInteractionService(
|
||||
await assertInteractionRunWriteAllowed(tx as unknown as Db, issue, actor);
|
||||
await notifyChatConfirmationResolution(tx, current);
|
||||
if (["ask_user_questions", "request_confirmation"].includes(current.kind)) await notifyChatRunMilestones(tx, current);
|
||||
else await notifyNativeReviewResponse(tx, current);
|
||||
await resolveLinkedToolActionRequests(tx, current, {
|
||||
status: "cancelled",
|
||||
fromStatuses: ["pending", "approved"],
|
||||
|
||||
@@ -11453,7 +11453,9 @@ export function issueService(db: Db) {
|
||||
await notifyDecisionTargetWork(tx, `issue:${id}`);
|
||||
}
|
||||
if ((patch.status !== undefined && patch.status !== receiptExisting.status) ||
|
||||
(patch.assigneeAgentId !== undefined && patch.assigneeAgentId !== receiptExisting.assigneeAgentId)) {
|
||||
(patch.assigneeAgentId !== undefined && patch.assigneeAgentId !== receiptExisting.assigneeAgentId) ||
|
||||
(patch.executionState !== undefined && parseObject(receiptExisting.executionState).status === "pending" &&
|
||||
parseObject(patch.executionState).status !== "pending")) {
|
||||
await notifyChatResponseAvailability(tx, { companyId: receiptExisting.companyId, issueId: id });
|
||||
}
|
||||
if (patch.status !== undefined && patch.status !== receiptExisting.status) {
|
||||
|
||||
@@ -1,10 +1,14 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { notifyNativeResponsePresentation } from "./native-response-presentation-notifications.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export type NativeFinalizationWorkScope = { companyId: string; runId?: string; issueId?: string };
|
||||
|
||||
export async function notifyNativeFinalizationWork(transaction: object, scope: NativeFinalizationWorkScope) {
|
||||
if (scope.issueId || scope.runId) await notifyNativeResponsePresentation(transaction, scope);
|
||||
if (scope.issueId || scope.runId) {
|
||||
await notifyNativeResponsePresentation(transaction, scope);
|
||||
await notifyNativeReviewResponse(transaction, scope);
|
||||
}
|
||||
await notifyDeliveryWork(transaction, DELIVERY_QUEUES.nativePendingEffects,
|
||||
JSON.stringify({ companyId: scope.companyId, runId: scope.runId }));
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export type NativeReviewResponseScope = { companyId: string; issueId?: string; runId?: string; agentId?: string };
|
||||
|
||||
export function notifyNativeReviewResponse(tx: object, scope: NativeReviewResponseScope) {
|
||||
return notifyDeliveryWork(tx, DELIVERY_QUEUES.nativeReviewResponse,
|
||||
JSON.stringify({ companyId: scope.companyId, issueId: scope.issueId, runId: scope.runId, agentId: scope.agentId }));
|
||||
}
|
||||
@@ -1,3 +1,6 @@
|
||||
import { notifyChatRunMilestones } from "../chat-run-milestone-notifications.js";
|
||||
import { notifyNativeFinalizationWork } from "../native-finalization-notifications.js";
|
||||
import { accessService } from "../access.js";
|
||||
import { initializeRunIdentity } from "../run-identity.js";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
@@ -28,6 +31,7 @@ import {
|
||||
completionContracts,
|
||||
closeRegisteredClients,
|
||||
createDb,
|
||||
subscribeDatabaseWork,
|
||||
heartbeatRuns,
|
||||
issueComments,
|
||||
issueAttachments,
|
||||
@@ -41,6 +45,7 @@ import {
|
||||
statusDecisions,
|
||||
toolApplications,
|
||||
toolConnections,
|
||||
toolOauthStates,
|
||||
chatVoiceSessions, chatVoiceReplies, chatVoiceToolCalls,
|
||||
workspaceOperations,
|
||||
} from "@paperclipai/db";
|
||||
@@ -51,7 +56,17 @@ import type {
|
||||
} from "../../vendor/paperclip-runner/index.js";
|
||||
|
||||
import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import { finalizeNativeRun } from "./native-run-finalizer.js";
|
||||
import { finalizeNativeRun, recoverCommittedNativeReviewResponse } from "./native-run-finalizer.js";
|
||||
import { commitNativeStatusDecision } from "./status-decision-committer.js";
|
||||
import { configureAgentLifecycle, createAgentLifecycle, processAgentLifecycle } from "../agent-lifecycle.js";
|
||||
import { createOAuthStateStore } from "../tool-access/oauth/state.js";
|
||||
import { approvalRecords } from "../approvals.js";
|
||||
import { issueApprovalService } from "../issue-approvals.js";
|
||||
import { NativeRunCoordinatorStore } from "./native-run-coordinator-store.js";
|
||||
import { nativeReviewResponseWork } from "./native-review-response-work.js";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { idleWorkSnapshot } from "../task-admission.js";
|
||||
import { nativeCompletionFeedback } from "./native-completion-feedback.js";
|
||||
import { hasPublishedGitHubReview, readInheritedGitHubReviewWait } from "../chat-channels/github/review-wait.js";
|
||||
import * as chatWaitAuthority from "./chat-attachment-reuse.js";
|
||||
@@ -422,6 +437,10 @@ describe("native external-chat response wait", () => {
|
||||
userId,
|
||||
commentId,
|
||||
principalId,
|
||||
storeBinding: { companyId, issueId, runId, agentId, normalizedSessionId: sessionId,
|
||||
runnerSourceInstanceId: runnerInstanceId, completionContractId: contractId,
|
||||
completionContractSha256: contractSha256, completionContractRevision: "external-chat-wait-v1",
|
||||
completionContractCriterionIds: ["response"] },
|
||||
};
|
||||
}
|
||||
|
||||
@@ -2869,6 +2888,653 @@ describe("native external-chat response wait", () => {
|
||||
return run!.resultJson!;
|
||||
}
|
||||
|
||||
async function seedRecoverableReviewResponse() {
|
||||
const fixture = await seedWaitTurn();
|
||||
const gate = await seedPriorCompletionReview(fixture);
|
||||
await finalizeNativeRun({ db, runId: fixture.runId, workspaceFinalizeStatus: "succeeded", projectRunStatus: false });
|
||||
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() })
|
||||
.where(eq(heartbeatRuns.id, fixture.runId));
|
||||
const [coordinator] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, fixture.runId));
|
||||
expect(coordinator!.phase).toBe("committed");
|
||||
const input = { companyId: fixture.companyId, issueId: fixture.issueId, runId: fixture.runId,
|
||||
decisionId: coordinator!.decisionId!, resultId: coordinator!.resultId!, assessmentId: coordinator!.assessmentId!, onlyWhenDue: true };
|
||||
return { ...fixture, gate, input };
|
||||
}
|
||||
|
||||
it.each(["reconciler", "finalizer"] as const)("defers periodic governed-response %s repair to its registered worker", async path => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(heartbeatRuns).set({ resultJson: sql`${heartbeatRuns.resultJson} - 'externalChatReviewPresentation'` })
|
||||
.where(eq(heartbeatRuns.id, fixture.runId));
|
||||
if (path === "reconciler") await reconcileNativeFinalizations(db, [fixture.runId], { skipNativeReviewResponse: true });
|
||||
else await finalizeNativeRun({ db, runId: fixture.runId, workspaceFinalizeStatus: "succeeded",
|
||||
projectRunStatus: true, skipNativeReviewResponse: true });
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, fixture.runId));
|
||||
expect(run.resultJson?.externalChatReviewPresentation).toBeUndefined();
|
||||
const work = nativeReviewResponseWork(db);
|
||||
const [entry] = await work.load({ companyId: fixture.companyId, runId: fixture.runId });
|
||||
expect(entry?.id).toBe(fixture.runId);
|
||||
expect(await work.check(entry)).toBeNull();
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("recovers scheduled governed responses once under concurrent callers and preserves deleted receipts", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const work = nativeReviewResponseWork(db);
|
||||
const [entry] = await work.load({ companyId: fixture.companyId });
|
||||
expect(entry?.id).toBe(fixture.runId);
|
||||
expect(await Promise.all([recoverCommittedNativeReviewResponse(db, fixture.input),
|
||||
recoverCommittedNativeReviewResponse(db, fixture.input)])).toEqual(expect.arrayContaining([true, false]));
|
||||
const comments = await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
expect(comments).toHaveLength(1);
|
||||
expect(await db.select().from(chatPublications).where(eq(chatPublications.commentId, comments[0].id))).toHaveLength(1);
|
||||
expect(await work.load({ companyId: fixture.companyId })).toEqual([]);
|
||||
await db.update(issueComments).set({ deletedAt: new Date() }).where(eq(issueComments.id, comments[0].id));
|
||||
expect(await work.load({ companyId: fixture.companyId })).toEqual([]);
|
||||
expect(await recoverCommittedNativeReviewResponse(db, fixture.input)).toBe(false);
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("lets status replay finish while governed response recovery waits for its coordinator", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [issue] = await db.select().from(issues).where(eq(issues.id, fixture.issueId));
|
||||
const [decision] = await db.select().from(statusDecisions).where(eq(statusDecisions.id, fixture.input.decisionId));
|
||||
let recovery: Promise<{ value?: boolean; error?: unknown }> | undefined;
|
||||
try {
|
||||
await db.transaction(async tx => {
|
||||
await tx.select().from(nativeRunFinalizations)
|
||||
.where(eq(nativeRunFinalizations.runId, fixture.runId)).for("update");
|
||||
const [backend] = await tx.execute<{ pid: number }>(sql`select pg_backend_pid() as pid`);
|
||||
recovery = recoverCommittedNativeReviewResponse(db, fixture.input)
|
||||
.then(value => ({ value }), error => ({ error }));
|
||||
await vi.waitFor(async () => {
|
||||
const [waiting] = await db.execute<{ count: number }>(sql`
|
||||
select count(*)::int as count from pg_stat_activity
|
||||
where datname = current_database() and wait_event_type = 'Lock'
|
||||
and ${backend.pid} = any(pg_blocking_pids(pid))`);
|
||||
expect(waiting.count).toBe(1);
|
||||
});
|
||||
|
||||
|
||||
await tx.execute(sql`set local lock_timeout = '500ms'`);
|
||||
const replay = await commitNativeStatusDecision({
|
||||
db: tx as unknown as typeof db, companyId: fixture.companyId,
|
||||
issueId: fixture.issueId, runId: fixture.runId,
|
||||
assessmentId: fixture.input.assessmentId,
|
||||
priorStatus: issue.status, priorStatusVersion: issue.statusVersion,
|
||||
priorDecisionId: issue.lastStatusDecisionId,
|
||||
decision: decision.decisionJson as unknown as Parameters<typeof commitNativeStatusDecision>[0]["decision"],
|
||||
});
|
||||
expect(replay.replayed).toBe(true);
|
||||
});
|
||||
} finally {
|
||||
if (recovery) {
|
||||
const result = await recovery;
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(result.value).toBe(true);
|
||||
}
|
||||
}
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(1);
|
||||
expect(await recoverCommittedNativeReviewResponse(db, fixture.input)).toBe(false);
|
||||
});
|
||||
|
||||
it.each(["future", "claim"])("defers scheduled governed responses behind a new %s under the owner lock", async kind => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const work = nativeReviewResponseWork(db);
|
||||
const [entry] = await work.load({ companyId: fixture.companyId });
|
||||
const at = new Date(Date.now() + 60_000);
|
||||
const transaction = db.transaction.bind(db);
|
||||
const intercept = vi.spyOn(db, "transaction").mockImplementationOnce((async (callback, options) => {
|
||||
await transaction(async tx => {
|
||||
await tx.update(nativeRunFinalizations).set(kind === "future" ? { nextAttemptAt: at }
|
||||
: { leaseOwner: "another-finalizer", leaseExpiresAt: at }).where(eq(nativeRunFinalizations.runId, fixture.runId));
|
||||
});
|
||||
return transaction(callback, options);
|
||||
}) as typeof db.transaction);
|
||||
try {
|
||||
expect(await work.check(entry)).toEqual(kind === "future" ? { ...entry, dueAt: at.getTime() } : null);
|
||||
} finally { intercept.mockRestore(); }
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
await db.update(nativeRunFinalizations).set({ nextAttemptAt: null, leaseOwner: null, leaseExpiresAt: null })
|
||||
.where(eq(nativeRunFinalizations.runId, fixture.runId));
|
||||
expect(await recoverCommittedNativeReviewResponse(db, fixture.input)).toBe(true);
|
||||
});
|
||||
|
||||
it("propagates scheduled governed response contention and rolls back its proof restoration", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(heartbeatRuns).set({ resultJson: sql`${heartbeatRuns.resultJson} - 'externalChatReviewPresentation'` })
|
||||
.where(eq(heartbeatRuns.id, fixture.runId));
|
||||
let release!: () => void, observed!: () => void;
|
||||
const hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const locked = new Promise<void>(resolve => { observed = resolve; });
|
||||
const holder = db.transaction(async tx => {
|
||||
await tx.select().from(chatEndpoints).where(eq(chatEndpoints.id, fixture.endpointId)).for("update");
|
||||
observed(); await hold;
|
||||
});
|
||||
await locked;
|
||||
try {
|
||||
await expect(recoverCommittedNativeReviewResponse(db, fixture.input)).rejects.toBeDefined();
|
||||
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, fixture.runId));
|
||||
expect(run.resultJson?.externalChatReviewPresentation).toBeUndefined();
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
} finally { release(); await holder; }
|
||||
expect(await recoverCommittedNativeReviewResponse(db, fixture.input)).toBe(true);
|
||||
});
|
||||
|
||||
it.each(["gate", "agent", "principal", "company"])("leaves scheduled governed response authority with its owner after %s changes", async kind => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
if (kind === "gate") await db.update(issueThreadInteractions).set({ status: "accepted" }).where(eq(issueThreadInteractions.id, fixture.gate.id));
|
||||
if (kind === "agent") await db.update(agents).set({ status: "paused" }).where(eq(agents.id, fixture.agentId));
|
||||
if (kind === "principal") await db.update(companyMemberships).set({ status: "suspended" })
|
||||
.where(and(eq(companyMemberships.companyId, fixture.companyId), eq(companyMemberships.principalId, fixture.userId)));
|
||||
expect(await recoverCommittedNativeReviewResponse(db, { ...fixture.input,
|
||||
...(kind === "company" ? { companyId: randomUUID() } : {}) })).toBe(false);
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
});
|
||||
|
||||
it.each(["milestone", "finalization"] as const)("ignores repeated unscoped routine %s hints without reconstructing dormant governed responses", async source => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(companyMemberships).set({ status: "suspended" }).where(and(
|
||||
eq(companyMemberships.companyId, fixture.companyId), eq(companyMemberships.principalId, fixture.userId)));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 10_000 })
|
||||
.toEqual({ active: 0, next: null });
|
||||
const debug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => {
|
||||
if (query.startsWith("select") && query.includes("native_run_finalizations")) queries.push(query);
|
||||
};
|
||||
try {
|
||||
for (let i = 0; i < 3; i++) {
|
||||
await db.transaction(tx => source === "milestone" ? notifyChatRunMilestones(tx, { companyId: fixture.companyId })
|
||||
: notifyNativeFinalizationWork(tx, { companyId: fixture.companyId }));
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 5000 })
|
||||
.toEqual({ active: 0, next: null });
|
||||
}
|
||||
expect(queries).toEqual([]);
|
||||
} finally { db.$client.options.debug = debug; }
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("wakes dormant governed responses through the real company membership restoration owner", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [member] = await db.select().from(companyMemberships).where(and(
|
||||
eq(companyMemberships.companyId, fixture.companyId), eq(companyMemberships.principalId, fixture.userId)));
|
||||
await db.update(companyMemberships).set({ status: "suspended" }).where(eq(companyMemberships.id, member.id));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 10_000 })
|
||||
.toEqual({ active: 0, next: null });
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
await accessService(db).updateMember(fixture.companyId, member.id, { status: "active" });
|
||||
await expect.poll(async () => ({ count: (await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).length,
|
||||
active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }), { timeout: 5000 })
|
||||
.toEqual({ count: 1, active: 0, next: null });
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it.each(["approve", "reject", "cancel", "unlink"] as const)("reactivates dormant governed responses after approval %s", async action => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [approval] = await db.insert(approvals).values({ companyId: fixture.companyId,
|
||||
type: "hire_agent", status: "pending", payload: {} }).returning();
|
||||
await issueApprovalService(db).link(fixture.issueId, approval.id);
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
if (action === "unlink") await issueApprovalService(db).unlink(fixture.issueId, approval.id);
|
||||
else if (action === "cancel") await approvalRecords(db).cancel(approval.id);
|
||||
else await approvalRecords(db).resolveApproval(approval.id, action === "approve" ? "approved" : "rejected", fixture.userId, null);
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it.each([
|
||||
["resolver", "single"], ["resolver", "bulk"],
|
||||
["linker", "single"], ["linker", "bulk"],
|
||||
] as const)("retains a settled task wake when %s starts first during concurrent %s approval linking", async (first, kind) => {
|
||||
const fixture = await seedWaitTurn();
|
||||
const [approval] = await db.insert(approvals).values({ companyId: fixture.companyId,
|
||||
type: "request_board_approval", status: "pending", payload: {} }).returning();
|
||||
const observed: string[] = [];
|
||||
const reads: Promise<void>[] = [];
|
||||
const unsubscribe = subscribeDatabaseWork(db, DELIVERY_QUEUES.nativeReviewResponse, (event, hints) => {
|
||||
if (event !== "settled" || ![...(hints ?? [])].some(hint => {
|
||||
const scope = JSON.parse(hint);
|
||||
return scope.companyId === fixture.companyId && scope.issueId === fixture.issueId;
|
||||
})) return;
|
||||
reads.push(db.select({ status: approvals.status }).from(issueApprovals)
|
||||
.innerJoin(approvals, eq(issueApprovals.approvalId, approvals.id))
|
||||
.where(and(eq(issueApprovals.issueId, fixture.issueId), eq(approvals.id, approval.id)))
|
||||
.then(rows => { for (const row of rows) observed.push(row.status); }));
|
||||
});
|
||||
let release!: () => void, ready!: () => void;
|
||||
const hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const held = new Promise<void>(resolve => { ready = resolve; });
|
||||
let holderPid = 0, resolverPid = 0, linkerPid = 0;
|
||||
let linked = false, resolved = false;
|
||||
const link = async (tx: typeof db) => {
|
||||
if (kind === "single") await issueApprovalService(tx).link(fixture.issueId, approval.id);
|
||||
else await issueApprovalService(tx).linkManyForApproval(approval.id, [fixture.issueId]);
|
||||
};
|
||||
const holder = db.transaction(async tx => {
|
||||
const [backend] = await tx.execute(sql`select pg_backend_pid() as pid`);
|
||||
holderPid = Number(backend.pid);
|
||||
if (first === "resolver") {
|
||||
|
||||
|
||||
await tx.execute(sql`lock table approvals in share mode`);
|
||||
} else {
|
||||
await link(tx as unknown as typeof db);
|
||||
}
|
||||
ready();
|
||||
await hold;
|
||||
});
|
||||
let resolver: Promise<unknown> | undefined, linker: Promise<unknown> | undefined;
|
||||
const blocked = async (pid: number, by: number) => {
|
||||
if (!pid || !by) return false;
|
||||
const [row] = await db.execute(sql`select ${by} = any(pg_blocking_pids(${pid})) as blocked`);
|
||||
return row.blocked === true;
|
||||
};
|
||||
try {
|
||||
await held;
|
||||
resolver = db.transaction(async tx => {
|
||||
const [backend] = await tx.execute(sql`select pg_backend_pid() as pid`);
|
||||
resolverPid = Number(backend.pid);
|
||||
await approvalRecords(tx as unknown as typeof db).resolveApproval(approval.id, "approved", fixture.userId, null);
|
||||
}).then(() => { resolved = true; });
|
||||
await expect.poll(async () => resolved || await blocked(resolverPid, holderPid), { timeout: 5000 }).toBe(true);
|
||||
if (first === "resolver") {
|
||||
linker = db.transaction(async tx => {
|
||||
const [backend] = await tx.execute(sql`select pg_backend_pid() as pid`);
|
||||
linkerPid = Number(backend.pid);
|
||||
await link(tx as unknown as typeof db);
|
||||
}).then(() => { linked = true; });
|
||||
await expect.poll(async () => linked || await blocked(linkerPid, resolverPid), { timeout: 5000 }).toBe(true);
|
||||
}
|
||||
|
||||
|
||||
await Promise.all(reads);
|
||||
release();
|
||||
await Promise.all([holder, resolver, linker]);
|
||||
await Promise.all(reads);
|
||||
expect(observed).toContain("approved");
|
||||
} finally {
|
||||
release();
|
||||
await Promise.allSettled([holder, resolver, linker]);
|
||||
await Promise.all(reads);
|
||||
unsubscribe();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["cancel", "unlink"] as const)("keeps governed response approval %s behind outer commit and scoped to linked tasks", async action => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [approval, unrelated] = await db.insert(approvals).values([
|
||||
{ companyId: fixture.companyId, type: "hire_agent", status: "pending", payload: {} },
|
||||
{ companyId: fixture.companyId, type: "hire_agent", status: "pending", payload: {} },
|
||||
]).returning();
|
||||
await issueApprovalService(db).link(fixture.issueId, approval.id);
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
const mutate = async (tx: typeof db) => {
|
||||
if (action === "cancel") await approvalRecords(tx).cancel(approval.id);
|
||||
else await issueApprovalService(tx).unlink(fixture.issueId, approval.id);
|
||||
};
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
const previousDebug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => { if (query.includes('"native_run_finalizations"')) queries.push(query); };
|
||||
try {
|
||||
await approvalRecords(db).cancel(unrelated.id);
|
||||
await expect.poll(() => idleWorkSnapshot().active).toBe(0);
|
||||
expect(queries).toEqual([]);
|
||||
} finally { db.$client.options.debug = previousDebug; }
|
||||
const rollback = new Error("rollback governed approval change");
|
||||
await expect(db.transaction(async tx => {
|
||||
await mutate(tx as unknown as typeof db);
|
||||
expect(await comments()).toHaveLength(0);
|
||||
throw rollback;
|
||||
})).rejects.toBe(rollback);
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
await db.transaction(async tx => {
|
||||
await mutate(tx as unknown as typeof db);
|
||||
expect(await comments()).toHaveLength(0);
|
||||
});
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("wakes dormant governed response recovery after real agent resume", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(agents).set({ status: "paused", lifecycleState: "paused", lifecycleHolds: ["manual"], lifecycleOperation: null })
|
||||
.where(eq(agents.id, fixture.agentId));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const lifecycle = configureAgentLifecycle(db, {
|
||||
requiredPluginIds: async () => [], runHost: async () => "complete", runPlugin: async () => "complete",
|
||||
});
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(0);
|
||||
await createAgentLifecycle(db).resumeAgent(fixture.agentId);
|
||||
await processAgentLifecycle(db, fixture.agentId);
|
||||
await expect.poll(async () => (await db.select().from(agents).where(eq(agents.id, fixture.agentId)))[0].status).toBe("idle");
|
||||
await expect.poll(async () => (await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).length,
|
||||
{ timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await lifecycle.stop(); await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it.each(["skip", "withdraw", "reject_suggestions", "complete_verdicts", "decline_connection"] as const)(
|
||||
"reactivates governed responses after competing interaction %s", async action => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const service = issueThreadInteractionService(db);
|
||||
const issue = { id: fixture.issueId, companyId: fixture.companyId };
|
||||
const actor = { userId: fixture.userId };
|
||||
const competing = action === "decline_connection"
|
||||
? await service.createConnectionIntent(issue, { sourceRunId: fixture.runId,
|
||||
addresseeUserId: fixture.userId, idempotencyKey: `review-competing:${randomUUID()}`,
|
||||
payload: { version: 1, serviceSlug: "notion", serviceName: "Notion", serviceLogoUrl: null,
|
||||
requestingAgentId: fixture.agentId, requestingAgentName: "Agent", phase: "requested" } })
|
||||
: action === "complete_verdicts"
|
||||
? await service.create(issue, { kind: "request_item_verdicts", payload: { version: 1,
|
||||
prompt: "Review this item", items: [{ id: "one", label: "One" }] } }, actor)
|
||||
: await service.create(issue, { kind: "suggest_tasks", payload: { version: 1,
|
||||
tasks: [{ clientKey: "child", title: "A suggested task" }] } }, actor);
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
if (action === "skip") await service.skipInteraction(issue, competing.id, {}, actor);
|
||||
else if (action === "withdraw") await service.withdrawInteraction(issue, competing.id, {}, actor);
|
||||
else if (action === "reject_suggestions") await service.rejectInteraction(issue, competing.id, {}, actor);
|
||||
else if (action === "complete_verdicts") await service.submitItemVerdicts(issue, competing.id,
|
||||
{ verdicts: [{ id: "one", verdict: "approve" }] }, actor);
|
||||
else await service.resolveConnectionIntent(issue, competing.id, { version: 1, outcome: "declined" }, actor);
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it.each(["status", "assignment"] as const)("preserves governed response policy through task %s changes", async change => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(issues).set(change === "status" ? { status: "blocked" } : { assigneeAgentId: null })
|
||||
.where(eq(issues.id, fixture.issueId));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
if (change === "assignment") {
|
||||
await expect(issueService(db).update(fixture.issueId, { assigneeAgentId: fixture.agentId }))
|
||||
.rejects.toThrow("Agent assignment cannot change while this task is bound to an external channel");
|
||||
expect(await comments()).toHaveLength(0);
|
||||
} else {
|
||||
await issueService(db).update(fixture.issueId, { status: "in_review" });
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
}
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("reactivates governed responses after OAuth confirmation denial", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [connection] = await db.select().from(toolConnections).where(eq(toolConnections.companyId, fixture.companyId));
|
||||
const competing = await issueThreadInteractionService(db).create({ id: fixture.issueId, companyId: fixture.companyId },
|
||||
{ kind: "request_confirmation", payload: { version: 1, prompt: "Connect your account", target: {
|
||||
type: "custom", key: `connection:${connection.id}`, revisionId: randomUUID(), label: "Connect account" } } },
|
||||
{ userId: fixture.userId });
|
||||
const [state] = await db.insert(toolOauthStates).values({ state: randomUUID(), companyId: fixture.companyId,
|
||||
connectionId: connection.id, codeVerifier: "test-verifier", issueId: fixture.issueId, interactionId: competing.id,
|
||||
subjectUserId: fixture.userId, expiresAt: new Date(Date.now() + 60_000) }).returning();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
const rollback = new Error("rollback OAuth denial");
|
||||
await expect(db.transaction(async tx => {
|
||||
await createOAuthStateStore(tx as unknown as typeof db).rejectPendingOAuthInteraction(state,
|
||||
{ actorType: "user", actorId: fixture.userId });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
throw rollback;
|
||||
})).rejects.toBe(rollback);
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, competing.id)))[0].status).toBe("pending");
|
||||
await db.transaction(async tx => {
|
||||
await createOAuthStateStore(tx as unknown as typeof db).rejectPendingOAuthInteraction(state,
|
||||
{ actorType: "user", actorId: fixture.userId });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
});
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("reactivates governed responses when pending execution policy is cleared without changing task status", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
await db.update(issues).set({ executionState: { status: "pending", currentStageId: randomUUID(),
|
||||
currentStageIndex: 0, currentStageType: "review", currentParticipant: { type: "user", userId: fixture.userId, agentId: null },
|
||||
returnAssignee: { type: "agent", agentId: fixture.agentId, userId: null }, reviewRequest: null,
|
||||
completedStageIds: [], lastDecisionId: null, lastDecisionOutcome: null, monitor: null, changesRequestedCount: 0 } })
|
||||
.where(eq(issues.id, fixture.issueId));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
await issueService(db).updateForCompany(fixture.issueId, fixture.companyId, { executionState: null });
|
||||
expect((await db.select().from(issues).where(eq(issues.id, fixture.issueId)))[0].status).toBe("in_review");
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("wakes governed responses from the real coordinator retry release", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const store = new NativeRunCoordinatorStore(db, fixture.storeBinding);
|
||||
await store.claimFinalization({ leaseOwner: "review-response-fixture", leaseTtlMs: 30_000 });
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
await store.markFinalizationRetry({ leaseOwner: "review-response-fixture", failureCode: "review_response_retry", retryAfterMs: 1000 });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
await expect.poll(async () => (await comments()).length, { timeout: 5_000 }).toBe(1);
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("retains the first governed-response wake through idle drain without passive SQL", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const [approval] = await db.insert(approvals).values({ companyId: fixture.companyId,
|
||||
type: "hire_agent", status: "pending", payload: {} }).returning();
|
||||
await issueApprovalService(db).link(fixture.issueId, approval.id);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
let canRun = true;
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => canRun,
|
||||
canReconcile: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => canRun);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
canRun = false;
|
||||
await approvalRecords(db).cancel(approval.id);
|
||||
await expect.poll(() => idleWorkSnapshot().active).toBe(0);
|
||||
expect(await comments()).toHaveLength(0);
|
||||
expect(coordinator.nextWakeAt()).not.toBeNull();
|
||||
const previousDebug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => { queries.push(query); };
|
||||
try { await vi.advanceTimersByTimeAsync(120_000); expect(queries).toEqual([]); }
|
||||
finally { db.$client.options.debug = previousDebug; }
|
||||
canRun = true; worker.wake();
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(await comments()).toHaveLength(1);
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
}, { timeout: 10_000 });
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it("retains failed governed-response recovery across drain and retries after resume", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
let release!: () => void, observed!: () => void;
|
||||
const hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const locked = new Promise<void>(resolve => { observed = resolve; });
|
||||
const holder = db.transaction(async tx => {
|
||||
await tx.select().from(chatEndpoints).where(eq(chatEndpoints.id, fixture.endpointId)).for("update");
|
||||
observed(); await hold;
|
||||
});
|
||||
await locked;
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
let canRun = true;
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => canRun,
|
||||
canReconcile: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => canRun);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ errors: errors.length, active: idleWorkSnapshot().active }),
|
||||
{ timeout: 10_000 }).toEqual({ errors: 1, active: 0 });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
expect(coordinator.nextWakeAt()).toBeGreaterThan(Date.now());
|
||||
canRun = false;
|
||||
worker.wake();
|
||||
await vi.waitFor(() => expect(idleWorkSnapshot().active).toBe(0));
|
||||
const previousDebug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => { queries.push(query); };
|
||||
try { await vi.advanceTimersByTimeAsync(120_000); expect(queries).toEqual([]); }
|
||||
finally { db.$client.options.debug = previousDebug; }
|
||||
release(); await holder;
|
||||
canRun = true; worker.wake();
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(await comments()).toHaveLength(1);
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
}, { timeout: 10_000 });
|
||||
expect(errors).toHaveLength(1);
|
||||
} finally { release(); await holder; await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it("reconstructs the governed-response deadline after shutdown without early presentation", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const store = new NativeRunCoordinatorStore(db, fixture.storeBinding);
|
||||
await store.claimFinalization({ leaseOwner: "review-future", leaseTtlMs: 30_000 });
|
||||
await store.markFinalizationRetry({ leaseOwner: "review-future", failureCode: "review_fixture", retryAfterMs: 60_000 });
|
||||
const [state] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, fixture.runId));
|
||||
const at = state.nextAttemptAt!.getTime();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
const comments = () => db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId));
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: at });
|
||||
await coordinator.stop();
|
||||
expect(coordinator.nextWakeAt()).toBeNull();
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const previousDebug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => { queries.push(query); };
|
||||
try { await vi.advanceTimersByTimeAsync(30_000); expect(queries).toEqual([]); }
|
||||
finally { db.$client.options.debug = previousDebug; }
|
||||
const resumed = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
const restarted = nativeReviewResponseWork(db).register(resumed, () => true);
|
||||
await restarted.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: resumed.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: at });
|
||||
expect(await comments()).toHaveLength(0);
|
||||
await vi.advanceTimersByTimeAsync(Math.max(0, at - Date.now()));
|
||||
await vi.waitFor(async () => {
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(await comments()).toHaveLength(1);
|
||||
expect(idleWorkSnapshot().active).toBe(0);
|
||||
expect(resumed.nextWakeAt()).toBeNull();
|
||||
}, { timeout: 10_000 });
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await resumed.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it("reconstructs scheduled governed responses at startup and leaves their history quiet", async () => {
|
||||
const fixture = await seedRecoverableReviewResponse();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
const worker = nativeReviewResponseWork(db).register(coordinator, () => true);
|
||||
try {
|
||||
await worker.ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 10_000 }).toEqual({ active: 0, next: null });
|
||||
expect(await db.select().from(issueComments).where(eq(issueComments.createdByRunId, fixture.runId))).toHaveLength(1);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const previousDebug = db.$client.options.debug;
|
||||
const queries: string[] = [];
|
||||
db.$client.options.debug = (_connection, query) => { queries.push(query); };
|
||||
try { await vi.advanceTimersByTimeAsync(120_000); expect(queries).toEqual([]); }
|
||||
finally { db.$client.options.debug = previousDebug; }
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it.each(["telegram", "discord"] as const)(
|
||||
"presents the exact %s response without resolving a pre-existing native completion review",
|
||||
async (provider) => {
|
||||
|
||||
@@ -500,6 +500,7 @@ export async function restoreNativeChatReviewPresentationInTransaction(
|
||||
decisionId: string;
|
||||
resultId: string;
|
||||
assessmentId: string;
|
||||
onlyWhenDue?: boolean;
|
||||
},
|
||||
): Promise<boolean> {
|
||||
const [coordinator] = await db
|
||||
@@ -519,6 +520,8 @@ export async function restoreNativeChatReviewPresentationInTransaction(
|
||||
.for("update")
|
||||
.limit(1);
|
||||
if (!coordinator) return false;
|
||||
if (input.onlyWhenDue && (coordinator.leaseOwner !== null
|
||||
|| coordinator.nextAttemptAt !== null && coordinator.nextAttemptAt.getTime() > Date.now())) return false;
|
||||
const [decision] = await db
|
||||
.select()
|
||||
.from(statusDecisions)
|
||||
|
||||
@@ -645,6 +645,7 @@ type NativeReconciliationOptions = {
|
||||
skipUnsafeWorkspaceExports?: boolean;
|
||||
skipNativePendingEffects?: boolean;
|
||||
skipNativeResponsePresentation?: boolean;
|
||||
skipNativeReviewResponse?: boolean;
|
||||
pluginWorkerManager?: WorkspaceWorkers;
|
||||
environmentRuntime?: EnvironmentRuntimeService;
|
||||
onWorkspaceSettled?: (input: {
|
||||
@@ -875,7 +876,8 @@ async function reconcileNativeFinalizationsPass(
|
||||
row.decisionId &&
|
||||
row.resultId &&
|
||||
row.assessmentId &&
|
||||
currentDecisionJson.externalChatReviewPresentation
|
||||
currentDecisionJson.externalChatReviewPresentation &&
|
||||
!options.skipNativeReviewResponse
|
||||
) {
|
||||
// A later chat turn may retain the same pending review. Recover the
|
||||
// earlier response independently before status reconciliation skips
|
||||
@@ -1125,6 +1127,7 @@ async function reconcileNativeFinalizationsPass(
|
||||
workspaceFinalizeStatus,
|
||||
projectRunStatus: true,
|
||||
skipNativeResponsePresentation: options.skipNativeResponsePresentation,
|
||||
skipNativeReviewResponse: options.skipNativeReviewResponse,
|
||||
preserveProviderAttempt: Boolean(
|
||||
readNativeWorkspaceSyncReference(
|
||||
row.nativeWorkspaceSync,
|
||||
@@ -1184,6 +1187,7 @@ async function reconcileNativeFinalizationsPass(
|
||||
workspaceFinalizeStatus: barrier.status as "succeeded" | "failed",
|
||||
projectRunStatus: true,
|
||||
skipNativeResponsePresentation: options.skipNativeResponsePresentation,
|
||||
skipNativeReviewResponse: options.skipNativeReviewResponse,
|
||||
preserveProviderAttempt: Boolean(
|
||||
readNativeWorkspaceSyncReference(
|
||||
row.nativeWorkspaceSync,
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import { and, asc, eq, gt, isNotNull, isNull, notExists, sql } from "drizzle-orm";
|
||||
import { heartbeatRuns, issueComments, nativeRunFinalizations, statusDecisions, type Db } from "@paperclipai/db";
|
||||
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import type { NativeReviewResponseScope } from "../native-review-response-notifications.js";
|
||||
import { registerScopedDeadlineWork } from "../scoped-deadline-work.js";
|
||||
import { recoverCommittedNativeReviewResponse } from "./native-run-finalizer.js";
|
||||
|
||||
export function registerNativeReviewResponseWork(db: Db,
|
||||
coordinator: ReturnType<typeof createDeliveryWorkCoordinator>, canRun: () => boolean) {
|
||||
return nativeReviewResponseWork(db).register(coordinator, canRun);
|
||||
}
|
||||
|
||||
type Entry = { id: string; companyId: string; issueId: string; agentId: string; dueAt: number };
|
||||
|
||||
export function nativeReviewResponseWork(db: Db) {
|
||||
function candidates(scope?: NativeReviewResponseScope, after?: string, runId?: string) {
|
||||
return db.select({ id: nativeRunFinalizations.runId, companyId: nativeRunFinalizations.companyId,
|
||||
issueId: nativeRunFinalizations.issueId, agentId: heartbeatRuns.agentId,
|
||||
decisionId: nativeRunFinalizations.decisionId, resultId: nativeRunFinalizations.resultId,
|
||||
assessmentId: nativeRunFinalizations.assessmentId, nextAttemptAt: nativeRunFinalizations.nextAttemptAt,
|
||||
}).from(nativeRunFinalizations)
|
||||
.innerJoin(heartbeatRuns, and(eq(heartbeatRuns.id, nativeRunFinalizations.runId),
|
||||
eq(heartbeatRuns.companyId, nativeRunFinalizations.companyId), eq(heartbeatRuns.nativeIssueId, nativeRunFinalizations.issueId)))
|
||||
.innerJoin(statusDecisions, and(eq(statusDecisions.id, nativeRunFinalizations.decisionId),
|
||||
eq(statusDecisions.companyId, nativeRunFinalizations.companyId), eq(statusDecisions.issueId, nativeRunFinalizations.issueId),
|
||||
eq(statusDecisions.runId, nativeRunFinalizations.runId), eq(statusDecisions.assessmentId, nativeRunFinalizations.assessmentId)))
|
||||
.where(and(eq(nativeRunFinalizations.phase, "committed"), isNull(nativeRunFinalizations.leaseOwner),
|
||||
isNotNull(nativeRunFinalizations.resultId), isNotNull(nativeRunFinalizations.assessmentId),
|
||||
eq(heartbeatRuns.runtimeMode, "native"), eq(heartbeatRuns.status, "succeeded"),
|
||||
eq(statusDecisions.reasonCode, "governed_response_waiting"), eq(statusDecisions.applicationState, "applied"),
|
||||
sql`${statusDecisions.decisionJson}->'externalChatReviewPresentation'->>'schema' = 'paperclip.chat_review_response_presentation.v1'`,
|
||||
notExists(db.select({ id: issueComments.id }).from(issueComments).where(and(
|
||||
eq(issueComments.companyId, nativeRunFinalizations.companyId), eq(issueComments.issueId, nativeRunFinalizations.issueId),
|
||||
eq(issueComments.createdByRunId, nativeRunFinalizations.runId), eq(issueComments.authorAgentId, heartbeatRuns.agentId),
|
||||
eq(issueComments.authorType, "agent"), sql`${issueComments.metadata}->>'authorizationReason' = 'allow_chat_run_presentation'`,
|
||||
sql`${heartbeatRuns.resultJson}->'externalChatReviewPresentation'->>'decisionId' = ${nativeRunFinalizations.decisionId}::text`,
|
||||
))),
|
||||
scope ? eq(nativeRunFinalizations.companyId, scope.companyId) : undefined,
|
||||
scope?.issueId ? eq(nativeRunFinalizations.issueId, scope.issueId) : undefined,
|
||||
scope?.runId ? eq(nativeRunFinalizations.runId, scope.runId) : undefined,
|
||||
scope?.agentId ? eq(heartbeatRuns.agentId, scope.agentId) : undefined,
|
||||
runId ? eq(nativeRunFinalizations.runId, runId) : undefined,
|
||||
after ? gt(nativeRunFinalizations.runId, after) : undefined,
|
||||
)).orderBy(asc(nativeRunFinalizations.runId)).limit(100);
|
||||
}
|
||||
async function load(scope?: NativeReviewResponseScope, signal?: AbortSignal, canRun = () => true): Promise<Entry[]> {
|
||||
const entries: Entry[] = [];
|
||||
let after: string | undefined;
|
||||
while (canRun() && !signal?.aborted) {
|
||||
const rows = await candidates(scope, after);
|
||||
for (const row of rows) entries.push({ id: row.id, companyId: row.companyId,
|
||||
issueId: row.issueId, agentId: row.agentId, dueAt: row.nextAttemptAt?.getTime() ?? 0 });
|
||||
if (rows.length < 100) return entries;
|
||||
after = rows[rows.length - 1].id;
|
||||
}
|
||||
throw new Error("Governed review response discovery stopped");
|
||||
}
|
||||
async function check(entry: Entry, signal?: AbortSignal): Promise<Entry | null> {
|
||||
if (signal?.aborted) throw new Error("Governed review response recovery stopped");
|
||||
const scope = { companyId: entry.companyId, issueId: entry.issueId };
|
||||
const [current] = await candidates(scope, undefined, entry.id);
|
||||
if (!current) return null;
|
||||
if (current.nextAttemptAt && current.nextAttemptAt.getTime() > Date.now()) return { ...entry, dueAt: current.nextAttemptAt.getTime() };
|
||||
if (signal?.aborted) throw new Error("Governed review response recovery stopped");
|
||||
const recoveryStartedAt = Date.now();
|
||||
await recoverCommittedNativeReviewResponse(db, { ...scope, runId: entry.id,
|
||||
decisionId: current.decisionId!, resultId: current.resultId!, assessmentId: current.assessmentId!, onlyWhenDue: true });
|
||||
const [remaining] = await candidates(scope, undefined, entry.id);
|
||||
return remaining?.nextAttemptAt && remaining.nextAttemptAt.getTime() > recoveryStartedAt
|
||||
? { ...entry, dueAt: remaining.nextAttemptAt.getTime() } : null;
|
||||
}
|
||||
function register(coordinator: ReturnType<typeof createDeliveryWorkCoordinator>, canRun: () => boolean) {
|
||||
return registerScopedDeadlineWork(coordinator, {
|
||||
queue: DELIVERY_QUEUES.nativeReviewResponse,
|
||||
decodeScope: (hint: string): NativeReviewResponseScope | undefined => hint === "*" ? undefined : JSON.parse(hint),
|
||||
matches: (entry: Entry, scope?: NativeReviewResponseScope) => !scope || entry.companyId === scope.companyId
|
||||
&& (!scope.issueId || entry.issueId === scope.issueId) && (!scope.runId || entry.id === scope.runId)
|
||||
&& (!scope.agentId || entry.agentId === scope.agentId),
|
||||
load: (scope, signal) => load(scope, signal, canRun),
|
||||
check: (entry, signal) => {
|
||||
if (!canRun()) throw new Error("Governed review response recovery paused");
|
||||
return check(entry, signal);
|
||||
},
|
||||
}, canRun);
|
||||
}
|
||||
return { load, check, register };
|
||||
}
|
||||
@@ -723,65 +723,62 @@ async function projectCommittedRun(input: {
|
||||
}
|
||||
}
|
||||
|
||||
async function materializeCommittedReviewResponse(db: Db, runId: string) {
|
||||
try {
|
||||
const [run] = await db
|
||||
.select({ id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||||
nativeIssueId: heartbeatRuns.nativeIssueId, resultJson: heartbeatRuns.resultJson })
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, runId))
|
||||
.limit(1);
|
||||
if (!run?.nativeIssueId || !run.resultJson?.externalChatReviewPresentation)
|
||||
return;
|
||||
const decisionId = record(
|
||||
run.resultJson.externalChatReviewPresentation,
|
||||
).decisionId;
|
||||
if (
|
||||
typeof decisionId !== "string" ||
|
||||
(await hasMaterializedNativeReviewResponse(db, {
|
||||
companyId: run.companyId,
|
||||
issueId: run.nativeIssueId,
|
||||
runId,
|
||||
decisionId,
|
||||
}))
|
||||
)
|
||||
return;
|
||||
const summary = record(run.resultJson.nativeResult).summary;
|
||||
if (
|
||||
typeof summary !== "string" ||
|
||||
!(await authorizeNativeChatReviewPresentation(db, {
|
||||
companyId: run.companyId,
|
||||
issueId: run.nativeIssueId,
|
||||
runId,
|
||||
resultJson: run.resultJson,
|
||||
}))
|
||||
)
|
||||
return;
|
||||
// The coordinator transaction is already committed. addComment owns a
|
||||
// fresh issue transaction, rechecks the proof and carries selected files;
|
||||
// its same-run/text and publication keys make recovery replay idempotent.
|
||||
await issueService(db).addComment(
|
||||
run.nativeIssueId,
|
||||
summary,
|
||||
{ agentId: run.agentId, runId },
|
||||
{
|
||||
authorizationReason: "allow_chat_run_presentation",
|
||||
},
|
||||
);
|
||||
publishChatPublicationCommitSignal({
|
||||
async function materializeNativeReviewResponse(db: Db, runId: string, dbOrTx: Db = db) {
|
||||
const [run] = await dbOrTx
|
||||
.select({ id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
||||
nativeIssueId: heartbeatRuns.nativeIssueId, resultJson: heartbeatRuns.resultJson })
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, runId))
|
||||
.limit(1);
|
||||
if (!run?.nativeIssueId || !run.resultJson?.externalChatReviewPresentation)
|
||||
return null;
|
||||
const decisionId = record(
|
||||
run.resultJson.externalChatReviewPresentation,
|
||||
).decisionId;
|
||||
if (
|
||||
typeof decisionId !== "string" ||
|
||||
(await hasMaterializedNativeReviewResponse(dbOrTx, {
|
||||
companyId: run.companyId,
|
||||
issueId: run.nativeIssueId,
|
||||
runId: run.id,
|
||||
agentId: run.agentId,
|
||||
eventType: "run.presentation.resolved",
|
||||
});
|
||||
runId,
|
||||
decisionId,
|
||||
}))
|
||||
)
|
||||
return null;
|
||||
const summary = record(run.resultJson.nativeResult).summary;
|
||||
if (
|
||||
typeof summary !== "string" ||
|
||||
!(await authorizeNativeChatReviewPresentation(dbOrTx, {
|
||||
companyId: run.companyId,
|
||||
issueId: run.nativeIssueId,
|
||||
runId,
|
||||
resultJson: run.resultJson,
|
||||
}))
|
||||
)
|
||||
return null;
|
||||
await issueService(db).addComment(
|
||||
run.nativeIssueId,
|
||||
summary,
|
||||
{ agentId: run.agentId, runId },
|
||||
{
|
||||
authorizationReason: "allow_chat_run_presentation",
|
||||
},
|
||||
dbOrTx,
|
||||
);
|
||||
return run;
|
||||
}
|
||||
|
||||
function publishNativeReviewResponse(run: Pick<typeof heartbeatRuns.$inferSelect, "id" | "companyId" | "agentId" | "nativeIssueId">) {
|
||||
publishChatPublicationCommitSignal({ companyId: run.companyId, issueId: run.nativeIssueId!,
|
||||
runId: run.id, agentId: run.agentId, eventType: "run.presentation.resolved" });
|
||||
}
|
||||
|
||||
async function materializeCommittedReviewResponse(db: Db, runId: string) {
|
||||
try {
|
||||
const run = await materializeNativeReviewResponse(db, runId);
|
||||
if (run) publishNativeReviewResponse(run);
|
||||
} catch (error) {
|
||||
// The durable committed proof remains retryable by the next reconciler
|
||||
// sweep. Presentation failure must not reclassify a successful native run.
|
||||
logger.warn(
|
||||
{ runId, error },
|
||||
"Committed chat review response is awaiting presentation retry",
|
||||
);
|
||||
logger.warn({ runId, error }, "Committed chat review response is awaiting presentation retry");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1183,6 +1180,28 @@ export async function repairCommittedNativeReviewResponse(
|
||||
);
|
||||
if (restored) await materializeCommittedReviewResponse(db, input.runId);
|
||||
}
|
||||
export async function recoverCommittedNativeReviewResponse(db: Db, input: {
|
||||
companyId: string; issueId: string; runId: string; decisionId: string;
|
||||
resultId: string; assessmentId: string; onlyWhenDue?: boolean;
|
||||
}): Promise<boolean> {
|
||||
const materialized = await db.transaction(async transaction => {
|
||||
const tx = transaction as unknown as Db;
|
||||
|
||||
|
||||
const [coordinator] = await tx.select({ id: nativeRunFinalizations.runId }).from(nativeRunFinalizations)
|
||||
.where(and(eq(nativeRunFinalizations.companyId, input.companyId),
|
||||
eq(nativeRunFinalizations.issueId, input.issueId), eq(nativeRunFinalizations.runId, input.runId))).for("update");
|
||||
if (!coordinator) return null;
|
||||
const [issue] = await tx.select({ id: issues.id }).from(issues)
|
||||
.where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.issueId))).for("update");
|
||||
if (!issue || await hasMaterializedNativeReviewResponse(tx, input)) return null;
|
||||
if (!await restoreNativeChatReviewPresentationInTransaction(tx, input)) return null;
|
||||
return materializeNativeReviewResponse(db, input.runId, tx);
|
||||
});
|
||||
if (materialized) publishNativeReviewResponse(materialized);
|
||||
return materialized !== null;
|
||||
}
|
||||
|
||||
export async function finalizeNativeRun(input: {
|
||||
db: Db;
|
||||
runId: string;
|
||||
@@ -1190,6 +1209,7 @@ export async function finalizeNativeRun(input: {
|
||||
/** Reconciliation owns terminal run projection; the live heartbeat does it afterward. */
|
||||
projectRunStatus?: boolean;
|
||||
skipNativeResponsePresentation?: boolean;
|
||||
skipNativeReviewResponse?: boolean;
|
||||
/** Workspace-only replay must not consume the provider recovery budget. */
|
||||
preserveProviderAttempt?: boolean;
|
||||
failpoint?: NativeStatusCommitFailpoint;
|
||||
@@ -1214,14 +1234,14 @@ export async function finalizeNativeRun(input: {
|
||||
const coordinator = claim.coordinator;
|
||||
if (!claim.leaseOwner && coordinator.phase === "committed") {
|
||||
if (isNativeRunnerOwnershipHeld(admittedRun)) return coordinator;
|
||||
const presentationAlreadyMaterialized = coordinator.decisionId
|
||||
const presentationAlreadyMaterialized = input.skipNativeReviewResponse || (coordinator.decisionId
|
||||
? await hasMaterializedNativeReviewResponse(input.db, {
|
||||
companyId: admittedRun.companyId,
|
||||
issueId: coordinator.issueId,
|
||||
runId: admittedRun.id,
|
||||
decisionId: coordinator.decisionId,
|
||||
})
|
||||
: false;
|
||||
: false);
|
||||
if (
|
||||
!presentationAlreadyMaterialized &&
|
||||
coordinator.decisionId &&
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "./native-review-response-notifications.js";
|
||||
import { notifyChatConfirmationResolution } from "./chat-confirmation-notifications.js";
|
||||
import { enqueueTerminalIssueInteractionChatPublications } from "./chat-interaction-publications.js";
|
||||
import { connectionManagementService } from "./connection-management.js";
|
||||
@@ -155,6 +156,7 @@ export function satisfiedConnectionIntentService(db: Db) {
|
||||
}
|
||||
const connection = await findSatisfiedToolConnection(txDb, task, intent);
|
||||
if (!connection) return false;
|
||||
await notifyNativeReviewResponse(tx, candidate);
|
||||
const now = new Date();
|
||||
// Retire the obsolete request without inventing a human approval.
|
||||
await notifyChatConfirmationResolution(tx, intent);
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "../../native-review-response-notifications.js";
|
||||
import { notifyToolConnectionHealth } from "../../tool-connection-health-notifications.js";
|
||||
import { managedConnectorProfile } from "./client-settings.js";
|
||||
import type { ActorInfo } from "../types.js";
|
||||
@@ -408,18 +409,19 @@ export function createCloudOAuthCallback(db: Db, dependencies: Dependencies) {
|
||||
personalCredential || agentCredential ? credentialSecretRefs : [],
|
||||
tx,
|
||||
);
|
||||
const linkedInteractionKind = stateRow.interactionId
|
||||
const linkedInteraction = stateRow.interactionId
|
||||
? await tx
|
||||
.select({ kind: issueThreadInteractions.kind })
|
||||
.select({ kind: issueThreadInteractions.kind, issueId: issueThreadInteractions.issueId })
|
||||
.from(issueThreadInteractions)
|
||||
.where(eq(issueThreadInteractions.id, stateRow.interactionId))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0]?.kind ?? null)
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
if (
|
||||
stateRow.interactionId &&
|
||||
linkedInteractionKind === "request_confirmation"
|
||||
linkedInteraction?.kind === "request_confirmation"
|
||||
) {
|
||||
await notifyNativeReviewResponse(tx, { companyId: connection.companyId, issueId: linkedInteraction.issueId });
|
||||
await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "../../native-review-response-notifications.js";
|
||||
import { notifyToolConnectionHealth } from "../../tool-connection-health-notifications.js";
|
||||
import { credentialScope } from "./client-settings.js";
|
||||
import { withoutOAuthRefreshLease } from "./refresh-leases.js";
|
||||
@@ -374,18 +375,19 @@ export function createDirectOAuthCallback(db: Db, dependencies: Dependencies) {
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(toolApplications.id, connection.applicationId));
|
||||
const linkedInteractionKind = stateRow.interactionId
|
||||
const linkedInteraction = stateRow.interactionId
|
||||
? await tx
|
||||
.select({ kind: issueThreadInteractions.kind })
|
||||
.select({ kind: issueThreadInteractions.kind, issueId: issueThreadInteractions.issueId })
|
||||
.from(issueThreadInteractions)
|
||||
.where(eq(issueThreadInteractions.id, stateRow.interactionId))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0]?.kind ?? null)
|
||||
.then((rows) => rows[0] ?? null)
|
||||
: null;
|
||||
if (
|
||||
stateRow.interactionId &&
|
||||
linkedInteractionKind === "request_confirmation"
|
||||
linkedInteraction?.kind === "request_confirmation"
|
||||
) {
|
||||
await notifyNativeReviewResponse(tx, { companyId: connection.companyId, issueId: linkedInteraction.issueId });
|
||||
await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyNativeReviewResponse } from "../../native-review-response-notifications.js";
|
||||
import type { ActorInfo } from "../types.js";
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
@@ -124,17 +125,19 @@ async function rejectPendingOAuthInteraction(
|
||||
stateRow: typeof toolOauthStates.$inferSelect,
|
||||
actor: ActorInfo | undefined,
|
||||
) {
|
||||
if (!stateRow.interactionId) return;
|
||||
const interactionId = stateRow.interactionId;
|
||||
if (!interactionId) return;
|
||||
const now = new Date();
|
||||
const linked = await db
|
||||
.select({
|
||||
kind: issueThreadInteractions.kind,
|
||||
issueId: issueThreadInteractions.issueId,
|
||||
payload: issueThreadInteractions.payload,
|
||||
})
|
||||
.from(issueThreadInteractions)
|
||||
.where(
|
||||
and(
|
||||
eq(issueThreadInteractions.id, stateRow.interactionId),
|
||||
eq(issueThreadInteractions.id, interactionId),
|
||||
eq(issueThreadInteractions.companyId, stateRow.companyId),
|
||||
),
|
||||
)
|
||||
@@ -152,36 +155,40 @@ async function rejectPendingOAuthInteraction(
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(issueThreadInteractions.id, stateRow.interactionId),
|
||||
eq(issueThreadInteractions.id, interactionId),
|
||||
eq(issueThreadInteractions.companyId, stateRow.companyId),
|
||||
eq(issueThreadInteractions.status, "pending"),
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
await db
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
status: "rejected",
|
||||
result: {
|
||||
version: 1,
|
||||
outcome: "rejected",
|
||||
// Paperclip's own words: the provider's explanation is untrusted and
|
||||
// this reason is rendered in the thread (PAP-17108).
|
||||
reason:
|
||||
"Authorization was declined or cancelled in the provider's window",
|
||||
},
|
||||
resolvedByUserId: actor?.actorType === "user" ? actor.actorId : null,
|
||||
resolvedAt: now,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(issueThreadInteractions.id, stateRow.interactionId),
|
||||
eq(issueThreadInteractions.companyId, stateRow.companyId),
|
||||
eq(issueThreadInteractions.status, "pending"),
|
||||
),
|
||||
);
|
||||
if (!linked) return;
|
||||
await db.transaction(async tx => {
|
||||
await notifyNativeReviewResponse(tx, { companyId: stateRow.companyId, issueId: linked.issueId });
|
||||
await tx
|
||||
.update(issueThreadInteractions)
|
||||
.set({
|
||||
status: "rejected",
|
||||
result: {
|
||||
version: 1,
|
||||
outcome: "rejected",
|
||||
|
||||
|
||||
reason:
|
||||
"Authorization was declined or cancelled in the provider's window",
|
||||
},
|
||||
resolvedByUserId: actor?.actorType === "user" ? actor.actorId : null,
|
||||
resolvedAt: now,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(issueThreadInteractions.id, interactionId),
|
||||
eq(issueThreadInteractions.companyId, stateRow.companyId),
|
||||
eq(issueThreadInteractions.status, "pending"),
|
||||
),
|
||||
);
|
||||
});
|
||||
}
|
||||
return { peekOAuthState, validateOAuthState, consumeOAuthState, rejectPendingOAuthInteraction };
|
||||
}
|
||||
|
||||
@@ -2689,7 +2689,7 @@ export function createToolGatewayService(
|
||||
approvalId: approval.id,
|
||||
linkedByAgentId: input.session.agentId,
|
||||
})
|
||||
.onConflictDoNothing());
|
||||
.onConflictDoNothing(), { issueIds: [approvalIssueId] });
|
||||
}
|
||||
|
||||
const interaction = await interactions.create(
|
||||
|
||||
Reference in new issue
Block a user