mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 02:32:07 +01:00
fix(server): schedule GitHub review checks from committed changes (#17662)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/213/changes/paperclip/213 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub review checks report the state of admitted agent work. > - The chat maintenance loop searches for check work even when none remains. > - Existing action rows already record unfinished publication and retry deadlines. > - Committed changes can wake those rows through the shared coordinator. > - An unresolved check must still detect a remote head change when a webhook is missed. > - This pull request schedules only outstanding checks and leaves settled history quiet. ## Linked Issues or Issue Description **What happened?** The GitHub chat maintenance callback repeatedly queries review-check actions. Empty queues and completed history still incur database reads. Local state changes wait for the maintenance callback. **Expected behavior** Committed check creation, review binding, run changes, and publication-lease release wake the worker. Existing retry and remote-head deadlines apply only to unfinished checks. A settled worker makes no recurring database or provider calls. **Steps to reproduce** Configure a GitHub bot with no outstanding checks, or finish all of its checks. Leave the instance idle. The old maintenance callback continues to call the check service. Refs #17661, the merged terminal-response parent. Related #17656 migrates response acknowledgements. Related #16385 changes branch-policy admission; this PR preserves that separate policy boundary. Late stale-head projection repair remains a separate follow-up slice. ## What Changed - Move the existing check-state reader into the GitHub provider owner. Preserve current authority, run selection, assessment ownership, and check replacement behavior. - Register a scoped worker with the existing coordinator and scheduler. Existing received and retryable-failed actions remain durable truth. No new table is added. - Signal check enqueue, task review receipts, manual review insertion/binding, run lifecycle changes, and publication-lease release through existing transaction ownership. - Preserve the saved failure backoff, eight-attempt policy, and competing lease expiry. Keep five-second remote-head reconciliation only while a received check remains unresolved. - Check admission before each HTTP request. Shutdown leaves durable work recoverable without spending a provider failure attempt. - Remove check dispatch from the periodic GitHub callback. Keep the separate late-projection repair and the manual combined recovery method compatible. - Add an idempotent concurrent partial index for unfinished check selection. Retain the newest five generated snapshots. - Add eleven real-database worker cases and startup wiring coverage. Document producer coverage and the external reconciliation limit. ## Verification Current reviewed source: `2bba08403b195bf646eda48992b46d6f5cebf756`. - Normal merges preserve the complete review-check patch, the landed terminal-response parent, and current master changes. No landed SQL migration or journal entry changes. - At `6995f84b240a14a7b8660e1cf095b3aa1e1e3168`, **331 distinct tests pass**: 264 complete GitHub workflow cases, 30 completion cases, eight issue-tree producer cases, 16 login-cleanup cases, and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. Login cleanup uses its real embedded PostgreSQL fixture. - Combined-master acceptance: **277 distinct tests pass** at `9c538aed1251fd7f7cecb829a0ecbce6110fb1ed`: 264 complete GitHub workflow cases and 13 startup cases. The GitHub command filters out 1,485 unrelated cases. A new PostgreSQL database receives all migrations through 0402.. - Migration `0402_ancient_malice.sql` preserves the original partial index definition and concurrent idempotent creation. All 400 prior migration receipts stay unchanged during upgrade. One new receipt is added. Replay adds no receipt. The index is ready and valid on fresh and upgraded PostgreSQL databases. The five newest snapshots are retained. - A rollback-only query probe adds 10,000 processed actions and one pending action. The actual parameterized candidate predicate selects the one pending row through the partial index, with two shared buffers and 0.007 ms execution time. This measures candidate discovery, not all lease or provenance reads. - Database build, server typecheck and build, module boundaries, migration safety and migration order pass. Audit: [GitHub review-check scheduling](https://github.com/paperclipai/paperclip/blob/2bba08403b195bf646eda48992b46d6f5cebf756/doc/plans/2026-10-25-github-review-check-sleep-audit.md). - Fresh checks and Greptile review are required for this head. Prior scores do not count. These checks do not prove deployed host suspension or fleet savings. ## Risks - Unresolved queued/running checks still make remote-head requests at the existing five-second deadline. GitHub webhook receipt does not prove future pushes cannot be missed. This PR does not claim active reviews become network-idle. - Run and configuration hints rebuild outstanding checks for the affected company. Enqueue, review, and lease hints narrow to the affected endpoint/repository/PR. Startup and scope reconstruction read existing pending rows; they are not constant-cost operations. - Delivery admission failures remain discoverable through the unresolved check's external reconciliation deadline. Not every shared delivery mutation gains a new immediate signal. - Notifications use the existing process-local database ownership contract. Independent SQL writers and other replicas need their own supported integration. - Provider markers, publication leases, and saved actions preserve recovery after uncertainty. This change does not expand Cloud alarm coverage or prove deployed host/database suspension. - Each remaining sleep slice needs its own review and CI checks before merge. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context-window size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
f8226b0515
commit
65fa1cd617
20 files changed
+972
-210
No files matched your search
@@ -99,6 +99,8 @@ describe("createChatReconciliationCoordinator", () => {
|
||||
expect(github).not.toContain("githubResponseCommentService(db, fetchImpl).processPending()");
|
||||
expect(github).not.toContain("githubResponseCommentService(db, fetchImpl).processTerminalResponses()");
|
||||
expect(source).toContain("chatChannels.registerGitHubTerminalResponseWork(deliveryWork,");
|
||||
expect(source).toContain("chatChannels.registerGitHubReviewCheckWork(deliveryWork,");
|
||||
expect(github).not.toContain("githubReviewCheckService(db, fetchImpl).processPending()");
|
||||
expect(source).toContain("chatChannels.registerTerminalConfirmationWork(deliveryWork,");
|
||||
expect(source).toContain("chatFailedRetryNoticeWork(chatChannels).register(deliveryWork,");
|
||||
});
|
||||
|
||||
@@ -4351,15 +4351,18 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
const baseline = f.responseWrites.length;
|
||||
sending = githubChatReviewService(db, f.providerFetch).execute(f.session, "comment", { body: "Authored final response", idempotencyKey: "terminal-owner-race" });
|
||||
await ready;
|
||||
await expect.poll(() => coordinator.nextWakeAt(), { timeout: 5000 }).toBeNull();
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 5000 }).toEqual({ active: 0, next: null });
|
||||
expect(f.responseWrites).toHaveLength(baseline);
|
||||
release(); await sending;
|
||||
expect((await action()).status).toBe("cancelled");
|
||||
if (!uncertain) await expect.poll(() => [...f.responseComments.values()][0]?.body, { timeout: 5000 }).toContain("stopped before completing");
|
||||
await expect.poll(() => coordinator.nextWakeAt(), { timeout: 5000 }).toBeNull();
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 5000 }).toEqual({ active: 0, next: null });
|
||||
if (uncertain) expect(f.responseWrites).toHaveLength(baseline);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const reads = vi.spyOn(db, "select"), writes = vi.spyOn(db, "transaction");
|
||||
onTestFinished(() => { reads.mockRestore(); writes.mockRestore(); });
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(reads).not.toHaveBeenCalled(); expect(writes).not.toHaveBeenCalled();
|
||||
reads.mockRestore(); writes.mockRestore(); expect(errors).toEqual([]);
|
||||
@@ -7249,6 +7252,237 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
|
||||
runId: run.id,
|
||||
}, "read_pull_request", { section: "metadata" })).rejects.toThrow("older pull request head");
|
||||
});
|
||||
async function reviewCheckWorkerFixture(status: "running" | "failed" = "failed") {
|
||||
const f = await toolReplyFixture("github:paperclipai/paperclip:418");
|
||||
const config = await f.management.configuration(f.endpoint.id, "owner-user");
|
||||
const event = githubAutomaticReviewEvent({ action: "opened", repository: { id: 97531, full_name: "paperclipai/paperclip" }, sender: { id: 42, login: "octocat" },
|
||||
pull_request: { number: 418, title: "Review", body: "", draft: false, base: { sha: "a".repeat(40), ref: "master" }, head: { sha: "b".repeat(40) }, user: { id: 42, login: "octocat" }, labels: [] } }, randomUUID())!;
|
||||
const [review] = await db.insert(chatGitHubReviews).values({ companyId: f.companyId, endpointId: f.endpoint.id,
|
||||
issueId: f.conversation.issueId!, runId: f.run.id, repositoryId: "97531", repository: event.repository, pullNumber: 418, headSha: event.headSha,
|
||||
deliveryId: event.deliveryId, configurationRevision: config.revision, policySnapshot: config.configuration.defaults, event }).returning();
|
||||
await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
const writes: Record<string, unknown>[] = [], requests: string[] = [];
|
||||
let remoteHead = event.headSha, unavailable = false;
|
||||
f.setSupplementalProviderFetch(async (input, init) => {
|
||||
const url = String(input);
|
||||
if (url.endsWith("/pulls/418")) { requests.push(url); return Response.json({ head: { sha: remoteHead } }); }
|
||||
if (url.includes("/check-runs?")) { requests.push(url); return Response.json({ check_runs: [] }); }
|
||||
if (url.endsWith("/check-runs") && init?.method === "POST") {
|
||||
requests.push(url);
|
||||
if (unavailable) return Response.json({ message: "injected check outage" }, { status: 503 });
|
||||
writes.push(JSON.parse(String(init.body)));
|
||||
return Response.json({ id: 418, html_url: "https://github.com/checks/418" });
|
||||
}
|
||||
return undefined;
|
||||
});
|
||||
const checks = githubReviewCheckService(db, f.providerFetch);
|
||||
const [endpoint] = await readChatEndpointsById(f.endpoint.id);
|
||||
const action = async () => (await db.select().from(chatActions).where(and(eq(chatActions.endpointId, endpoint.id), eq(chatActions.kind, "github_review_check"))))[0];
|
||||
return { ...f, event, review, endpoint, checks, writes, requests, action,
|
||||
enqueue: () => checks.enqueue(endpoint, event, true, "authorized"),
|
||||
setRemoteHead: (value: string) => { remoteHead = value; }, setUnavailable: (value: boolean) => { unavailable = value; } };
|
||||
}
|
||||
|
||||
it("keeps GitHub review check history quiet and wakes from the real enqueue producer", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 5000 }).toEqual({ active: 0, next: null });
|
||||
await f.enqueue();
|
||||
await expect.poll(async () => (await f.action())?.status, { timeout: 5000 }).toBe("processed");
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 5000 }).toEqual({ active: 0, next: null });
|
||||
expect(f.writes).toEqual([expect.objectContaining({ status: "completed" })]);
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const reads = vi.spyOn(db, "select"), transactions = vi.spyOn(db, "transaction");
|
||||
onTestFinished(() => { reads.mockRestore(); transactions.mockRestore(); });
|
||||
const count = f.requests.length;
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(reads).not.toHaveBeenCalled(); expect(transactions).not.toHaveBeenCalled();
|
||||
expect(f.requests).toHaveLength(count);
|
||||
reads.mockRestore(); transactions.mockRestore();
|
||||
expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it("defers GitHub review check enqueue until outer commit and discards rollback", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => { throw error; } });
|
||||
const enqueue = (tx: Parameters<Parameters<Db["transaction"]>[0]>[0]) =>
|
||||
githubReviewCheckService(tx as unknown as Db, f.providerFetch).enqueue(f.endpoint, f.event, true, "authorized");
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
await expect(db.transaction(async tx => { await enqueue(tx); throw new Error("rollback check"); })).rejects.toThrow("rollback check");
|
||||
expect(await f.action()).toBeUndefined(); expect(f.writes).toEqual([]);
|
||||
let entered!: () => void, release!: () => void;
|
||||
const ready = new Promise<void>(resolve => { entered = resolve; }), hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const pending = db.transaction(async tx => { await enqueue(tx); entered(); await hold; });
|
||||
try { await ready; expect(f.writes).toEqual([]); } finally { release(); await pending; }
|
||||
await expect.poll(async () => (await f.action())?.status, { timeout: 5000 }).toBe("processed");
|
||||
expect(f.writes).toHaveLength(1);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("recovers a missed GitHub head webhook only while a review check remains unresolved", async () => {
|
||||
const f = await reviewCheckWorkerFixture("running");
|
||||
await f.enqueue();
|
||||
const errors: unknown[] = [];
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
await expect.poll(() => f.writes.length, { timeout: 5000 }).toBe(1);
|
||||
expect(f.writes[0].status).toBe("in_progress");
|
||||
const saved = await f.action();
|
||||
expect(Date.parse(String(saved.result?.retryAt))).toBeGreaterThan(Date.now());
|
||||
f.setRemoteHead("c".repeat(40));
|
||||
await vi.advanceTimersByTimeAsync(6000);
|
||||
await expect.poll(async () => (await f.action()).status, { timeout: 5000 }).toBe("cancelled");
|
||||
await expect.poll(() => ({ active: idleWorkSnapshot().active, next: coordinator.nextWakeAt() }),
|
||||
{ timeout: 5000 }).toEqual({ active: 0, next: null });
|
||||
expect((await db.select().from(chatGitHubReviews).where(eq(chatGitHubReviews.id, f.review.id)))[0].state).toBe("superseded");
|
||||
const reads = vi.spyOn(db, "select"), count = f.requests.length;
|
||||
onTestFinished(() => reads.mockRestore());
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(reads).not.toHaveBeenCalled(); expect(f.requests).toHaveLength(count); reads.mockRestore();
|
||||
expect(f.writes).toHaveLength(1); expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it.each(["restart", "drain", "standby"] as const)("retains GitHub review check work across %s", async mode => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
let canRun = true;
|
||||
const errors: unknown[] = [];
|
||||
let coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => canRun,
|
||||
canReconcile: () => mode !== "standby" || canRun, onError: error => errors.push(error) });
|
||||
let worker = f.checks.registerCheckWork(coordinator, () => canRun);
|
||||
try {
|
||||
await worker.ready;
|
||||
if (mode === "restart") await coordinator.stop(); else canRun = false;
|
||||
await f.enqueue(); expect(f.writes).toEqual([]);
|
||||
if (mode === "restart") {
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
worker = f.checks.registerCheckWork(coordinator, () => true); await worker.ready;
|
||||
} else { canRun = true; worker.wake(); }
|
||||
await expect.poll(async () => (await f.action())?.status, { timeout: 5000 }).toBe("processed");
|
||||
await expect.poll(() => coordinator.nextWakeAt(), { timeout: 5000 }).toBeNull();
|
||||
expect(f.writes).toHaveLength(1); expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("retains the saved GitHub review check retry through restart without an early provider call", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
f.setUnavailable(true); await f.enqueue(); await f.checks.processPending();
|
||||
const failed = await f.action();
|
||||
expect(failed).toMatchObject({ status: "failed", result: { attempts: 1, retryable: true } });
|
||||
const future = new Date(Date.now() + 60_000).toISOString();
|
||||
await db.update(chatActions).set({ result: { ...failed.result, retryAt: future } }).where(eq(chatActions.id, failed.id));
|
||||
f.setUnavailable(false);
|
||||
const errors: unknown[] = [];
|
||||
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
const count = f.requests.length;
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
expect(coordinator.nextWakeAt()).toBe(Date.parse(future)); expect(f.requests).toHaveLength(count);
|
||||
await vi.advanceTimersByTimeAsync(61_000);
|
||||
await expect.poll(async () => (await f.action()).status, { timeout: 5000 }).toBe("processed");
|
||||
expect(f.writes).toHaveLength(1); expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); vi.useRealTimers(); }
|
||||
});
|
||||
|
||||
it("waits for and wakes from the actual GitHub review check publication lease", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
let entered!: () => void, release!: () => void;
|
||||
const ready = new Promise<void>(resolve => { entered = resolve; }), hold = new Promise<void>(resolve => { release = resolve; });
|
||||
let lease: Promise<unknown> | undefined;
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
lease = withGitHubPublicationLease(db, { companyId: f.companyId, endpointId: f.endpoint.id, repositoryId: "97531", number: 418 }, f.providerFetch,
|
||||
async () => { entered(); await hold; });
|
||||
await ready; await f.enqueue();
|
||||
await expect.poll(() => coordinator.nextWakeAt(), { timeout: 5000 }).not.toBeNull();
|
||||
expect(f.writes).toEqual([]); release(); await lease;
|
||||
await expect.poll(async () => (await f.action())?.status, { timeout: 5000 }).toBe("processed");
|
||||
expect(f.writes).toHaveLength(1); expect(errors).toEqual([]);
|
||||
} finally { release(); await lease; await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("wakes a GitHub review check from committed run identity before its future head deadline", async () => {
|
||||
const f = await reviewCheckWorkerFixture("running");
|
||||
await f.enqueue(); await f.checks.processPending();
|
||||
const action = await f.action(), future = new Date(Date.now() + 60_000).toISOString();
|
||||
await db.update(chatActions).set({ result: { ...action.result, retryAt: future } }).where(eq(chatActions.id, action.id));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
expect(coordinator.nextWakeAt()).toBe(Date.parse(future));
|
||||
const change = async (tx: Parameters<Parameters<Db["transaction"]>[0]>[0]) => {
|
||||
await tx.update(heartbeatRuns).set({ status: "failed", updatedAt: new Date() }).where(eq(heartbeatRuns.id, f.run.id));
|
||||
await initializeRunIdentity(tx as unknown as Db, { companyId: f.companyId, runId: f.run.id, issueId: f.conversation.issueId!,
|
||||
responsibleUserId: "owner-user", messageIds: [String(f.run.contextSnapshot?.wakeCommentId)], cause: "instruction" });
|
||||
};
|
||||
await expect(db.transaction(async tx => { await change(tx); throw new Error("rollback run check"); })).rejects.toThrow("rollback run check");
|
||||
expect(f.writes).toHaveLength(1);
|
||||
await db.transaction(change);
|
||||
await expect.poll(async () => (await f.action()).status, { timeout: 5000 }).toBe("processed");
|
||||
expect(f.writes).toHaveLength(2); expect(f.writes[1].status).toBe("completed");
|
||||
expect(Date.now()).toBeLessThan(Date.parse(future)); expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("finds a GitHub review check after a full page of revoked work and ignores settled history", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
await f.enqueue(); const original = await f.action();
|
||||
const pageKey = randomUUID().replaceAll("-", "").slice(0, 9);
|
||||
const prefix = Array.from({ length: 101 }, (_, index) => ({ ...original,
|
||||
id: `00000000-0000-4000-8000-${pageKey}${String(index + 1).padStart(3, "0")}`,
|
||||
providerActionId: `revoked-check-${index}`, payload: { ...original.payload, runtimeGeneration: -1 } }));
|
||||
await db.insert(chatActions).values(prefix);
|
||||
await db.insert(chatActions).values(Array.from({ length: 101 }, (_, index) => ({ ...original,
|
||||
id: randomUUID(), providerActionId: `settled-check-${index}`, status: "processed" })));
|
||||
const errors: unknown[] = [];
|
||||
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
await expect.poll(async () => (await db.select().from(chatActions).where(and(eq(chatActions.id, original.id))))[0].status,
|
||||
{ timeout: 15_000 }).toBe("processed");
|
||||
await expect.poll(() => coordinator.nextWakeAt(), { timeout: 15_000 }).toBeNull();
|
||||
expect(f.writes).toHaveLength(1);
|
||||
const cancelled = await db.select().from(chatActions).where(and(eq(chatActions.endpointId, f.endpoint.id), eq(chatActions.status, "cancelled")));
|
||||
expect(cancelled).toHaveLength(101); expect(errors).toEqual([]);
|
||||
} finally { await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("stops GitHub review check HTTP after shutdown admission and recovers its durable action", async () => {
|
||||
const f = await reviewCheckWorkerFixture();
|
||||
await f.enqueue();
|
||||
let entered!: () => void, release!: () => void;
|
||||
const ready = new Promise<void>(resolve => { entered = resolve; }), hold = new Promise<void>(resolve => { release = resolve; });
|
||||
const guardedProvider: typeof fetch = async (input, init) => {
|
||||
if (String(input).endsWith("/pulls/418")) { entered(); await hold; }
|
||||
return f.providerFetch(input, init);
|
||||
};
|
||||
const errors: unknown[] = [];
|
||||
let coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
try {
|
||||
const starting = githubReviewCheckService(db, guardedProvider).registerCheckWork(coordinator, () => true).ready;
|
||||
await ready;
|
||||
const stopped = coordinator.stop(); release(); await stopped; await starting;
|
||||
expect(f.writes).toEqual([]); expect((await f.action()).status).toBe("received");
|
||||
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: error => errors.push(error) });
|
||||
await f.checks.registerCheckWork(coordinator, () => true).ready;
|
||||
await expect.poll(async () => (await f.action()).status, { timeout: 5000 }).toBe("processed");
|
||||
expect(f.writes).toHaveLength(1); expect(errors).toEqual([]);
|
||||
} finally { release(); await coordinator.stop(); }
|
||||
});
|
||||
|
||||
it("supersedes a late review projection after its check already rejected the old head", async () => {
|
||||
const f = await reviewBotFixture();
|
||||
const deliveryId = randomUUID();
|
||||
|
||||
@@ -1289,6 +1289,7 @@ export async function createApp(
|
||||
chatChannels.registerTeamsConsentExpiryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerTeamsAttemptRecoveryWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerInteractionContinuationWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerGitHubReviewCheckWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerGitHubTerminalResponseWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerGitHubAcknowledgementWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
chatChannels.registerTerminalConfirmationWork(deliveryWork, () => !isWarmStandby() && !isIdleTaskDrainActive());
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { githubReviewCheckService } from "./chat-github-checks.js";
|
||||
import { leasedChatProvider } from "./chat-channels/runtime/receiver-policy.js";
|
||||
import { chatReceiverSchedule } from "./chat-channels/runtime/receiver-schedule.js";
|
||||
import { registerChatReceiverWork } from "./chat-channels/runtime/receiver-work.js";
|
||||
@@ -3504,6 +3505,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
|
||||
const { getPublicationBatchStatus, publishComment, publishBoardMessage } = createChatPublicationRequests(db, { endpointRecord, stageTeamsBoardFileIntent, processPendingSlackBoardMessages, processPendingPublications, prepareTeamsBoardSend });
|
||||
return {
|
||||
registerInteractionContinuationWork,
|
||||
registerGitHubReviewCheckWork: githubReviewCheckService(db, fetchImpl).registerCheckWork,
|
||||
registerGitHubTerminalResponseWork: githubResponseCommentService(db, fetchImpl).registerTerminalResponseWork,
|
||||
registerGitHubAcknowledgementWork: githubResponseCommentService(db, fetchImpl).registerAcknowledgementWork,
|
||||
registerTerminalConfirmationWork,
|
||||
|
||||
@@ -12,7 +12,7 @@ export function createGitHubMaintenance(db: Db, fetchImpl: typeof globalThis.fet
|
||||
options.scheduleMessageProcessing(async () => {
|
||||
try {
|
||||
// Provider I/O must not block Slack, Teams, or ordinary message receipts.
|
||||
await githubReviewCheckService(db, fetchImpl).processPending();
|
||||
await githubReviewCheckService(db, fetchImpl).repairSupersededReviews();
|
||||
} finally {
|
||||
githubMaintenancePending = false;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
import { and, desc, eq, sql } from "drizzle-orm";
|
||||
import { chatActions, chatDeliveries, chatEndpoints, chatEndpointResources,
|
||||
chatGitHubConfigurations, chatGitHubReviews, chatMessageLinks, companies,
|
||||
heartbeatRuns, type Db } from "@paperclipai/db";
|
||||
import type { GitHubReviewEventContext } from "@paperclipai/shared";
|
||||
|
||||
export async function githubReviewCheckState(db: Db, action: typeof chatActions.$inferSelect) {
|
||||
const event = action.payload.event as GitHubReviewEventContext;
|
||||
const [source] = await db
|
||||
.select({
|
||||
endpoint: chatEndpoints,
|
||||
resource: chatEndpointResources,
|
||||
configuration: chatGitHubConfigurations.configuration,
|
||||
companyPrefix: companies.issuePrefix,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.innerJoin(companies, eq(companies.id, chatEndpoints.companyId))
|
||||
.innerJoin(
|
||||
chatGitHubConfigurations,
|
||||
eq(chatGitHubConfigurations.endpointId, chatEndpoints.id),
|
||||
)
|
||||
.innerJoin(
|
||||
chatEndpointResources,
|
||||
eq(chatEndpointResources.endpointId, chatEndpoints.id),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, action.endpointId),
|
||||
eq(chatEndpoints.companyId, action.companyId),
|
||||
eq(chatEndpointResources.providerResourceId, event.repository),
|
||||
),
|
||||
);
|
||||
if (
|
||||
!source ||
|
||||
!source.configuration.toolsEnabled ||
|
||||
!["active", "verifying"].includes(source.endpoint.status) ||
|
||||
!source.resource.enabled ||
|
||||
source.resource.availability !== "available" ||
|
||||
String(source.resource.metadata?.providerRepositoryId) !==
|
||||
event.repositoryId ||
|
||||
((source.endpoint.setup as { runtimeGeneration?: number })
|
||||
.runtimeGeneration ?? 0) !== action.payload.runtimeGeneration
|
||||
) {
|
||||
return { disposition: "connection_changed" as const };
|
||||
}
|
||||
const [review] = await db
|
||||
.select()
|
||||
.from(chatGitHubReviews)
|
||||
.where(
|
||||
and(
|
||||
eq(chatGitHubReviews.endpointId, action.endpointId),
|
||||
eq(chatGitHubReviews.repositoryId, event.repositoryId),
|
||||
eq(chatGitHubReviews.pullNumber, event.pullNumber),
|
||||
eq(chatGitHubReviews.headSha, event.headSha),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(chatGitHubReviews.createdAt))
|
||||
.limit(1);
|
||||
if (review?.assessment) return { disposition: "assessment_owns_check" as const, reviewId: review.id };
|
||||
let run = review?.runId
|
||||
? (
|
||||
await db
|
||||
.select({ id: heartbeatRuns.id, status: heartbeatRuns.status })
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, review.runId))
|
||||
)[0]
|
||||
: undefined;
|
||||
if (review && !run) {
|
||||
const [sourceRun] = await db
|
||||
.select({ run: { id: heartbeatRuns.id, status: heartbeatRuns.status } })
|
||||
.from(chatDeliveries)
|
||||
.innerJoin(
|
||||
chatMessageLinks,
|
||||
eq(chatMessageLinks.deliveryId, chatDeliveries.id),
|
||||
)
|
||||
.innerJoin(
|
||||
heartbeatRuns,
|
||||
and(
|
||||
eq(heartbeatRuns.companyId, action.companyId),
|
||||
eq(heartbeatRuns.agentId, source.endpoint.assignedAgentId),
|
||||
sql`${heartbeatRuns.contextSnapshot}->>'wakeCommentId' = ${chatMessageLinks.commentId}::text`,
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, action.endpointId),
|
||||
sql`${chatDeliveries.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId' = ${event.deliveryId}`,
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRuns.createdAt))
|
||||
.limit(1);
|
||||
run = sourceRun?.run;
|
||||
}
|
||||
const [delivery] = await db
|
||||
.select({ state: chatDeliveries.state })
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, action.endpointId),
|
||||
sql`${chatDeliveries.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId' = ${event.deliveryId}`,
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
const admissionEnded =
|
||||
!review &&
|
||||
(delivery?.state === "filtered" || delivery?.state === "failed");
|
||||
const terminal =
|
||||
!!run &&
|
||||
[
|
||||
"succeeded",
|
||||
"failed",
|
||||
"cancelled",
|
||||
"timed_out",
|
||||
"interrupted",
|
||||
].includes(run.status);
|
||||
const state: (typeof chatGitHubReviews.$inferSelect)["state"] =
|
||||
(!action.payload.admitted && !review) || admissionEnded
|
||||
? "manual_required"
|
||||
: terminal
|
||||
? run!.status === "succeeded"
|
||||
? "incomplete"
|
||||
: "error"
|
||||
: run?.status === "running"
|
||||
? "running"
|
||||
: "queued";
|
||||
const status =
|
||||
state === "running"
|
||||
? "in_progress"
|
||||
: state === "queued"
|
||||
? "queued"
|
||||
: "completed";
|
||||
return { disposition: "publish" as const, source, review, run, terminal, state, status };
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import { and, asc, eq, gt, or, sql } from "drizzle-orm";
|
||||
import { chatActions, chatEndpointLeases, type Db } from "@paperclipai/db";
|
||||
import type { GitHubReviewEventContext } from "@paperclipai/shared";
|
||||
import type { createDeliveryWorkCoordinator } from "../../delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../delivery-work-notifications.js";
|
||||
import { registerScopedDeadlineWork } from "../../scoped-deadline-work.js";
|
||||
import { githubReviewCheckState } from "./review-check-state.js";
|
||||
|
||||
type Scope = { companyId: string; endpointId?: string; repositoryId?: string; number?: number };
|
||||
type Entry = Scope & { id: string; dueAt: number };
|
||||
|
||||
export async function notifyGitHubReviewChecks(tx: object, scope: Scope) {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubReviewChecks, JSON.stringify(scope));
|
||||
}
|
||||
|
||||
export function registerGitHubReviewCheckWork(
|
||||
db: Db,
|
||||
coordinator: ReturnType<typeof createDeliveryWorkCoordinator>,
|
||||
dispatch: (id: string, canContinue: () => boolean) => Promise<unknown>,
|
||||
canRun: () => boolean,
|
||||
) {
|
||||
async function page(scope?: Scope, cursor?: string, actionId?: string) {
|
||||
return db.select({ action: chatActions,
|
||||
leaseExpiresAt: sql<string | null>`(select max(l.expires_at) from ${chatEndpointLeases} l
|
||||
where l.company_id = ${chatActions.companyId} and l.endpoint_id = ${chatActions.endpointId}
|
||||
and l.lease_key = 'github-publication:' || (${chatActions.payload}->'event'->>'repositoryId')
|
||||
|| ':' || (${chatActions.payload}->'event'->>'pullNumber'))`,
|
||||
}).from(chatActions).where(and(eq(chatActions.kind, "github_review_check"),
|
||||
or(eq(chatActions.status, "received"), and(eq(chatActions.status, "failed"), sql`${chatActions.result}->>'retryable' = 'true'`)),
|
||||
scope ? eq(chatActions.companyId, scope.companyId) : undefined,
|
||||
scope?.endpointId ? eq(chatActions.endpointId, scope.endpointId) : undefined,
|
||||
scope?.repositoryId ? sql`${chatActions.payload}->'event'->>'repositoryId' = ${scope.repositoryId}` : undefined,
|
||||
scope?.number ? sql`${chatActions.payload}->'event'->>'pullNumber' = ${String(scope.number)}` : undefined,
|
||||
cursor ? gt(chatActions.id, cursor) : undefined, actionId ? eq(chatActions.id, actionId) : undefined,
|
||||
)).orderBy(asc(chatActions.id)).limit(100);
|
||||
}
|
||||
async function entry(row: Awaited<ReturnType<typeof page>>[number]): Promise<Entry> {
|
||||
const { action } = row;
|
||||
const event = action.payload.event as GitHubReviewEventContext;
|
||||
let dueAt = action.result?.retryAt ? Date.parse(String(action.result.retryAt)) : 0;
|
||||
if (!Number.isFinite(dueAt)) throw new Error("GitHub review check has an invalid retry deadline");
|
||||
if (action.status === "received") {
|
||||
const current = await githubReviewCheckState(db, action);
|
||||
if (current.disposition !== "publish" || current.state !== action.result?.state) dueAt = 0;
|
||||
}
|
||||
if (row.leaseExpiresAt) dueAt = Math.max(dueAt, Date.parse(row.leaseExpiresAt));
|
||||
return { id: action.id, companyId: action.companyId, endpointId: action.endpointId,
|
||||
repositoryId: event.repositoryId, number: event.pullNumber, dueAt };
|
||||
}
|
||||
return registerScopedDeadlineWork(coordinator, {
|
||||
queue: DELIVERY_QUEUES.chatGitHubReviewChecks,
|
||||
decodeScope: (hint: string): Scope | undefined => hint === "*" ? undefined : JSON.parse(hint),
|
||||
matches: (value: Entry, scope?: Scope) => !scope || (value.companyId === scope.companyId &&
|
||||
(!scope.endpointId || value.endpointId === scope.endpointId) &&
|
||||
(!scope.repositoryId || value.repositoryId === scope.repositoryId) && (!scope.number || value.number === scope.number)),
|
||||
load: async (scope?: Scope, signal?: AbortSignal) => {
|
||||
const entries: Entry[] = [];
|
||||
let cursor: string | undefined;
|
||||
while (canRun() && !signal?.aborted) {
|
||||
const rows = await page(scope, cursor);
|
||||
for (const row of rows) {
|
||||
if (!canRun() || signal?.aborted) throw signal?.reason ?? new Error("GitHub review check recovery stopped");
|
||||
entries.push(await entry(row));
|
||||
}
|
||||
if (rows.length < 100) return entries;
|
||||
cursor = rows[rows.length - 1].action.id;
|
||||
}
|
||||
throw signal?.reason ?? new Error("GitHub review check recovery stopped");
|
||||
},
|
||||
check: async (value: Entry, signal?: AbortSignal) => {
|
||||
const [row] = await page(undefined, undefined, value.id);
|
||||
const current = row ? await entry(row) : null;
|
||||
if (!current || current.dueAt > Date.now() || !canRun() || signal?.aborted) return current;
|
||||
await dispatch(current.id, () => canRun() && !signal?.aborted);
|
||||
const [remaining] = await page(undefined, undefined, value.id);
|
||||
return remaining ? entry(remaining) : null;
|
||||
},
|
||||
}, canRun);
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyGitHubReviewChecks } from "./review-check-work.js";
|
||||
import { agents, chatDeliveries, chatGitHubReviews } from "@paperclipai/db";
|
||||
import { and, eq, sql } from "drizzle-orm";
|
||||
import { LOW_TRUST_REVIEW_PRESET, type GitHubReviewEventContext, type GitHubIssueEventContext, type GitHubReviewPolicy } from "@paperclipai/shared";
|
||||
@@ -41,6 +42,8 @@ export async function retainGitHubDeliveryAuthority(
|
||||
}
|
||||
|
||||
export async function retainGitHubReviewReceipt(taskTx: DbOrTransaction, endpoint: EndpointRow, issueId: string, deliveryId: string, githubAutomatic: AutomaticRequest): Promise<void> {
|
||||
await notifyGitHubReviewChecks(taskTx, { companyId: endpoint.companyId, endpointId: endpoint.id,
|
||||
repositoryId: githubAutomatic.context.repositoryId, number: githubAutomatic.context.pullNumber });
|
||||
await taskTx.insert(chatGitHubReviews).values({
|
||||
companyId: endpoint.companyId, endpointId: endpoint.id, issueId: issueId,
|
||||
repositoryId: githubAutomatic.context.repositoryId, repository: githubAutomatic.context.repository,
|
||||
|
||||
@@ -1,18 +1,17 @@
|
||||
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
|
||||
import { notifyGitHubReviewChecks, registerGitHubReviewCheckWork } from "./chat-channels/github/review-check-work.js";
|
||||
import { githubReviewCheckState } from "./chat-channels/github/review-check-state.js";
|
||||
import {
|
||||
GitHubPublicationLeaseLost,
|
||||
withGitHubPublicationLease,
|
||||
} from "./chat-github-publication-lease.js";
|
||||
import { and, desc, eq, inArray, isNull, or, sql } from "drizzle-orm";
|
||||
import { and, eq, inArray, isNull, or, sql } from "drizzle-orm";
|
||||
import {
|
||||
chatActions,
|
||||
chatDeliveries,
|
||||
chatEndpoints,
|
||||
chatEndpointResources,
|
||||
chatGitHubConfigurations,
|
||||
chatGitHubReviews,
|
||||
chatMessageLinks,
|
||||
companies,
|
||||
heartbeatRuns,
|
||||
type Db,
|
||||
} from "@paperclipai/db";
|
||||
import type { GitHubReviewEventContext } from "@paperclipai/shared";
|
||||
@@ -41,27 +40,32 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
...config.configuration.repositories[event.repositoryId],
|
||||
};
|
||||
if (policy.ratingThreshold === null && !admitted) return;
|
||||
await db
|
||||
.insert(chatActions)
|
||||
.values({
|
||||
companyId: endpoint.companyId,
|
||||
endpointId: endpoint.id,
|
||||
kind: "github_review_check",
|
||||
providerActionId: `github-check:${event.deliveryId}`,
|
||||
payload: {
|
||||
version: 1,
|
||||
event,
|
||||
admitted,
|
||||
reason,
|
||||
runtimeGeneration:
|
||||
(endpoint.setup as { runtimeGeneration?: number })
|
||||
.runtimeGeneration ?? 0,
|
||||
},
|
||||
status: "received",
|
||||
})
|
||||
.onConflictDoNothing();
|
||||
await db.transaction(async tx => {
|
||||
await notifyGitHubReviewChecks(tx, { companyId: endpoint.companyId, endpointId: endpoint.id,
|
||||
repositoryId: event.repositoryId, number: event.pullNumber });
|
||||
await tx
|
||||
.insert(chatActions)
|
||||
.values({
|
||||
companyId: endpoint.companyId,
|
||||
endpointId: endpoint.id,
|
||||
kind: "github_review_check",
|
||||
providerActionId: `github-check:${event.deliveryId}`,
|
||||
payload: {
|
||||
version: 1,
|
||||
event,
|
||||
admitted,
|
||||
reason,
|
||||
runtimeGeneration:
|
||||
(endpoint.setup as { runtimeGeneration?: number })
|
||||
.runtimeGeneration ?? 0,
|
||||
},
|
||||
status: "received",
|
||||
})
|
||||
.onConflictDoNothing();
|
||||
});
|
||||
}
|
||||
async function dispatch(id: string) {
|
||||
async function dispatch(id: string, canContinue = () => true) {
|
||||
if (!canContinue()) return;
|
||||
const [action] = await db
|
||||
.select()
|
||||
.from(chatActions)
|
||||
@@ -74,6 +78,11 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
.limit(1);
|
||||
if (!action || ["processed", "cancelled"].includes(action.status)) return;
|
||||
const event = action.payload.event as GitHubReviewEventContext;
|
||||
const guardedFetch: typeof fetch = (input, init) => {
|
||||
if (!canContinue()) throw new GitHubPublicationLeaseLost();
|
||||
return fetchImpl(input, init);
|
||||
};
|
||||
if (!canContinue()) return;
|
||||
await withGitHubPublicationLease(
|
||||
db,
|
||||
{
|
||||
@@ -82,7 +91,7 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
repositoryId: event.repositoryId,
|
||||
number: event.pullNumber,
|
||||
},
|
||||
fetchImpl,
|
||||
guardedFetch,
|
||||
async (lease) => {
|
||||
const [fresh] = await db
|
||||
.select()
|
||||
@@ -91,155 +100,19 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
if (!fresh || ["processed", "cancelled"].includes(fresh.status)) return;
|
||||
let attempts = Number(fresh.result?.attempts ?? 0);
|
||||
try {
|
||||
const [source] = await db
|
||||
.select({
|
||||
endpoint: chatEndpoints,
|
||||
resource: chatEndpointResources,
|
||||
configuration: chatGitHubConfigurations.configuration,
|
||||
companyPrefix: companies.issuePrefix,
|
||||
})
|
||||
.from(chatEndpoints)
|
||||
.innerJoin(companies, eq(companies.id, chatEndpoints.companyId))
|
||||
.innerJoin(
|
||||
chatGitHubConfigurations,
|
||||
eq(chatGitHubConfigurations.endpointId, chatEndpoints.id),
|
||||
)
|
||||
.innerJoin(
|
||||
chatEndpointResources,
|
||||
eq(chatEndpointResources.endpointId, chatEndpoints.id),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatEndpoints.id, action.endpointId),
|
||||
eq(chatEndpoints.companyId, action.companyId),
|
||||
eq(chatEndpointResources.providerResourceId, event.repository),
|
||||
),
|
||||
);
|
||||
if (
|
||||
!source ||
|
||||
!source.configuration.toolsEnabled ||
|
||||
!["active", "verifying"].includes(source.endpoint.status) ||
|
||||
!source.resource.enabled ||
|
||||
source.resource.availability !== "available" ||
|
||||
String(source.resource.metadata?.providerRepositoryId) !==
|
||||
event.repositoryId ||
|
||||
((source.endpoint.setup as { runtimeGeneration?: number })
|
||||
.runtimeGeneration ?? 0) !== action.payload.runtimeGeneration
|
||||
) {
|
||||
await lease.commit(async (tx) => {
|
||||
await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "cancelled",
|
||||
result: { code: "connection_changed" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatActions.id, id));
|
||||
const currentState = await githubReviewCheckState(db, fresh);
|
||||
if (currentState.disposition !== "publish") {
|
||||
await lease.commit(async tx => {
|
||||
await tx.update(chatActions).set({
|
||||
status: currentState.disposition === "connection_changed" ? "cancelled" : "processed",
|
||||
result: { code: currentState.disposition,
|
||||
...(currentState.disposition === "assessment_owns_check" ? { reviewId: currentState.reviewId } : {}) },
|
||||
updatedAt: new Date(),
|
||||
}).where(eq(chatActions.id, id));
|
||||
});
|
||||
return;
|
||||
}
|
||||
const [review] = await db
|
||||
.select()
|
||||
.from(chatGitHubReviews)
|
||||
.where(
|
||||
and(
|
||||
eq(chatGitHubReviews.endpointId, action.endpointId),
|
||||
eq(chatGitHubReviews.repositoryId, event.repositoryId),
|
||||
eq(chatGitHubReviews.pullNumber, event.pullNumber),
|
||||
eq(chatGitHubReviews.headSha, event.headSha),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(chatGitHubReviews.createdAt))
|
||||
.limit(1);
|
||||
if (review?.assessment) {
|
||||
// The assessment outbox owns the terminal score and its retries.
|
||||
await lease.commit(async (tx) => {
|
||||
await tx
|
||||
.update(chatActions)
|
||||
.set({
|
||||
status: "processed",
|
||||
result: {
|
||||
code: "assessment_owns_check",
|
||||
reviewId: review.id,
|
||||
},
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(chatActions.id, id));
|
||||
});
|
||||
return;
|
||||
}
|
||||
let run = review?.runId
|
||||
? (
|
||||
await db
|
||||
.select({ id: heartbeatRuns.id, status: heartbeatRuns.status })
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, review.runId))
|
||||
)[0]
|
||||
: undefined;
|
||||
if (review && !run) {
|
||||
const [sourceRun] = await db
|
||||
.select({ run: { id: heartbeatRuns.id, status: heartbeatRuns.status } })
|
||||
.from(chatDeliveries)
|
||||
.innerJoin(
|
||||
chatMessageLinks,
|
||||
eq(chatMessageLinks.deliveryId, chatDeliveries.id),
|
||||
)
|
||||
.innerJoin(
|
||||
heartbeatRuns,
|
||||
and(
|
||||
eq(heartbeatRuns.companyId, action.companyId),
|
||||
eq(heartbeatRuns.agentId, source.endpoint.assignedAgentId),
|
||||
sql`${heartbeatRuns.contextSnapshot}->>'wakeCommentId' = ${chatMessageLinks.commentId}::text`,
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, action.endpointId),
|
||||
sql`${chatDeliveries.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId' = ${event.deliveryId}`,
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRuns.createdAt))
|
||||
.limit(1);
|
||||
run = sourceRun?.run;
|
||||
}
|
||||
const [delivery] = await db
|
||||
.select({ state: chatDeliveries.state })
|
||||
.from(chatDeliveries)
|
||||
.where(
|
||||
and(
|
||||
eq(chatDeliveries.endpointId, action.endpointId),
|
||||
sql`${chatDeliveries.normalizedEvent}->'githubAutomatic'->'context'->>'deliveryId' = ${event.deliveryId}`,
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
const admissionEnded =
|
||||
!review &&
|
||||
(delivery?.state === "filtered" || delivery?.state === "failed");
|
||||
const terminal =
|
||||
!!run &&
|
||||
[
|
||||
"succeeded",
|
||||
"failed",
|
||||
"cancelled",
|
||||
"timed_out",
|
||||
"interrupted",
|
||||
].includes(run.status);
|
||||
const state =
|
||||
(!action.payload.admitted && !review) || admissionEnded
|
||||
? "manual_required"
|
||||
: terminal
|
||||
? run!.status === "succeeded"
|
||||
? "incomplete"
|
||||
: "error"
|
||||
: run?.status === "running"
|
||||
? "running"
|
||||
: "queued";
|
||||
const status =
|
||||
state === "running"
|
||||
? "in_progress"
|
||||
: state === "queued"
|
||||
? "queued"
|
||||
: "completed";
|
||||
const { source, review, run, terminal, state, status } = currentState;
|
||||
const policy = review?.policySnapshot ?? {
|
||||
...source.configuration.defaults,
|
||||
...source.configuration.repositories[event.repositoryId],
|
||||
@@ -389,7 +262,7 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
.where(eq(chatActions.id, id));
|
||||
});
|
||||
} catch (error) {
|
||||
if (error instanceof GitHubPublicationLeaseLost) return;
|
||||
if (error instanceof GitHubPublicationLeaseLost || !canContinue()) return;
|
||||
attempts++;
|
||||
await lease.commit(async (tx) => {
|
||||
await tx
|
||||
@@ -412,7 +285,7 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
},
|
||||
);
|
||||
}
|
||||
async function processPending(limit = 10) {
|
||||
async function repairSupersededReviews() {
|
||||
// Check delivery and task admission run independently. A rapid push can
|
||||
// cancel an obsolete check before admission inserts its review projection.
|
||||
// Reconcile that late row from the durable cancellation receipt, without
|
||||
@@ -447,6 +320,9 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
) is not null`,
|
||||
),
|
||||
);
|
||||
}
|
||||
async function processPending(limit = 10) {
|
||||
await repairSupersededReviews();
|
||||
const rows = await db
|
||||
.select({ id: chatActions.id })
|
||||
.from(chatActions)
|
||||
@@ -470,5 +346,8 @@ export function githubReviewCheckService(db: Db, fetchImpl = fetch) {
|
||||
.limit(limit);
|
||||
for (const row of rows) await dispatch(row.id);
|
||||
}
|
||||
return { enqueue, processPending };
|
||||
return { enqueue, processPending, repairSupersededReviews,
|
||||
registerCheckWork: (coordinator: ReturnType<typeof createDeliveryWorkCoordinator>, canRun: () => boolean) =>
|
||||
registerGitHubReviewCheckWork(db, coordinator, dispatch, canRun),
|
||||
};
|
||||
}
|
||||
@@ -115,6 +115,7 @@ export async function withGitHubPublicationLease<T>(
|
||||
clearInterval(timer);
|
||||
await renewal;
|
||||
await db.transaction(async tx => {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubReviewChecks, JSON.stringify(scope));
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubPublications);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubTerminalResponses, JSON.stringify(scope));
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubAcknowledgements, JSON.stringify(scope));
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyGitHubReviewChecks } from "./chat-channels/github/review-check-work.js";
|
||||
import { notifyChatResponseAvailability } from "./chat-response-availability.js";
|
||||
import { githubRunReplyState } from "./chat-run-publications.js";
|
||||
import { stageReceiptReactionRemovals } from "./chat-receipt-reactions.js";
|
||||
@@ -350,16 +351,20 @@ export function githubChatReviewService(db: Db, fetchImpl = fetch) {
|
||||
.limit(1);
|
||||
if (prior && prior.runId === source.run.id) return prior;
|
||||
if (prior && !prior.runId && prior.deliveryId === source.delivery.id) {
|
||||
const [claimed] = await db
|
||||
.update(chatGitHubReviews)
|
||||
.set({ runId: source.run.id, state: "running", updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatGitHubReviews.id, prior.id),
|
||||
isNull(chatGitHubReviews.runId),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
const [claimed] = await db.transaction(async tx => {
|
||||
await notifyGitHubReviewChecks(tx, { companyId: source.endpoint.companyId, endpointId: source.endpoint.id,
|
||||
repositoryId: source.repositoryId, number: source.number });
|
||||
return tx
|
||||
.update(chatGitHubReviews)
|
||||
.set({ runId: source.run.id, state: "running", updatedAt: new Date() })
|
||||
.where(
|
||||
and(
|
||||
eq(chatGitHubReviews.id, prior.id),
|
||||
isNull(chatGitHubReviews.runId),
|
||||
),
|
||||
)
|
||||
.returning();
|
||||
});
|
||||
if (claimed) return claimed;
|
||||
return reviewForHead(source, pull);
|
||||
}
|
||||
@@ -407,26 +412,30 @@ export function githubChatReviewService(db: Db, fetchImpl = fetch) {
|
||||
labels: pull.labels.map((l) => l.name),
|
||||
...(automatic ? automatic.context : {}),
|
||||
};
|
||||
const [review] = await db
|
||||
.insert(chatGitHubReviews)
|
||||
.values({
|
||||
companyId: source.endpoint.companyId,
|
||||
endpointId: source.endpoint.id,
|
||||
issueId: source.issue.id,
|
||||
runId: source.run.id,
|
||||
repositoryId: source.repositoryId,
|
||||
repository: source.repository,
|
||||
pullNumber: source.number,
|
||||
headSha: pull.head.sha,
|
||||
deliveryId: `run:${source.run.id}:${pull.head.sha}`,
|
||||
configurationRevision:
|
||||
manual?.revision ?? automatic?.revision ?? source.config.revision,
|
||||
policySnapshot: manual?.policy ?? automatic?.policy ?? source.policy,
|
||||
event: context,
|
||||
state: "running",
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning();
|
||||
const [review] = await db.transaction(async tx => {
|
||||
await notifyGitHubReviewChecks(tx, { companyId: source.endpoint.companyId, endpointId: source.endpoint.id,
|
||||
repositoryId: source.repositoryId, number: source.number });
|
||||
return tx
|
||||
.insert(chatGitHubReviews)
|
||||
.values({
|
||||
companyId: source.endpoint.companyId,
|
||||
endpointId: source.endpoint.id,
|
||||
issueId: source.issue.id,
|
||||
runId: source.run.id,
|
||||
repositoryId: source.repositoryId,
|
||||
repository: source.repository,
|
||||
pullNumber: source.number,
|
||||
headSha: pull.head.sha,
|
||||
deliveryId: `run:${source.run.id}:${pull.head.sha}`,
|
||||
configurationRevision:
|
||||
manual?.revision ?? automatic?.revision ?? source.config.revision,
|
||||
policySnapshot: manual?.policy ?? automatic?.policy ?? source.policy,
|
||||
event: context,
|
||||
state: "running",
|
||||
})
|
||||
.onConflictDoNothing()
|
||||
.returning();
|
||||
});
|
||||
if (review) {
|
||||
await githubReviewCheckService(db, fetchImpl).enqueue(
|
||||
source.endpoint,
|
||||
|
||||
@@ -9,6 +9,7 @@ import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notificatio
|
||||
export type ChatRunMilestoneScope = { companyId: string; issueId?: string };
|
||||
|
||||
export async function notifyChatRunMilestones(tx: object, scope: ChatRunMilestoneScope) {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubReviewChecks, JSON.stringify({ companyId: scope.companyId }));
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatGitHubTerminalResponses, JSON.stringify({ companyId: scope.companyId, issueId: scope.issueId }));
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.chatRunMilestones, JSON.stringify({ companyId: scope.companyId, issueId: scope.issueId }));
|
||||
// Lifecycle and coarse native progress share lineage and authority. Extend
|
||||
|
||||
@@ -41,6 +41,7 @@ export const DELIVERY_QUEUES = {
|
||||
chatProviderEffects: "chat-provider-effects",
|
||||
chatGitHubIngress: "chat-github-ingress",
|
||||
chatGitHubPublications: "chat-github-publications",
|
||||
chatGitHubReviewChecks: "chat-github-review-checks",
|
||||
chatGitHubTerminalResponses: "chat-github-terminal-responses",
|
||||
chatGitHubAcknowledgements: "chat-github-acknowledgements",
|
||||
chatFailedRunRetries: "chat-failed-run-retries",
|
||||
|
||||
Reference in new issue
Block a user