mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 11:59:26 +01:00
fix: Defer chat milestone result reads and fix regression test transaction typing (#19875)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16728 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: PostgreSQL could evaluate both result_json predicates before publication filtering, detoasting already-published runs. The regression test also passed transaction handles to helpers typed for full database handles. - This pull request: Defer chat milestone result reads and fix regression test transaction typing ## What Changed - `server/src/services/chat-run-publications.ts`: Retain publication-dependent guards around both result JSON predicates in the shared milestone query. - `server/src/__tests__/chat-run-milestone-toast.test.ts`: Use the neighboring tests' transaction cast convention. Confirmed both cases fail without the guards and pass with them; all seven selected milestone test files and typecheck passed. ## Verification - Reproducing test(s), added in this PR: `server/src/__tests__/chat-run-milestone-candidates.test.ts`, `server/src/__tests__/chat-run-milestone-toast.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - The repository's compile command passes with the fix applied. - 8 existing test file(s) near the change failed, and fail on the base too, so they cannot show whether the change breaks them. - The full test suite passes except for the 15 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. ## Model Used - opus (Anthropic Claude, via Claude Code), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16728 The reviewer found the base already handles: - Milestone projection still runs on a one-second cadence. (server/src/services/chat-run-milestone-work.ts:12 describes and implements discovery on startup/committed changes with retries for failures or unfinished batches; server/src/app.ts:1269 registers this work owner. This code exists independently of the proposed predicate change.) Other pull requests that name #16728: - #18686 (open): perf(runs): read result_json once per run in run lists and index agent runs by created_at ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16728 base=d1ee25d0f722 report=5924b3e8dd6d2fe4 --> --------- Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
633c8076c0
commit
0eeee6ae96
3 files changed
+113
-5
No files matched your search
@@ -2,12 +2,13 @@ import { randomUUID } from "node:crypto";
|
||||
import { and, asc, gte, sql } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
agents, chatConversations, chatEndpoints, companies, createDb, heartbeatRuns,
|
||||
agents, chatConversations, chatEndpoints, chatPublications, companies, createDb, heartbeatRuns,
|
||||
issues, toolApplications, toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { chatRunMilestoneSource } from "../services/chat-run-milestone-candidates.js";
|
||||
import { boundRunIssueIdSql } from "../services/run-task-binding.js";
|
||||
import { enqueueChatRunMilestones } from "../services/chat-run-publications.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
(support.supported ? describe : describe.skip)("chat milestone candidate prefilter", () => {
|
||||
@@ -22,6 +23,39 @@ const support = await getEmbeddedPostgresTestSupport();
|
||||
await database?.cleanup();
|
||||
});
|
||||
|
||||
it("does not fetch run result TOAST blocks for already published milestones", async () => {
|
||||
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID();
|
||||
const applicationId = randomUUID(), connectionId = randomUUID(), endpointId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Settled milestones", issuePrefix: "SETT" });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Chat agent" });
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Chat task" });
|
||||
await db.insert(toolApplications).values({ id: applicationId, companyId, name: "Slack", type: "chat" });
|
||||
await db.insert(toolConnections).values({ id: connectionId, companyId, applicationId, name: "Slack", uid: connectionId, transport: "chat_sdk", connectionPurpose: "channel" });
|
||||
await db.insert(chatEndpoints).values({ id: endpointId, companyId, connectionId, provider: "slack", publicId: endpointId, assignedAgentId: agentId });
|
||||
const [conversation] = await db.insert(chatConversations).values({ companyId, endpointId, issueId, externalConversationId: "settled", externalLabel: "Settled", state: "active" }).returning();
|
||||
const runIds = Array.from({ length: 6 }, () => randomUUID());
|
||||
for (const runId of runIds) {
|
||||
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "succeeded", contextSnapshot: { issueId, source: "chat:slack" }, resultJson: { presentationDecision: {}, output: Array.from({ length: 4000 }, () => randomUUID()).join("") } });
|
||||
await db.insert(chatPublications).values({ companyId, endpointId, conversationId: conversation.id, issueId, idempotencyKey: `run:${runId}:completed:${endpointId}`, state: "published", payload: { text: "Completed", progressState: "completed" } });
|
||||
}
|
||||
try {
|
||||
await db.transaction(async (tx) => {
|
||||
const fetched = async () => {
|
||||
const rows = await tx.execute(sql`select pg_stat_get_xact_blocks_fetched(reltoastrelid)::int as blocks from pg_class where oid = 'heartbeat_runs'::regclass`);
|
||||
return Number(rows[0].blocks);
|
||||
};
|
||||
const before = await fetched();
|
||||
expect(await enqueueChatRunMilestones(tx as unknown as typeof db)).toBe(0);
|
||||
const after = await fetched();
|
||||
await tx.execute(sql`select result_json -> 'presentationDecision' from heartbeat_runs where id = ${runIds[0]}`);
|
||||
expect(await fetched()).toBeGreaterThan(after);
|
||||
expect(after).toBe(before);
|
||||
});
|
||||
} finally {
|
||||
await db.update(chatConversations).set({ state: "completed" }).where(sql`${chatConversations.id} = ${conversation.id}`);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves complete rows, binding precedence, company scope and the recovery window", async () => {
|
||||
const companyId = randomUUID(), foreignCompanyId = randomUUID(), agentId = randomUUID();
|
||||
const issueId = randomUUID(), foreignIssueId = randomUUID(), closedIssueId = randomUUID();
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
agents, chatConversations, chatEndpoints, chatPublications, companies, createDb,
|
||||
heartbeatRuns, issues, toolApplications, toolConnections,
|
||||
} from "@paperclipai/db";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { enqueueChatRunMilestones, loadChatRunLifecycleMilestoneScopes } from "../services/chat-run-publications.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
(support.supported ? describe : describe.skip)("chat milestone result TOAST reads", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
beforeAll(async () => {
|
||||
database = await startEmbeddedPostgresTestDatabase("chat-milestone-toast-");
|
||||
db = createDb(database.connectionString);
|
||||
}, 90_000);
|
||||
afterAll(async () => {
|
||||
await db?.$client.end({ timeout: 0 });
|
||||
await database?.cleanup();
|
||||
});
|
||||
|
||||
it.each([false, true])("does not read published runs' results (decision present: %s)", async (hasDecision) => {
|
||||
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID();
|
||||
const applicationId = randomUUID(), connectionId = randomUUID(), endpointId = randomUUID();
|
||||
const conversationId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Toast", issuePrefix: hasDecision ? "TYES" : "TNO" });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Chat agent" });
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Active chat" });
|
||||
await db.insert(toolApplications).values({ id: applicationId, companyId, name: "Slack", type: "chat" });
|
||||
await db.insert(toolConnections).values({ id: connectionId, companyId, applicationId, name: "Slack", uid: connectionId, transport: "chat_sdk", connectionPurpose: "channel" });
|
||||
await db.insert(chatEndpoints).values({ id: endpointId, companyId, connectionId, provider: "slack", publicId: endpointId, assignedAgentId: agentId, publicationMode: "automatic" });
|
||||
await db.insert(chatConversations).values({ id: conversationId, companyId, endpointId, issueId, externalConversationId: conversationId, externalLabel: "Active", state: "active" });
|
||||
const runIds = Array.from({ length: 6 }, () => randomUUID());
|
||||
for (const runId of runIds) {
|
||||
await db.insert(heartbeatRuns).values({
|
||||
id: runId, companyId, agentId, status: "succeeded",
|
||||
contextSnapshot: { issueId, source: "chat:slack" },
|
||||
resultJson: {
|
||||
output: randomBytes(256_000).toString("hex"),
|
||||
...(hasDecision ? { presentationDecision: { commentId: null } } : {}),
|
||||
nativeResult: { reportedWorkDisposition: "completed" },
|
||||
},
|
||||
});
|
||||
await db.insert(chatPublications).values({
|
||||
companyId, endpointId, conversationId, issueId, state: "published",
|
||||
idempotencyKey: `run:${runId}:completed:${endpointId}`,
|
||||
payload: { text: "Completed", progressState: "completed" },
|
||||
});
|
||||
}
|
||||
await db.transaction(async (tx) => {
|
||||
const fetched = async () => {
|
||||
const [row] = await tx.execute<{ blocks: number }>(
|
||||
sql`select pg_stat_get_xact_blocks_fetched(reltoastrelid)::int as blocks
|
||||
from pg_class where oid = 'heartbeat_runs'::regclass`,
|
||||
);
|
||||
return row.blocks;
|
||||
};
|
||||
const before = await fetched();
|
||||
expect(await enqueueChatRunMilestones(tx as unknown as typeof db)).toBe(0);
|
||||
expect(await fetched()).toBe(before);
|
||||
expect(await loadChatRunLifecycleMilestoneScopes(tx as unknown as typeof db, { companyId })).toEqual([]);
|
||||
expect(await fetched()).toBe(before);
|
||||
await tx.execute(sql`select result_json -> 'presentationDecision'
|
||||
from heartbeat_runs where id = ${runIds[0]}`);
|
||||
expect(await fetched()).toBeGreaterThan(before);
|
||||
await tx.delete(chatPublications).where(eq(chatPublications.companyId, companyId));
|
||||
const withoutLineage = await fetched();
|
||||
expect(await enqueueChatRunMilestones(tx as unknown as typeof db)).toBe(0);
|
||||
expect(await fetched()).toBe(withoutLineage);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -473,15 +473,15 @@ function lifecycleMilestonePages(db: Db, since: Date, scope?: ChatRunMilestoneSc
|
||||
// Heartbeat marks a run succeeded before the presentation resolver
|
||||
// finishes. Waiting for its durable decision prevents a generic
|
||||
// completion from racing an explicitly authored final comment.
|
||||
sql`(
|
||||
sql`case when ${chatPublications.id} is null then (
|
||||
${heartbeatRuns.status} <> 'succeeded'
|
||||
or ${heartbeatRuns.resultJson} -> 'presentationDecision' is not null
|
||||
)`,
|
||||
sql`(${heartbeatRuns.status} <> 'succeeded' or coalesce(
|
||||
) else false end`,
|
||||
sql`case when ${chatPublications.id} is null then (${heartbeatRuns.status} <> 'succeeded' or coalesce(
|
||||
${heartbeatRuns.resultJson}->'nativeResult'->>'reportedWorkDisposition',
|
||||
${heartbeatRuns.resultJson}->'acceptedResult'->>'reportedWorkDisposition',
|
||||
${heartbeatRuns.resultJson}->'semanticResult'->'result'->>'reportedWorkDisposition', ''
|
||||
) <> 'yielded')`,
|
||||
) <> 'yielded') else false end`,
|
||||
// A successful or interrupted run's selected final remains its
|
||||
// authoritative response. Filter it before LIMIT so a page of settled
|
||||
// runs cannot starve later milestones.
|
||||
|
||||
Reference in new issue
Block a user