From 1529bca5c1a45a5efbfb5a2ac401aad810ec22c7 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Fri, 13 Nov 2026 09:49:22 -0600 Subject: [PATCH] fix(chat): do not post a completion report that an earlier reply already gave (#21401) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - In agent chat, a delegated task that finishes wakes the chat agent with `chat_task_completed`, so it tells the person in the same conversation > - A turn that runs for another reason, such as the person's form answer, can read the task when it is already done and report it > - The completion turn then starts after it and reports the same news again > - The person gets two completion messages for one task > - This pull request records when the chat agent reads its handed-off task in the done state, and skips the completion turn when an earlier reply already reported that completion > - The benefit is one completion report per task, and no extra model call for it ## Linked Issues or Issue Description **What happened?** In chat eval cohort 12, case `questions-and-approvals.duplicate-answer`: the delegated draft task finished at 13:48:23 while the chat agent's turn (woken by the form answer at 13:48:16) was running. That turn read the task, saw it done and replied "Acorn is confirmed ... the draft is finished" with a task link (13:48:50). The `chat_task_completed` turn started at 13:48:53 and replied "The Acorn draft is done" (13:49:22). **Expected behavior** The person gets one completion report per finished task in the conversation. **Steps to reproduce** 1. In chat, delegate a short task. 2. While it runs, send a message that wakes the chat agent. 3. The task finishes during that turn, and the turn reads and reports it. 4. See a second completion report from the `chat_task_completed` turn. ## What Changed - `server/src/services/chat-completion-delivery.ts`: `recordCompletedHandoffRead` appends a `chat_handoff.completed_read` run event when the conversation agent reads its own handed-off task in the done state (with the task's status version). - `prepareChatCompletionTurn`: a delivery is marked delivered (with the earlier reply as its response) when an earlier reply of the same agent in the conversation was posted after completion, links the task, and comes from a run that read the task at that exact done status. Both conditions are structural (run event and task reference), not reply wording. A reply that links the task without such a read (for example "Morgan is working on it", from a read before completion) does not count. - `server/src/services/heartbeat.ts`: when every delivery of the completion turn was already reported, the run ends as succeeded before the conversation turn starts, with no model call. - `server/src/routes/issues.ts`: task reads by an agent run call the recorder (one primary-key read of the handoff, and an event only for the chat agent's own done handoffs). ## Verification - `cd server && npx vitest run src/__tests__/chat-completion-delivery.test.ts`: 53 pass, including two new tests: the already-reported completion is delivered without a turn, and a stale linking reply still gets the completion turn. - Red: with the earlier-reply check disabled, "does not report a completion again ..." fails (`expected false to be true`). - A narrow paid eval run of `questions-and-approvals.duplicate-answer` follows on this branch. ## Risks - If the earlier reply linked the task but did not say it was finished, the person does not get a second message. The read-at-done-status condition makes this unlikely. - The skipped completion run is a short succeeded run with no model call. Boot and time to first token are unchanged for all other turns. ## Model Used - Claude, claude-opus-5-5, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../chat-completion-delivery.test.ts | 39 ++++++++++- server/src/routes/issues.ts | 8 ++- .../src/services/chat-completion-delivery.ts | 66 ++++++++++++++++++- server/src/services/heartbeat.ts | 9 +++ 4 files changed, 116 insertions(+), 6 deletions(-) diff --git a/server/src/__tests__/chat-completion-delivery.test.ts b/server/src/__tests__/chat-completion-delivery.test.ts index df769c99372..86595527ae0 100644 --- a/server/src/__tests__/chat-completion-delivery.test.ts +++ b/server/src/__tests__/chat-completion-delivery.test.ts @@ -1,7 +1,7 @@ import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js"; import { subscribeDeliveryWork, DELIVERY_QUEUES } from "../services/delivery-work-notifications.js"; import { randomUUID } from "node:crypto"; -import { and, asc, eq } from "drizzle-orm"; +import { and, asc, eq, sql } from "drizzle-orm"; import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { agents, agentWakeupRequests, chatCompletionDeliveries as deliveries, chatTaskHandoffs as handoffs, companies, createDb, documents, environmentLeases, executionWorkspaces, heartbeatRuns, issueComments, issueRecoveryActions, issueThreadInteractions, issues, workspaceOperations } from "@paperclipai/db"; @@ -13,7 +13,7 @@ import { executeIssuePostCommitActions, type IssuePostCommitAction, issueService import { buildLowTrustSourceTrust } from "../services/source-trust.js"; import { documentService } from "../services/documents.js"; import { instanceSettingsService } from "../services/instance-settings.js"; -import { chatCompletionDeliveryService, isCompletedOnboardingHandoffWake, prepareChatCompletionTurn, recordChatCompletion, recordChatHandoff } from "../services/chat-completion-delivery.js"; +import { chatCompletionAlreadyReported, chatCompletionDeliveryService, isCompletedOnboardingHandoffWake, prepareChatCompletionTurn, recordChatCompletion, recordChatHandoff, recordCompletedHandoffRead } from "../services/chat-completion-delivery.js"; import { heartbeatService, shouldQueueFollowupForRunningIssueWake } from "../services/heartbeat.js"; import { createHeartbeatQueue } from "../services/heartbeat/queue.js"; import { createHeartbeatQueueTestDependencies } from "./helpers/heartbeat-queue.js"; @@ -54,6 +54,41 @@ import { createHeartbeatQueueTestDependencies } from "./helpers/heartbeat-queue. }; return { companyId, agentId, sourceId, runId, task, create, finish, rows, due, wakeup, service, run }; } + it("does not report a completion again when an earlier reply read the done task and linked it", async () => { + const f = await seed(), identifier = `CHAT-${Math.floor(Math.random() * 1e6)}`; + await db.update(issues).set({ identifier }).where(eq(issues.id, f.task.id)); + // The person's own answer woke a turn that was running when the task finished. + const earlierRunId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: earlierRunId, companyId: f.companyId, agentId: f.agentId, status: "running", + contextSnapshot: { issueId: f.sourceId, conversationSessionGeneration: 0 } }); + await f.finish(); + // An owner change while done advances the version without a new completion. + await db.update(issues).set({ statusVersion: sql`${issues.statusVersion} + 1` }).where(eq(issues.id, f.task.id)); + const [done] = await db.select().from(issues).where(eq(issues.id, f.task.id)); + await recordCompletedHandoffRead(db, { companyId: f.companyId, agentId: f.agentId, runId: earlierRunId, task: done }); + // Unrelated replies from other turns come first and must not hide the report. + await db.insert(issueComments).values(Array.from({ length: 25 }, (_, index) => ({ companyId: f.companyId, issueId: f.sourceId, + authorAgentId: f.agentId, createdByRunId: f.runId, body: `Other update ${index}`, createdAt: new Date(Date.now() + 500) }))); + const [reply] = await db.insert(issueComments).values({ companyId: f.companyId, issueId: f.sourceId, authorAgentId: f.agentId, + createdByRunId: earlierRunId, body: `The draft is finished: [${identifier}](/CHAT/issues/${identifier}).`, createdAt: new Date(Date.now() + 1000) }).returning(); + const prepared = await f.run(); + expect(chatCompletionAlreadyReported(prepared)).toBe(true); + expect(await f.rows()).toMatchObject([{ status: "delivered", responseCommentId: reply.id }]); + }); + it("still reports a completion when the earlier reply linked the task without reading it done", async () => { + const f = await seed(), identifier = `CHAT-${Math.floor(Math.random() * 1e6)}`; + await db.update(issues).set({ identifier }).where(eq(issues.id, f.task.id)); + const earlierRunId = randomUUID(); + await db.insert(heartbeatRuns).values({ id: earlierRunId, companyId: f.companyId, agentId: f.agentId, status: "running", + contextSnapshot: { issueId: f.sourceId, conversationSessionGeneration: 0 } }); + await f.finish(); + // A turn that read the task while it was still in progress, then replied after it finished. + await db.insert(issueComments).values({ companyId: f.companyId, issueId: f.sourceId, authorAgentId: f.agentId, + createdByRunId: earlierRunId, body: `Morgan is working on [${identifier}](/CHAT/issues/${identifier}).`, createdAt: new Date(Date.now() + 1000) }); + const prepared = await f.run(); + expect(chatCompletionAlreadyReported(prepared)).toBe(false); + expect(prepared.contextSnapshot?.chatCompletionUpdates).toEqual([expect.objectContaining({ id: f.task.id, status: "done" })]); + }); it("supplies canonical human output verdicts without copying review prose or unrelated approvals", async () => { const f = await seed(), reviewId = randomUUID(), revisionId = randomUUID(); await db.insert(issueThreadInteractions).values([ diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.ts index b1443ab04f6..a13c0bba7a0 100644 --- a/server/src/routes/issues.ts +++ b/server/src/routes/issues.ts @@ -25,6 +25,7 @@ import { assertNoCaseHandoffMutation } from "../services/heartbeat/case-handoff- import { assertHandedOffRunTaskOwnership } from "../run-task-ownership.js"; import { recordNativeHumanSelfHandoff } from "../services/native-runtime/native-human-self-handoff.js"; import { recordChildTaskRead } from "../services/native-runtime/native-child-completion-delivery.js"; +import { recordCompletedHandoffRead } from "../services/chat-completion-delivery.js"; import { runRetainsIssueIdentity } from "../services/task-repository-acquisition.js"; import { withConversationReadState } from "../services/agent-chat-read-state.js"; import { markIssueReadSchema } from "@paperclipai/shared"; @@ -3613,8 +3614,11 @@ function logIssueListRequest(input: { }); } -async function recordAgentChildTaskRead(db: Db, req: Request, issue: { id: string; companyId: string; parentId: string | null; status: string; updatedAt: Date; lastStatusDecisionId?: string | null }) { - if (req.actor.type !== "agent" || !req.actor.agentId || !req.actor.runId || !issue.parentId || issue.status !== "done") return; +async function recordAgentChildTaskRead(db: Db, req: Request, issue: { id: string; companyId: string; parentId: string | null; status: string; updatedAt: Date; lastStatusDecisionId?: string | null; statusVersion?: number | null }) { + if (req.actor.type !== "agent" || !req.actor.agentId || !req.actor.runId || issue.status !== "done") return; + await recordCompletedHandoffRead(db, { companyId: issue.companyId, agentId: req.actor.agentId, runId: req.actor.runId, task: issue }) + .catch(error => logger.warn({ err: error, runId: req.actor.runId }, "could not record the completed handoff a run read")); + if (!issue.parentId) return; await recordChildTaskRead(db, { companyId: issue.companyId, agentId: req.actor.agentId, runId: req.actor.runId, parentIssueId: issue.parentId, children: [issue] }) .catch(error => logger.warn({ err: error, runId: req.actor.runId }, "could not record the child task a run read")); diff --git a/server/src/services/chat-completion-delivery.ts b/server/src/services/chat-completion-delivery.ts index e559044b894..4b65dd235e1 100644 --- a/server/src/services/chat-completion-delivery.ts +++ b/server/src/services/chat-completion-delivery.ts @@ -1,13 +1,17 @@ import { notifyDeliveryWork, DELIVERY_QUEUES } from "./delivery-work-notifications.js"; import { notifyNativeResponsePresentation } from "./native-response-presentation-notifications.js"; import { nativeResponsePresentationCandidates } from "./native-runtime/native-response-presentation-query.js"; -import { and, asc, desc, eq, inArray, isNull, lte, sql } from "drizzle-orm"; +import { and, asc, desc, eq, gte, inArray, isNotNull, isNull, lte, ne, sql } from "drizzle-orm"; import { agents, agentWakeupRequests, chatCompletionDeliveries as deliveries, chatTaskHandoffs as handoffs, - heartbeatRuns, issueComments, issueDocuments, issueThreadInteractions, issues, type Db } from "@paperclipai/db"; + heartbeatRunEvents, heartbeatRuns, issueComments, issueDocuments, issueThreadInteractions, issues, type Db } from "@paperclipai/db"; +import { extractIssueReferenceMatches } from "@paperclipai/shared"; +import { appendHeartbeatRunEvent } from "./heartbeat-run-events.js"; import { instanceSettingsService } from "./instance-settings.js"; import { conflict, forbidden, notFound } from "../errors.js"; export const CHAT_COMPLETION_WAKE_REASON = "chat_task_completed"; +/** Run event: the conversation agent read its handed-off task after it was done. */ +export const COMPLETED_HANDOFF_READ_EVENT = "chat_handoff.completed_read"; const MAX_ATTEMPTS = 5; const LEASE_MS = 60_000; type Issue = typeof issues.$inferSelect; @@ -139,6 +143,45 @@ export async function isCompletedOnboardingHandoffWake(db: Connection, input: { children.every(child => ["done", "cancelled"].includes(child.status)); } +/** Records that the conversation agent read a handed-off task in its done state. The + * completion turn uses this to recognize a completion that an earlier reply reported. */ +export async function recordCompletedHandoffRead(db: Db, input: { + companyId: string; agentId: string; runId: string; + task: { id: string; status: string; statusVersion?: number | null }; +}) { + if (input.task.status !== "done" || typeof input.task.statusVersion !== "number") return; + const [handoff] = await db.select({ agentId: handoffs.agentId }).from(handoffs) + .where(and(eq(handoffs.taskId, input.task.id), eq(handoffs.companyId, input.companyId))); + if (handoff?.agentId !== input.agentId) return; + await appendHeartbeatRunEvent(db, { companyId: input.companyId, runId: input.runId, agentId: input.agentId, + eventType: COMPLETED_HANDOFF_READ_EVENT, stream: "system", level: "info", + payload: { taskId: input.task.id, statusVersion: input.task.statusVersion } }); +} + +/** A reply in this conversation already reported this completion when a turn of the same + * agent read the task done for this completion and then linked the task in its reply. */ +async function earlierCompletionReply(tx: Connection, row: NonNullable>>, runId: string) { + const { task, handoff, delivery } = row; + if (!task.completedAt || !task.identifier) return null; + // A reopen supersedes this delivery and a new completion gets a higher status + // version, so any done read at or after this version belongs to this completion, + // including after an owner change that advanced the version while still done. + const readThisCompletion = sql`exists (select 1 from ${heartbeatRunEvents} handoff_read + where handoff_read.company_id = ${handoff.companyId} and handoff_read.run_id = ${issueComments.createdByRunId} + and handoff_read.event_type = ${COMPLETED_HANDOFF_READ_EVENT} and handoff_read.payload->>'taskId' = ${task.id} + and (handoff_read.payload->>'statusVersion')::bigint >= ${delivery.statusVersion} + and handoff_read.created_at <= ${issueComments.createdAt})`; + const replies = await tx.select({ id: issueComments.id, body: issueComments.body }).from(issueComments).where(and( + eq(issueComments.companyId, handoff.companyId), eq(issueComments.issueId, handoff.conversationId), + eq(issueComments.authorAgentId, handoff.agentId), isNull(issueComments.deletedAt), + isNotNull(issueComments.createdByRunId), ne(issueComments.createdByRunId, runId), + gte(issueComments.createdAt, task.completedAt), readThisCompletion, + )).orderBy(asc(issueComments.createdAt), asc(issueComments.id)).limit(20); + const identifier = task.identifier.toUpperCase(); + return replies.find(reply => extractIssueReferenceMatches(reply.body) + .some(match => match.identifier.toUpperCase() === identifier))?.id ?? null; +} + /** Freeze the input at turn start. New completions cannot be consumed by an already-running turn. */ export async function prepareChatCompletionTurn(db: Db, run: Run): Promise { const issueId = run.contextSnapshot?.issueId; @@ -165,15 +208,28 @@ export async function prepareChatCompletionTurn(db: Db, run: Run): Promise .orderBy(asc(deliveries.createdAt)).limit(20); const updates: Awaited>[] = []; const accepted: string[] = []; + const reported: string[] = []; for (const { delivery } of rows) { const row = await loadAudience(tx, delivery.id); if (!row || !current(row)) { await tx.update(deliveries).set({ status: "superseded" }).where(eq(deliveries.id, delivery.id)); continue; } + const reportedBy = await earlierCompletionReply(tx, row, run.id); + if (reportedBy) { + // An earlier turn already told the person; repeating it would post the same news twice. + await tx.update(deliveries).set({ status: "delivered", responseCommentId: reportedBy, error: null }).where(eq(deliveries.id, delivery.id)); + reported.push(delivery.id); + continue; + } accepted.push(delivery.id); updates.push(await taskResult(tx, row.task)); await tx.update(deliveries).set({ status: "queued", targetRunId: run.id }).where(eq(deliveries.id, delivery.id)); } + if (accepted.length === 0 && reported.length > 0) { + const contextSnapshot = { ...run.contextSnapshot, chatCompletionDeliveryIds: [], chatCompletionAlreadyReported: reported }; + await tx.update(heartbeatRuns).set({ contextSnapshot }).where(eq(heartbeatRuns.id, run.id)); + return { ...run, contextSnapshot }; + } if (accepted.length === 0) throw new Error("chat_completion_superseded"); const contextSnapshot = { ...run.contextSnapshot, chatCompletionDeliveryIds: accepted, chatCompletionUpdates: updates }; await tx.update(heartbeatRuns).set({ contextSnapshot }).where(eq(heartbeatRuns.id, run.id)); @@ -181,6 +237,12 @@ export async function prepareChatCompletionTurn(db: Db, run: Run): Promise }); } +/** True when every completion this turn was woken for had already been reported. */ +export function chatCompletionAlreadyReported(run: Run) { + const reported = run.contextSnapshot?.chatCompletionAlreadyReported; + return Array.isArray(reported) && reported.length > 0 && ids(run).length === 0; +} + export function chatCompletionInstruction(context: Record) { if (!Array.isArray(context.chatCompletionUpdates) || !context.chatCompletionUpdates.length) return ""; return `\n\nDelegated work has completed. Tell the user in this conversation what finished and provide access using the supplied task links. Report completion only for the tasks listed in this update; other tasks receive their own completion updates. Use the recorded status and result locations; do not repeat a promise to do work that is already Done. Do not start more work, change these tasks, or create duplicate review requests. Reuse and link the task’s existing canonical review/decision; completion updates are notifications, not fresh quality-review authority. recordedOutputDecisions are human verdicts on the exact versions shown. Link them instead of re-requesting review of those versions; do not treat approval as permission to publish. A separate explicit user request may authorize a new review. Keep opaque IDs and hashes out of your visible reply. The following JSON contains server-recorded lifecycle facts and result locations:\n${JSON.stringify(context.chatCompletionUpdates)}`; diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.ts index 2e22d63ef4f..64d5ad5804b 100644 --- a/server/src/services/heartbeat.ts +++ b/server/src/services/heartbeat.ts @@ -281,6 +281,7 @@ import { } from "./native-runtime/native-cancellation-request.js"; import { prepareChatCompletionTurn, + chatCompletionAlreadyReported, } from "./chat-completion-delivery.js"; import { @@ -2197,6 +2198,14 @@ export function heartbeatService( conversation: isConversation(conversationIdentity), })) return; run = await timePreparation("chat_completion", () => prepareChatCompletionTurn(db, run)); + if (chatCompletionAlreadyReported(run)) { + // An earlier reply in this conversation already reported each completion. + await setRunStatus(run.id, "succeeded", { finishedAt: new Date(), resultJson: { chatCompletionAlreadyReported: true }, issueCommentStatus: "not_applicable" }); + await setWakeupStatus(run.wakeupRequestId, "completed", { finishedAt: new Date() }); + await releaseIssueExecutionAndPromote((await getRun(run.id))!, { suppressImmediateRecovery: true }); + await finalizeAgentStatus(agent.id, "succeeded"); + return; + } const preparedConversation = await timePreparation("conversation", () => prepareConversationTurn(db, run)); run = { ...run, contextSnapshot: preparedConversation.context }; if (preparedConversation.reset) {