From 92271da8991cbfc144240ab37a12b40d8a00c603 Mon Sep 17 00:00:00 2001 From: paperclipbugbot Date: Fri, 20 Nov 2026 06:47:34 -0800 Subject: [PATCH] fix(native): keep a chat turn's message batch when a fresh-session retry continues it (#22413) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External chat turns (GitHub, Slack, Discord) wake an agent on a task with an admitted message batch. > - When a task is in review, dispatch checks that the run is bound to that reviewed chat batch. A run without a batch fails with `reviewed_chat_execution_binding_not_authorized` (`context_mismatch`). > - Automatic fresh-session retries (tool-receipt limit, provider capacity, environment blocker, agent blocker, and safe replacement) remove all consumed wake input from the failed run's context. This also removes the chat message IDs, but the context keeps `source: chat:`. > - Thus every automatic retry of a reviewed chat turn fails at setup. The one-shot review-path recovery that follows copies the same empty context and fails the same way. > - This pull request keeps the admitted chat message batch on those retries and removes the failed run's dispatch proofs, so dispatch checks ownership and current access again. > - The benefit is that a reviewed chat turn can continue after a transient failure, and access checks still apply. ## Linked Issues or Issue Description Refs #13818 (the same approach for review-path recovery). Related: #20689, #20697, #20842 (other causes of the same denial). **What happened?** A GitHub PR review turn on a task in review stopped on a transient failure (tool-receipt limit or sandbox environment blocker). Paperclip scheduled an automatic fresh-session retry. The retry failed at setup in less than one second with `reviewed_chat_execution_binding_not_authorized` and denial reason `context_mismatch`. The review-path recovery wake then failed with the same error. On staging this occurred 5 times on 3 tasks after #20689, #20697 and #20842 were deployed. **Expected behavior** An automatic retry of a chat turn keeps the admitted message batch. Dispatch then checks the retry against current task ownership and current endpoint, conversation and principal access. If access is still valid, the retry runs. If access was revoked, dispatch refuses the retry. **Steps to reproduce** 1. Put a task in review with a run-bound external chat turn (for example `source: chat:github` with `wakeCommentIds`). 2. Fail that native run with a fresh-session retry lane (for example `native_environment_blocker`). 3. Let dispatch claim the scheduled retry. The retry context has `source: chat:github` but no `wakeCommentIds`. 4. Dispatch refuses the retry with `context_mismatch`. **Paperclip version or commit** `master` at 414abc2dbf. ## What Changed - Add `server/src/services/native-runtime/fresh-session-retry-context.ts`. It holds the one list of consumed wake input keys and builds the retry context. For a run-bound external chat turn, it keeps the admitted message IDs (`wakeCommentIds`, `wakeCommentId`, `commentId`). It removes the dispatch proofs that the failed run got (`paperclipHarnessCheckedOut`, `paperclipExternalChatExecutionBound`). Other runs keep the existing behavior. - Use the helper in `materializeNativeFreshSessionRetry` (all its lanes) and in `reconcileSafeNativeReplacements`. Before, each producer had its own copy of the key list. - Add a unit test for the helper. - Add a regression test in `heartbeat-reviewed-chat-binding.integration.test.ts` for Slack and Discord. It checks that the retry passes the reviewed chat check, that revoked access still refuses the retry, and that review-path recovery keeps the batch. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-reviewed-chat-binding.integration.test.ts src/services/native-runtime/fresh-session-retry-context.test.ts src/services/native-runtime/native-safe-replacement.test.ts src/services/native-runtime/native-agent-blocker.integration.test.ts src/services/native-runtime/native-tool-receipt-limit.integration.test.ts` passes: 5 files, 127 tests. - Red check: with the `master` versions of `native-provider-capacity-retry.ts` and `native-safe-replacement.ts`, the new regression test fails (2 failed, 34 passed). The retry is refused with `context_mismatch`. - CI runs the full typecheck. The local full server typecheck stopped because the sandbox ran out of memory. ## Risks - Low risk. The change applies only to automatic fresh-session retries whose context has a run-bound external chat source and admitted message IDs. - The retry does not get the failed run's authority. Dispatch must check ownership and access again, and the regression test shows that a revoked principal is still refused. - Other retries keep the same context as before, because both producers now use the same key list. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 --- ...-reviewed-chat-binding.integration.test.ts | 46 +++++++++++++++++++ .../fresh-session-retry-context.test.ts | 32 +++++++++++++ .../fresh-session-retry-context.ts | 25 ++++++++++ .../native-provider-capacity-retry.ts | 13 +----- .../native-runtime/native-safe-replacement.ts | 12 +---- 5 files changed, 107 insertions(+), 21 deletions(-) create mode 100644 server/src/services/native-runtime/fresh-session-retry-context.test.ts create mode 100644 server/src/services/native-runtime/fresh-session-retry-context.ts diff --git a/server/src/__tests__/heartbeat-reviewed-chat-binding.integration.test.ts b/server/src/__tests__/heartbeat-reviewed-chat-binding.integration.test.ts index 4dc32cbdd09..1b538d99577 100644 --- a/server/src/__tests__/heartbeat-reviewed-chat-binding.integration.test.ts +++ b/server/src/__tests__/heartbeat-reviewed-chat-binding.integration.test.ts @@ -25,6 +25,7 @@ import { nativeRunResults, statusDecisionEffects, workAssessments, + type Db, } from "@paperclipai/db"; import { renderPaperclipWakePrompt } from "@paperclipai/adapter-utils/server-utils"; import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js"; @@ -38,6 +39,7 @@ import { } from "../services/native-runtime/chat-attachment-reuse.js"; import * as chatAttachmentReuse from "../services/native-runtime/chat-attachment-reuse.js"; import { decideIssueReviewPathRecovery } from "../services/recovery/review-path-recovery.js"; +import { materializeNativeToolReceiptLimitContinuation } from "../services/native-runtime/native-provider-capacity-retry.js"; import { resolveCurrentWakeCommentsBinding } from "../services/native-runtime/current-wake-comments.js"; import { commitNativeStatusDecision } from "../services/native-runtime/status-decision-committer.js"; import { NATIVE_STATUS_ARBITER_POLICY_VERSION } from "../services/native-runtime/status-arbiter.js"; @@ -451,6 +453,50 @@ describe.each(["slack", "discord"] as const)("reviewed %s execution binding", (p }); }); + it("reauthorizes a fresh-session continuation of a reviewed chat turn from its message batch", async () => { + const failedRunId = randomUUID(); + await db.insert(heartbeatRuns).values({ + id: failedRunId, companyId, agentId, status: "cancelled", runtimeMode: "native", nativeIssueId: issueId, + invocationSource: "assignment", triggerDetail: "system", + contextSnapshot: { ...context, paperclipHarnessCheckedOut: false, paperclipExternalChatExecutionBound: true }, + }); + const retryId = await db.transaction((tx) => materializeNativeToolReceiptLimitContinuation({ + tx: tx as unknown as Db, companyId, issueId, runId: failedRunId, agentId, + })); + const [retry] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, retryId)); + const retryContext = retry!.contextSnapshot as Record; + const [before] = await db.select().from(issues).where(eq(issues.id, issueId)); + await db.update(heartbeatRuns).set({ status: "running" }).where(eq(heartbeatRuns.id, retryId)); + await db.update(issues).set({ executionRunId: retryId }).where(eq(issues.id, issueId)); + const reasons: string[] = []; + const attestRetry = () => attestReviewedExternalChatRun({ + db, ...binding, runId: retryId, contextSnapshot: { ...retryContext }, onDenied: (reason) => reasons.push(reason), + }); + try { + await expect(attestRetry()).resolves.toBe(true); + expect(reasons).toEqual([]); + await db.update(companyMemberships).set({ status: "suspended" }).where(eq(companyMemberships.principalId, userId)); + try { + await expect(attestRetry()).resolves.toBe(false); + expect(reasons).toEqual(["principal_denied"]); + } finally { + await db.update(companyMemberships).set({ status: "active" }).where(eq(companyMemberships.principalId, userId)); + } + expect((await db.select().from(issues).where(eq(issues.id, issueId)))[0]) + .toEqual({ ...before, executionRunId: retryId, updatedAt: expect.any(Date) }); + } finally { + await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId)); + await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, retryId)); + } + expect(retryContext).toMatchObject({ source: `chat:${provider}`, wakeCommentIds: [commentId], retryOfRunId: failedRunId }); + expect(retryContext).not.toHaveProperty("paperclipHarnessCheckedOut"); + expect(retryContext).not.toHaveProperty("paperclipExternalChatExecutionBound"); + expect(decideIssueReviewPathRecovery({ + issueId, sourceRunId: retryId, assigneeAgentId: agentId, contextSnapshot: retryContext, + reviewAttention: { state: "stalled", paths: [], reason: "Review path consumed" }, existingWake: false, + })).toMatchObject({ kind: "enqueue", contextSnapshot: { wakeCommentIds: [commentId] } }); + }); + it("denies recovery after access revocation or a changed company, task, or message binding", async () => { await withReviewRecovery(async (recovery) => { await expect(attest(recovery)).resolves.toBe(true); diff --git a/server/src/services/native-runtime/fresh-session-retry-context.test.ts b/server/src/services/native-runtime/fresh-session-retry-context.test.ts new file mode 100644 index 00000000000..c7edd3f9146 --- /dev/null +++ b/server/src/services/native-runtime/fresh-session-retry-context.test.ts @@ -0,0 +1,32 @@ +import { describe, expect, it } from "vitest"; +import { freshSessionRetryContext } from "./fresh-session-retry-context.js"; + +const consumed = { + wakeCommentId: "c2", wakeCommentIds: ["c1", "c2"], commentId: "c2", commentIds: ["c1", "c2"], latestCommentId: "c2", + explicitUserContinuation: true, resumeIntent: true, followUpRequested: true, paperclipWake: {}, paperclipWakeComment: {}, + paperclipTaskMarkdown: "task", paperclipTurnContext: {}, +}; + +describe("freshSessionRetryContext", () => { + it("drops consumed wake input from an ordinary task run", () => { + const context = freshSessionRetryContext({ issueId: "issue", source: "issue.comment", paperclipHarnessCheckedOut: true, ...consumed }); + expect(context).toEqual({ issueId: "issue", source: "issue.comment", paperclipHarnessCheckedOut: true }); + }); + + it.each(["chat:github", "chat:slack:recovery"])("keeps the admitted %s message batch without the failed run's proofs", (source) => { + const context = freshSessionRetryContext({ + issueId: "issue", source, chatConversationId: "conversation", paperclipHarnessCheckedOut: false, + paperclipExternalChatExecutionBound: true, ...consumed, + }); + expect(context).toEqual({ + issueId: "issue", source, chatConversationId: "conversation", + wakeCommentIds: ["c1", "c2"], wakeCommentId: "c2", commentId: "c2", + }); + }); + + it("does not invent a batch or keep one for sources outside the run-bound chat boundary", () => { + expect(freshSessionRetryContext({ source: "chat:github", wakeCommentId: "c1" })).toEqual({ source: "chat:github" }); + expect(freshSessionRetryContext({ source: "chat:agentmail", wakeCommentIds: ["c1"] })).toEqual({ source: "chat:agentmail" }); + expect(freshSessionRetryContext(null)).toEqual({}); + }); +}); diff --git a/server/src/services/native-runtime/fresh-session-retry-context.ts b/server/src/services/native-runtime/fresh-session-retry-context.ts new file mode 100644 index 00000000000..de2bdd2d7b6 --- /dev/null +++ b/server/src/services/native-runtime/fresh-session-retry-context.ts @@ -0,0 +1,25 @@ +import { extractWakeCommentIds } from "../../modules/run-dispatch/index.js"; +import { boundExternalChatProvider } from "./external-chat-provider.js"; + +const CONSUMED_WAKE_INPUT_KEYS = [ + "explicitUserContinuation", "wakeCommentId", "wakeCommentIds", "commentId", + "commentIds", "latestCommentId", "resumeIntent", "followUpRequested", + "paperclipWake", "paperclipWakeComment", "paperclipTaskMarkdown", "paperclipTaskMarkdownCompact", + "paperclipTaskMarkdownAssignment", "paperclipTaskMarkdownAssignmentCompact", "paperclipTurnContext", +] as const; + +const DISPATCH_ATTESTATION_KEYS = ["paperclipHarnessCheckedOut", "paperclipExternalChatExecutionBound"] as const; + +export function freshSessionRetryContext(predecessor: Record | null | undefined) { + const context: Record = { ...predecessor }; + const chatCommentIds = boundExternalChatProvider(context.source) ? extractWakeCommentIds(context) : []; + for (const key of CONSUMED_WAKE_INPUT_KEYS) delete context[key]; + if (chatCommentIds.length === 0) return context; + for (const key of DISPATCH_ATTESTATION_KEYS) delete context[key]; + return { + ...context, + wakeCommentIds: chatCommentIds, + wakeCommentId: chatCommentIds.at(-1), + commentId: chatCommentIds.at(-1), + }; +} diff --git a/server/src/services/native-runtime/native-provider-capacity-retry.ts b/server/src/services/native-runtime/native-provider-capacity-retry.ts index 98d5db313e0..636bf4606d2 100644 --- a/server/src/services/native-runtime/native-provider-capacity-retry.ts +++ b/server/src/services/native-runtime/native-provider-capacity-retry.ts @@ -12,6 +12,7 @@ import { NATIVE_ENVIRONMENT_BLOCKER_CODE, NATIVE_ENVIRONMENT_BLOCKER_HINT, NATIV import { NATIVE_AGENT_BLOCKER_CONTINUATION_CODE, NATIVE_AGENT_BLOCKER_CONTINUATION_DELAY_MS, NATIVE_AGENT_BLOCKER_HINT, NATIVE_AGENT_BLOCKER_MAX_CONTINUATIONS, NATIVE_AGENT_BLOCKER_MESSAGE } from "./native-agent-blocker.js"; import { NativeStatusRaceError } from "./status-decision-committer.js"; +import { freshSessionRetryContext } from "./fresh-session-retry-context.js"; type NativeRetryTarget = { tx: Db; @@ -112,18 +113,8 @@ async function materializeNativeFreshSessionRetry(input: NativeRetryTarget, lane const now = new Date(); const dueAt = new Date(now.getTime() + lane.delayMs(attempt)); const retryId = randomUUID(); - const predecessorContext = { ...run.contextSnapshot }; - // Resume receipts and delivered wake input belong to the failed run. The - // new turn has ordinary task authority; history comes from retryOfRunId. - // Match the consumed-input boundary used by native safe replacements. - for (const key of [ - "explicitUserContinuation", "wakeCommentId", "wakeCommentIds", "commentId", - "commentIds", "latestCommentId", "resumeIntent", "followUpRequested", - "paperclipWake", "paperclipWakeComment", "paperclipTaskMarkdown", "paperclipTaskMarkdownCompact", - "paperclipTaskMarkdownAssignment", "paperclipTaskMarkdownAssignmentCompact", "paperclipTurnContext", - ]) delete predecessorContext[key]; const contextSnapshot = { - ...predecessorContext, + ...freshSessionRetryContext(run.contextSnapshot), issueId: input.issueId, taskId: input.issueId, forceFreshSession: true, diff --git a/server/src/services/native-runtime/native-safe-replacement.ts b/server/src/services/native-runtime/native-safe-replacement.ts index 0ca6f833b50..bcb960876bc 100644 --- a/server/src/services/native-runtime/native-safe-replacement.ts +++ b/server/src/services/native-runtime/native-safe-replacement.ts @@ -22,6 +22,7 @@ import { } from "@paperclipai/db"; import { readRemoteWorkspaceAuthority, remoteWorkspaceRunReferenceSchema } from "@paperclipai/shared"; import { decideNativeReplacement, type NativeReplacementDecision } from "./native-replacement-evidence.js"; +import { freshSessionRetryContext } from "./fresh-session-retry-context.js"; import { issueService } from "../issues.js"; import { issueRecoveryActionService } from "../issue-recovery-actions.js"; import { buildExecutionContinuation } from "../execution-continuation.js"; @@ -778,17 +779,8 @@ export async function reconcileSafeNativeReplacements( } const successorRunId = randomUUID(); const dueAt = new Date(now.getTime() + 30_000); - const predecessorContext = { ...record(run.contextSnapshot) }; - // History comes from the failed source run. Consumed wake fields must - // not grant this automatic retry fresh comment/resume authority. - for (const key of [ - "explicitUserContinuation", "wakeCommentId", "wakeCommentIds", "commentId", - "commentIds", "latestCommentId", "resumeIntent", "followUpRequested", - "paperclipWake", "paperclipWakeComment", "paperclipTaskMarkdown", "paperclipTaskMarkdownCompact", - "paperclipTaskMarkdownAssignment", "paperclipTaskMarkdownAssignmentCompact", "paperclipTurnContext", - ]) delete predecessorContext[key]; const context = { - ...predecessorContext, + ...freshSessionRetryContext(record(run.contextSnapshot)), issueId: task.id, retryOfRunId: run.id, wakeReason: NATIVE_SAFE_REPLACEMENT_REASON,