From 41b0f76d753b02dfbd87540aa7e1c11cd7250a3b Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 17 Nov 2026 03:53:12 -0600 Subject: [PATCH] fix(runner): tell a provider its accepted result is final when it reports again (#21984) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native run reports its result with `paperclip_finish` or `paperclip_block`, and a turn's first accepted report is final > - When the model reports again with a different result, the Runner refuses the second report > - After a committed blocker, the refusal tells the model to stop and end the turn (#21959) > - After a committed done, needs_review or yielded result, the refusal was only "A different semantic result was already committed", so the model did not know what to do and tried again > - Each extra call is one more model request, and the turn holds the agent's slot until it ends; the task's status commits only when the turn ends > - This pull request gives every refused later report a clear, final message: the result is already accepted, stop calling `paperclip_finish` and `paperclip_block`, put extra notes in the final response, and end the turn > - The benefit is that the model ends the turn after one refusal, so the slot is free sooner and the task completes sooner ## Linked Issues or Issue Description **What happened?** In a restart-chaos wave, an OpenCode agent (DeepSeek through OpenRouter) finished a task, and the Runner accepted its `done` report. The model then called `paperclip_finish` again with a different report. The Runner replied only "A different semantic result was already committed". The model repeated the call about 79 times over 277 seconds. The agent's slot stayed busy, and the task's completion was late, because the task status commits when the turn ends. **Expected behavior** After a turn commits any result (done, needs_review, yielded or blocked), each later report with a different result gets a clear and final instruction: the result is already accepted, so do not call `paperclip_finish` or `paperclip_block` again, put any extra notes in the final response, and end the turn. An identical repeat of the accepted report stays an idempotent success. **Steps to reproduce** 1. Start a native OpenCode turn that calls `paperclip_finish` with a done result. 2. In the same turn, call `paperclip_finish` again with a different summary, or call `paperclip_block`. 3. On master, the tool error is only "A different semantic result was already committed", with no instruction to stop. **Paperclip version or commit** master 54f8068df4 ## What Changed - `packages/paperclip-runner/src/contracts/completion-result.ts`: replace the bare `PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE` with `PRP_RESULT_ALREADY_ACCEPTED_MESSAGE`. `prpSemanticResultConflictMessage` returns it for every committed result that is not a blocker. The blocker messages share the same stop instruction, which now names both `paperclip_finish` and `paperclip_block`, and ask for extra notes in the final response. - The OpenCode, Codex and Codex ACPX drivers already return `prpSemanticResultConflictMessage(...)`, so all three get the new text with no driver change. - `packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs`: add a `finish-after-finish` scenario. After the accepted `paperclip_finish`, the fake provider sends an identical repeat, a changed `paperclip_finish` and a late `paperclip_block`. - `packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts`: a new test checks that the identical repeat succeeds, that both conflicting reports get the final message, that only one result is proposed, and that the turn completes. - `packages/paperclip-runner/src/contracts/completion-result.test.ts`: unit cases for done, needs_review, yielded and a missing committed result, and a check that both blocker messages name both tools and say to end the turn. - Not in this change: a hard bound that ends the turn after N refused reports. In the OpenCode driver the only way to stop a provider turn is `/abort`, which the driver reports as `turn.cancelled`, not `turn.completed`. A safe bound needs a new terminal path in each driver, so it is not "small". This change fixes the cause (the message) first. ## Verification - Red: with the old message, the 4 new `completion-result.test.ts` cases and the new OpenCode driver test fail (the refusal text does not contain the stop instruction). - Green: `npx vitest run src/contracts/completion-result.test.ts src/drivers/opencode/opencode-server-driver.test.ts` in `packages/paperclip-runner`: 2 files, 123 tests pass. - `npx vitest run src/drivers/acpx/codex-acpx-driver.test.ts src/drivers/codex`: 28 files, 563 tests pass. - `pnpm --filter @paperclipai/paperclip-runner typecheck` passes. - `pnpm verify` selected broad coverage (exit 2) because the shared OpenCode fixture changed. Required CI supplies that coverage. ## Risks - Low risk. Only the text of a tool error changes. Result admission, fingerprints and idempotent repeats do not change. - Mixed versions during a deploy: the Runner creates this text and sends it straight to the model. The server does not read or match it (no server, Rust or test code matches the old string). An old server with a new Runner gets the new text. A new server with an old Runner keeps the old text. Neither case changes a stored record or a protocol shape. - ACPX: no ACPX profile, pin, schema or `protocol/` file changes, so no provider `commandDigest` or pack contract changes. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 --- .../src/contracts/completion-result.test.ts | 20 +++++++++++-- .../src/contracts/completion-result.ts | 12 +++++--- .../opencode/opencode-server-driver.test.ts | 29 ++++++++++++++++++- .../test/fixtures/fake-opencode-server.mjs | 11 +++++++ 4 files changed, 65 insertions(+), 7 deletions(-) diff --git a/packages/paperclip-runner/src/contracts/completion-result.test.ts b/packages/paperclip-runner/src/contracts/completion-result.test.ts index 191754955c6..807ba5f0ec2 100644 --- a/packages/paperclip-runner/src/contracts/completion-result.test.ts +++ b/packages/paperclip-runner/src/contracts/completion-result.test.ts @@ -4,7 +4,7 @@ import { describe, expect, it } from "vitest"; import { PRP_BLOCKER_ALREADY_REPORTED_MESSAGE, PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE, - PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE, + PRP_RESULT_ALREADY_ACCEPTED_MESSAGE, prpSemanticResultConflictMessage, PRP_BLOCK_RESULT_OUTPUT_SCHEMA, PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA, @@ -303,7 +303,23 @@ describe("semantic result conflict feedback", () => { expect(prpSemanticResultConflictMessage(blocked("system", "tool_unavailable"))).toBe(PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE); expect(prpSemanticResultConflictMessage(blocked("user", "credential_missing"))).toBe(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE); expect(prpSemanticResultConflictMessage(blocked("system", "credential_missing"))).toBe(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE); - expect(prpSemanticResultConflictMessage({ reportedWorkDisposition: "done" })).toBe(PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE); expect(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE).not.toContain("automatically"); }); + + it.each(["done", "needs_review", "yielded"])("tells a provider that a %s result is final and to end the turn", disposition => { + const message = prpSemanticResultConflictMessage({ reportedWorkDisposition: disposition }); + expect(message).toBe(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE); + expect(message).toContain("already accepted"); + expect(message).toContain("Do not call paperclip_finish or paperclip_block again in this turn"); + expect(message).toContain("final response"); + expect(message).toContain("end the turn"); + expect(prpSemanticResultConflictMessage(null)).toBe(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE); + }); + + it("tells a provider to stop both report tools after a committed blocker", () => { + for (const message of [PRP_BLOCKER_ALREADY_REPORTED_MESSAGE, PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE]) { + expect(message).toContain("Do not call paperclip_finish or paperclip_block again in this turn"); + expect(message).toContain("end the turn"); + } + }); }); diff --git a/packages/paperclip-runner/src/contracts/completion-result.ts b/packages/paperclip-runner/src/contracts/completion-result.ts index 42750c88507..026becebbe6 100644 --- a/packages/paperclip-runner/src/contracts/completion-result.ts +++ b/packages/paperclip-runner/src/contracts/completion-result.ts @@ -6,10 +6,14 @@ export const PRP_COMPLETION_TOOL_DESCRIPTION = "Report completed work (done), work requiring review (needs_review), or an explicit wait (yielded with response_wake for a response, monitor after set_task_monitor confirms a schedule on this task, or omit continuation to use this run's authorized published-review wait or saved repository-preparation continuation). Use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome; do not claim completion while gated. Explain any required approval with its supplied link and action, then write the final response and end the turn without further tool calls."; export const PRP_BLOCK_TOOL_DESCRIPTION = "Report work that cannot continue because of a concrete blocker. Routine repository preparation is not a blocker: use paperclip_finish with yielded and omit continuation to resume through the saved workspace continuation. Identify the blocker, its owner, and the action needed to unblock it; use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome, explain the blocker and any required action in the final response, and end the turn without further tool calls."; -export const PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE = "A different semantic result was already committed"; +const PRP_STOP_REPORTING_INSTRUCTION = + "a turn's first accepted report is final, so this report was not recorded. Do not call paperclip_finish or paperclip_block again in this turn. "; +export const PRP_RESULT_ALREADY_ACCEPTED_MESSAGE = + "This turn's result was already accepted, and " + PRP_STOP_REPORTING_INSTRUCTION + + "Put any extra notes in your final response, then end the turn."; export const PRP_BLOCKER_ALREADY_REPORTED_MESSAGE = - "This turn already reported a blocker, and a turn's first accepted report is final, so this report was not recorded. Do not report again in this turn. " + - "In your final response, list the steps that are complete and say whether the blocker still applies, then end the turn."; + "This turn already reported a blocker, and " + PRP_STOP_REPORTING_INSTRUCTION + + "In your final response, list the steps that are complete, say whether the blocker still applies, and add any extra notes, then end the turn."; export const PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE = PRP_BLOCKER_ALREADY_REPORTED_MESSAGE + " Paperclip continues this environment or tool blocker automatically in a fresh session that checks the saved work."; @@ -20,7 +24,7 @@ export function prpSemanticResultConflictMessage(committed: { reportedWorkDisposition: string; blocker?: { owner?: { kind?: string } | null; reasonCode?: string } | null; } | null | undefined): string { - if (committed?.reportedWorkDisposition !== "blocked") return PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE; + if (committed?.reportedWorkDisposition !== "blocked") return PRP_RESULT_ALREADY_ACCEPTED_MESSAGE; return committed.blocker?.owner?.kind === "system" && PRP_ENVIRONMENT_BLOCKER_REASON_CODES.has(committed.blocker.reasonCode ?? "") ? PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE : PRP_BLOCKER_ALREADY_REPORTED_MESSAGE; } diff --git a/packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts b/packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts index f783077e503..84166b7c210 100644 --- a/packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts +++ b/packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts @@ -30,7 +30,7 @@ import { type NativeRuntimeContextSnapshot, } from "../../contracts/runtime-context.js"; import { createCodexTaskEnvelope } from "../../contracts/codex.js"; -import { PRP_BLOCKER_ALREADY_REPORTED_MESSAGE } from "../../contracts/completion-result.js"; +import { PRP_BLOCKER_ALREADY_REPORTED_MESSAGE, PRP_RESULT_ALREADY_ACCEPTED_MESSAGE } from "../../contracts/completion-result.js"; import { CONFIGURED_ENVIRONMENT_KEYS, configuredEnvironmentProjection } from "../../configured-environment.js"; import { localIntegrityBoundaryGolden } from "../../../test-support/local-integrity-boundary-golden.js"; import type { PrpEvent } from "../../protocol/replay-contract.js"; @@ -2889,6 +2889,33 @@ describe("OpenCodeServerDriver", () => { await session.close({ reason: "test" }); }); + it("tells a provider that a turn's accepted done result is final when it reports again", async () => { + await chmod(fixture, 0o755); + const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-finish-finish-")); + const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-")); + roots.push(root, workspace); + const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture, environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" } }); + const session = await driver.openSession({ runId: "finish-then-finish", normalizedSessionId: "finish-then-finish", workingDirectory: workspace }); + await session.startTurn({ message: { role: "user", text: "finish-after-finish" } }); + const events = await collectTurnEvents(session.events()); + const results = events.filter((event) => event.eventType === "run.result.proposed"); + expect(results).toHaveLength(1); + expect(results[0].payload).toMatchObject({ reportedWorkDisposition: "done", summary: "Fake provider completed the task." }); + expect(events.at(-1)?.eventType).toBe("turn.completed"); + const files = await readdir(root, { recursive: true }); + const evidence = files.find((name) => name.endsWith("fake-finish-after-finish.json")); + expect(evidence).toBeDefined(); + const calls = JSON.parse(await readFile(join(root, evidence!), "utf8")); + expect(calls.first.result.isError).not.toBe(true); + expect(calls.repeated.result.isError).not.toBe(true); + for (const rejected of [calls.rejected, calls.rejectedBlock]) { + expect(rejected).toMatchObject({ result: { isError: true, content: [{ text: expect.stringContaining(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE) }] } }); + expect(rejected.result.content[0].text).toContain("Do not call paperclip_finish or paperclip_block again in this turn"); + expect(rejected.result.content[0].text).toContain("end the turn"); + } + await session.close({ reason: "test" }); + }); + it.each(["monitor", "response_wake"] as const)("resolves an omitted %s through the real OpenCode MCP boundary only from current server state", async kind => { for (const eligible of [true, false]) { await chmod(fixture, 0o755); diff --git a/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs b/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs index 97ec466b340..753e2ad23c7 100644 --- a/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs +++ b/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs @@ -323,6 +323,17 @@ async function callTerminalTool(promptBody, signal) { const rejected = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: done }); await writeFile(join(process.env.XDG_DATA_HOME, "fake-finish-after-block.json"), JSON.stringify(rejected)); } + if (!blocked && String(prompt.message ?? "").includes("finish-after-finish")) { + const repeated = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: result }); + const changed = structuredClone(result); + changed.summary = "Fake provider completed the task and adds a late note."; + const rejected = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: changed }); + const blockedLate = structuredClone(result); + blockedLate.reportedWorkDisposition = "blocked"; + blockedLate.blocker = { reasonCode: "test_dependency", owner: { kind: "external", name: "fixture" }, unblockAction: "Release the fixture dependency.", scope: "current_track" }; + const rejectedBlock = await mcpRequest("tools/call", { name: "paperclip_block", arguments: blockedLate }); + await writeFile(join(process.env.XDG_DATA_HOME, "fake-finish-after-finish.json"), JSON.stringify({ first, repeated, rejected, rejectedBlock })); + } if (String(prompt.message ?? prompt.task?.prompt ?? "").includes("completion-feedback")) { const outcomes = [first]; if (first.result?.isError && String(prompt.message ?? prompt.task?.prompt ?? "").includes("current-wake-completion-feedback")) {