mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
fix(runner): tell a provider its accepted result is final when it reports again (#21984)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run reports its result with `paperclip_finish` or
`paperclip_block`, and a turn's first accepted report is final
> - When the model reports again with a different result, the Runner
refuses the second report
> - After a committed blocker, the refusal tells the model to stop and
end the turn (#21959)
> - After a committed done, needs_review or yielded result, the refusal
was only "A different semantic result was already committed", so the
model did not know what to do and tried again
> - Each extra call is one more model request, and the turn holds the
agent's slot until it ends; the task's status commits only when the turn
ends
> - This pull request gives every refused later report a clear, final
message: the result is already accepted, stop calling `paperclip_finish`
and `paperclip_block`, put extra notes in the final response, and end
the turn
> - The benefit is that the model ends the turn after one refusal, so
the slot is free sooner and the task completes sooner
## Linked Issues or Issue Description
**What happened?**
In a restart-chaos wave, an OpenCode agent (DeepSeek through OpenRouter)
finished a task, and the Runner accepted its `done` report. The model
then called `paperclip_finish` again with a different report. The Runner
replied only "A different semantic result was already committed". The
model repeated the call about 79 times over 277 seconds. The agent's
slot stayed busy, and the task's completion was late, because the task
status commits when the turn ends.
**Expected behavior**
After a turn commits any result (done, needs_review, yielded or
blocked), each later report with a different result gets a clear and
final instruction: the result is already accepted, so do not call
`paperclip_finish` or `paperclip_block` again, put any extra notes in
the final response, and end the turn. An identical repeat of the
accepted report stays an idempotent success.
**Steps to reproduce**
1. Start a native OpenCode turn that calls `paperclip_finish` with a
done result.
2. In the same turn, call `paperclip_finish` again with a different
summary, or call `paperclip_block`.
3. On master, the tool error is only "A different semantic result was
already committed", with no instruction to stop.
**Paperclip version or commit**
master 54f8068df4
## What Changed
- `packages/paperclip-runner/src/contracts/completion-result.ts`:
replace the bare `PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE` with
`PRP_RESULT_ALREADY_ACCEPTED_MESSAGE`.
`prpSemanticResultConflictMessage` returns it for every committed result
that is not a blocker. The blocker messages share the same stop
instruction, which now names both `paperclip_finish` and
`paperclip_block`, and ask for extra notes in the final response.
- The OpenCode, Codex and Codex ACPX drivers already return
`prpSemanticResultConflictMessage(...)`, so all three get the new text
with no driver change.
- `packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs`:
add a `finish-after-finish` scenario. After the accepted
`paperclip_finish`, the fake provider sends an identical repeat, a
changed `paperclip_finish` and a late `paperclip_block`.
-
`packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts`:
a new test checks that the identical repeat succeeds, that both
conflicting reports get the final message, that only one result is
proposed, and that the turn completes.
- `packages/paperclip-runner/src/contracts/completion-result.test.ts`:
unit cases for done, needs_review, yielded and a missing committed
result, and a check that both blocker messages name both tools and say
to end the turn.
- Not in this change: a hard bound that ends the turn after N refused
reports. In the OpenCode driver the only way to stop a provider turn is
`/abort`, which the driver reports as `turn.cancelled`, not
`turn.completed`. A safe bound needs a new terminal path in each driver,
so it is not "small". This change fixes the cause (the message) first.
## Verification
- Red: with the old message, the 4 new `completion-result.test.ts` cases
and the new OpenCode driver test fail (the refusal text does not contain
the stop instruction).
- Green: `npx vitest run src/contracts/completion-result.test.ts
src/drivers/opencode/opencode-server-driver.test.ts` in
`packages/paperclip-runner`: 2 files, 123 tests pass.
- `npx vitest run src/drivers/acpx/codex-acpx-driver.test.ts
src/drivers/codex`: 28 files, 563 tests pass.
- `pnpm --filter @paperclipai/paperclip-runner typecheck` passes.
- `pnpm verify` selected broad coverage (exit 2) because the shared
OpenCode fixture changed. Required CI supplies that coverage.
## Risks
- Low risk. Only the text of a tool error changes. Result admission,
fingerprints and idempotent repeats do not change.
- Mixed versions during a deploy: the Runner creates this text and sends
it straight to the model. The server does not read or match it (no
server, Rust or test code matches the old string). An old server with a
new Runner gets the new text. A new server with an old Runner keeps the
old text. Neither case changes a stored record or a protocol shape.
- ACPX: no ACPX profile, pin, schema or `protocol/` file changes, so no
provider `commandDigest` or pack contract changes.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, test runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
54f8068df4
commit
41b0f76d75
4 files changed
+65
-7
No files matched your search
@@ -4,7 +4,7 @@ import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
PRP_BLOCKER_ALREADY_REPORTED_MESSAGE,
|
||||
PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE,
|
||||
PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE,
|
||||
PRP_RESULT_ALREADY_ACCEPTED_MESSAGE,
|
||||
prpSemanticResultConflictMessage,
|
||||
PRP_BLOCK_RESULT_OUTPUT_SCHEMA,
|
||||
PRP_BLOCK_RESULT_PROVIDER_INPUT_SCHEMA,
|
||||
@@ -303,7 +303,23 @@ describe("semantic result conflict feedback", () => {
|
||||
expect(prpSemanticResultConflictMessage(blocked("system", "tool_unavailable"))).toBe(PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE);
|
||||
expect(prpSemanticResultConflictMessage(blocked("user", "credential_missing"))).toBe(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE);
|
||||
expect(prpSemanticResultConflictMessage(blocked("system", "credential_missing"))).toBe(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE);
|
||||
expect(prpSemanticResultConflictMessage({ reportedWorkDisposition: "done" })).toBe(PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE);
|
||||
expect(PRP_BLOCKER_ALREADY_REPORTED_MESSAGE).not.toContain("automatically");
|
||||
});
|
||||
|
||||
it.each(["done", "needs_review", "yielded"])("tells a provider that a %s result is final and to end the turn", disposition => {
|
||||
const message = prpSemanticResultConflictMessage({ reportedWorkDisposition: disposition });
|
||||
expect(message).toBe(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE);
|
||||
expect(message).toContain("already accepted");
|
||||
expect(message).toContain("Do not call paperclip_finish or paperclip_block again in this turn");
|
||||
expect(message).toContain("final response");
|
||||
expect(message).toContain("end the turn");
|
||||
expect(prpSemanticResultConflictMessage(null)).toBe(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE);
|
||||
});
|
||||
|
||||
it("tells a provider to stop both report tools after a committed blocker", () => {
|
||||
for (const message of [PRP_BLOCKER_ALREADY_REPORTED_MESSAGE, PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE]) {
|
||||
expect(message).toContain("Do not call paperclip_finish or paperclip_block again in this turn");
|
||||
expect(message).toContain("end the turn");
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -6,10 +6,14 @@ export const PRP_COMPLETION_TOOL_DESCRIPTION =
|
||||
"Report completed work (done), work requiring review (needs_review), or an explicit wait (yielded with response_wake for a response, monitor after set_task_monitor confirms a schedule on this task, or omit continuation to use this run's authorized published-review wait or saved repository-preparation continuation). Use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome; do not claim completion while gated. Explain any required approval with its supplied link and action, then write the final response and end the turn without further tool calls.";
|
||||
export const PRP_BLOCK_TOOL_DESCRIPTION =
|
||||
"Report work that cannot continue because of a concrete blocker. Routine repository preparation is not a blocker: use paperclip_finish with yielded and omit continuation to resume through the saved workspace continuation. Identify the blocker, its owner, and the action needed to unblock it; use the current completion contract and supporting evidence. If rejected, correct the report and retry. After acceptance, read the returned outcome, explain the blocker and any required action in the final response, and end the turn without further tool calls.";
|
||||
export const PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE = "A different semantic result was already committed";
|
||||
const PRP_STOP_REPORTING_INSTRUCTION =
|
||||
"a turn's first accepted report is final, so this report was not recorded. Do not call paperclip_finish or paperclip_block again in this turn. ";
|
||||
export const PRP_RESULT_ALREADY_ACCEPTED_MESSAGE =
|
||||
"This turn's result was already accepted, and " + PRP_STOP_REPORTING_INSTRUCTION +
|
||||
"Put any extra notes in your final response, then end the turn.";
|
||||
export const PRP_BLOCKER_ALREADY_REPORTED_MESSAGE =
|
||||
"This turn already reported a blocker, and a turn's first accepted report is final, so this report was not recorded. Do not report again in this turn. " +
|
||||
"In your final response, list the steps that are complete and say whether the blocker still applies, then end the turn.";
|
||||
"This turn already reported a blocker, and " + PRP_STOP_REPORTING_INSTRUCTION +
|
||||
"In your final response, list the steps that are complete, say whether the blocker still applies, and add any extra notes, then end the turn.";
|
||||
export const PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE = PRP_BLOCKER_ALREADY_REPORTED_MESSAGE +
|
||||
" Paperclip continues this environment or tool blocker automatically in a fresh session that checks the saved work.";
|
||||
|
||||
@@ -20,7 +24,7 @@ export function prpSemanticResultConflictMessage(committed: {
|
||||
reportedWorkDisposition: string;
|
||||
blocker?: { owner?: { kind?: string } | null; reasonCode?: string } | null;
|
||||
} | null | undefined): string {
|
||||
if (committed?.reportedWorkDisposition !== "blocked") return PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE;
|
||||
if (committed?.reportedWorkDisposition !== "blocked") return PRP_RESULT_ALREADY_ACCEPTED_MESSAGE;
|
||||
return committed.blocker?.owner?.kind === "system" && PRP_ENVIRONMENT_BLOCKER_REASON_CODES.has(committed.blocker.reasonCode ?? "")
|
||||
? PRP_ENVIRONMENT_BLOCKER_ALREADY_REPORTED_MESSAGE : PRP_BLOCKER_ALREADY_REPORTED_MESSAGE;
|
||||
}
|
||||
|
||||
@@ -30,7 +30,7 @@ import {
|
||||
type NativeRuntimeContextSnapshot,
|
||||
} from "../../contracts/runtime-context.js";
|
||||
import { createCodexTaskEnvelope } from "../../contracts/codex.js";
|
||||
import { PRP_BLOCKER_ALREADY_REPORTED_MESSAGE } from "../../contracts/completion-result.js";
|
||||
import { PRP_BLOCKER_ALREADY_REPORTED_MESSAGE, PRP_RESULT_ALREADY_ACCEPTED_MESSAGE } from "../../contracts/completion-result.js";
|
||||
import { CONFIGURED_ENVIRONMENT_KEYS, configuredEnvironmentProjection } from "../../configured-environment.js";
|
||||
import { localIntegrityBoundaryGolden } from "../../../test-support/local-integrity-boundary-golden.js";
|
||||
import type { PrpEvent } from "../../protocol/replay-contract.js";
|
||||
@@ -2889,6 +2889,33 @@ describe("OpenCodeServerDriver", () => {
|
||||
await session.close({ reason: "test" });
|
||||
});
|
||||
|
||||
it("tells a provider that a turn's accepted done result is final when it reports again", async () => {
|
||||
await chmod(fixture, 0o755);
|
||||
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-finish-finish-"));
|
||||
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
|
||||
roots.push(root, workspace);
|
||||
const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture, environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" } });
|
||||
const session = await driver.openSession({ runId: "finish-then-finish", normalizedSessionId: "finish-then-finish", workingDirectory: workspace });
|
||||
await session.startTurn({ message: { role: "user", text: "finish-after-finish" } });
|
||||
const events = await collectTurnEvents(session.events());
|
||||
const results = events.filter((event) => event.eventType === "run.result.proposed");
|
||||
expect(results).toHaveLength(1);
|
||||
expect(results[0].payload).toMatchObject({ reportedWorkDisposition: "done", summary: "Fake provider completed the task." });
|
||||
expect(events.at(-1)?.eventType).toBe("turn.completed");
|
||||
const files = await readdir(root, { recursive: true });
|
||||
const evidence = files.find((name) => name.endsWith("fake-finish-after-finish.json"));
|
||||
expect(evidence).toBeDefined();
|
||||
const calls = JSON.parse(await readFile(join(root, evidence!), "utf8"));
|
||||
expect(calls.first.result.isError).not.toBe(true);
|
||||
expect(calls.repeated.result.isError).not.toBe(true);
|
||||
for (const rejected of [calls.rejected, calls.rejectedBlock]) {
|
||||
expect(rejected).toMatchObject({ result: { isError: true, content: [{ text: expect.stringContaining(PRP_RESULT_ALREADY_ACCEPTED_MESSAGE) }] } });
|
||||
expect(rejected.result.content[0].text).toContain("Do not call paperclip_finish or paperclip_block again in this turn");
|
||||
expect(rejected.result.content[0].text).toContain("end the turn");
|
||||
}
|
||||
await session.close({ reason: "test" });
|
||||
});
|
||||
|
||||
it.each(["monitor", "response_wake"] as const)("resolves an omitted %s through the real OpenCode MCP boundary only from current server state", async kind => {
|
||||
for (const eligible of [true, false]) {
|
||||
await chmod(fixture, 0o755);
|
||||
|
||||
@@ -323,6 +323,17 @@ async function callTerminalTool(promptBody, signal) {
|
||||
const rejected = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: done });
|
||||
await writeFile(join(process.env.XDG_DATA_HOME, "fake-finish-after-block.json"), JSON.stringify(rejected));
|
||||
}
|
||||
if (!blocked && String(prompt.message ?? "").includes("finish-after-finish")) {
|
||||
const repeated = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: result });
|
||||
const changed = structuredClone(result);
|
||||
changed.summary = "Fake provider completed the task and adds a late note.";
|
||||
const rejected = await mcpRequest("tools/call", { name: "paperclip_finish", arguments: changed });
|
||||
const blockedLate = structuredClone(result);
|
||||
blockedLate.reportedWorkDisposition = "blocked";
|
||||
blockedLate.blocker = { reasonCode: "test_dependency", owner: { kind: "external", name: "fixture" }, unblockAction: "Release the fixture dependency.", scope: "current_track" };
|
||||
const rejectedBlock = await mcpRequest("tools/call", { name: "paperclip_block", arguments: blockedLate });
|
||||
await writeFile(join(process.env.XDG_DATA_HOME, "fake-finish-after-finish.json"), JSON.stringify({ first, repeated, rejected, rejectedBlock }));
|
||||
}
|
||||
if (String(prompt.message ?? prompt.task?.prompt ?? "").includes("completion-feedback")) {
|
||||
const outcomes = [first];
|
||||
if (first.result?.isError && String(prompt.message ?? prompt.task?.prompt ?? "").includes("current-wake-completion-feedback")) {
|
||||
|
||||
Reference in new issue
Block a user