fix(native): ask once to re-check Paperclip before resting blocked on its own stall (#20779)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs report their result through a completion report. The
server checks the report before it accepts it.
> - When the Paperclip server pauses or restarts, the agent's Paperclip
tool calls stay unanswered. They return later, all together.
> - The agent can read that silence as an outage. It then reports a
blocker owned by a "Paperclip ... service" with reasonCode
`external_service_unavailable`.
> - The server accepts that report. The task rests blocked with no
Retry. Paperclip's own stall stops the task. Our rule says that must not
happen.
> - This pull request rejects that blocker report one time per run. The
correction tells the agent that Paperclip is reachable now and to
re-check.
> - The benefit is that the agent continues in the same turn and reports
the real outcome.

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos test, the server was paused (SIGSTOP) for about 2 to
5 minutes while a Codex agent saved a task document. `write_document`
and `read_document` did not return during the pause. After the pause,
the save had succeeded, but the agent reported a blocker: owner
"Paperclip document service", reasonCode `external_service_unavailable`.
The run succeeded with disposition `blocked`. The task rested blocked
with `canRetry=false` and no automatic recovery. A second test (two long
server holds, then a same-run resume) gave the same result.

**Expected behavior**
A Paperclip stall or restart must not leave a task stuck. The agent must
re-check Paperclip after it is reachable and finish the task.

**Steps to reproduce**
1. Start a native Codex task that saves a task document.
2. Pause the server process with SIGSTOP for about 2 minutes during the
save, then resume it.
3. The agent reports a blocker owned by a Paperclip service. The task
rests blocked with no Retry.

**Paperclip version or commit**
master at 62b7733161 (test base) and eb75c9bb85.

## What Changed

- `nativeCompletionFeedback` now rejects a `blocked` report one time per
run when all of these are true: the blocker owner kind is `system`, the
owner name contains "Paperclip", the reasonCode is
`external_service_unavailable`, the run is still running, and it owns
the task.
- The correction tells the agent: Paperclip answered this report, so it
is reachable now. Re-read the current state before writing again. Repeat
a needed write with the same `idempotencyKey`. Report blocked only if a
fresh Paperclip call fails now.
- The second identical report in the same run is accepted as before, so
a persistent real failure still rests blocked.
- Added a regression test. It also covers concatenated owner names such
as "PaperclipAI document service" and "paperclip_runner" (the match is a
case-insensitive substring).

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-completion-feedback.test.ts` passes
(32 tests).
- Red on master: the new test fails with `promise resolved "'Blocker
report accepted. ...'" instead of rejecting`. Green with this change.
- `cd server && npx tsc --noEmit -p tsconfig.json` passes.

## Risks

- Low risk. The change only adds one correction message inside the same
turn. It does not change a status, a database row, or a schedule.
- The once-per-run limit is kept in process memory. After a server
restart, a run can get the correction one more time. This is bounded and
harmless.
- The match uses the agent's own owner name. That only selects the
correction. It does not prove anything or replay a side effect. The
agent's next Paperclip calls give the real proof. Writes use idempotency
keys, so a repeated write does not apply twice.
- This does not add a Retry to a task that already rests blocked from an
older report. That is a separate product decision.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
user docs change; the behavior is explained in code comments)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending on the latest head)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending on the latest head)
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
DottaandClaude authored and GitHub committed 2026-11-11 04:22:09 -06:00
1 parent eb75c9bb85
commit 59ded526be
2 files changed
+59

No files matched your search

@@ -410,6 +410,36 @@ describe("native final-response feedback", () => {
await db.update(issues).set({ status: "in_progress", executionRunId: newerRunId }).where(eq(issues.id, value.issueId));
await expect(nativeCompletionFeedback(db, value.runId, report)).resolves.toContain("newer run owns");
});
it("asks once to re-check Paperclip instead of resting blocked on its own stall", async () => {
// Chaos run 2b600576: a SIGSTOPped server left write_document/read_document
// unanswered; the agent reported the Paperclip document service as an
// external outage and the task rested blocked with no Retry.
const value = await fixture();
const report: PrpStructuredRunResult = { ...done, reportedWorkDisposition: "blocked",
completionClaim: { ...done.completionClaim, objectiveSatisfied: false },
blocker: { reasonCode: "external_service_unavailable", owner: { kind: "system", name: "Paperclip document service" },
scope: "task_wide", unblockAction: "Restore document API responsiveness" } };
await expect(nativeCompletionFeedback(db, value.runId, report)).rejects.toThrow(/Paperclip answered this report[\s\S]*same idempotencyKey/);
// Once per run: a Paperclip failure that persists after the re-check rests blocked.
await expect(nativeCompletionFeedback(db, value.runId, report)).resolves.toContain("Blocker report accepted");
// Concatenated names such as "PaperclipAI document service" also name a Paperclip service.
const concatenated = await fixture();
await expect(nativeCompletionFeedback(db, concatenated.runId, { ...report, blocker: { ...report.blocker!, owner: { kind: "system", name: "PaperclipAI document service" } } }))
.rejects.toThrow("Paperclip answered this report");
const runnerNamed = await fixture();
await expect(nativeCompletionFeedback(db, runnerNamed.runId, { ...report, blocker: { ...report.blocker!, owner: { kind: "system", name: "paperclip_runner" } } }))
.rejects.toThrow("Paperclip answered this report");
// A genuinely external service, a non-system owner, or a finished run is not re-checked.
const other = await fixture();
await expect(nativeCompletionFeedback(db, other.runId, { ...report, blocker: { ...report.blocker!, owner: { kind: "system", name: "GitHub API" } } }))
.resolves.toContain("Blocker report accepted");
await expect(nativeCompletionFeedback(db, other.runId, { ...report, blocker: { ...report.blocker!, owner: { kind: "user", name: "Paperclip admin" } } }))
.resolves.toContain("Blocker report accepted");
await expect(nativeCompletionFeedback(db, other.runId, { ...report, blocker: { ...report.blocker!, reasonCode: "missing_access" } }))
.resolves.toContain("Blocker report accepted");
await db.update(heartbeatRuns).set({ status: "succeeded" }).where(eq(heartbeatRuns.id, other.runId));
await expect(nativeCompletionFeedback(db, other.runId, report)).resolves.toContain("Blocker report accepted");
});
it("asks a blocked provider to explain the cause and action instead of describing completed work", async () => {
const value = await fixture();
const feedback = await nativeCompletionFeedback(db, value.runId, { ...done, reportedWorkDisposition: "blocked",
@@ -32,6 +32,31 @@ import {
const repositoryYieldCorrection = 'This task has a saved repository preparation for its next admission. This is routine setup, not a blocker. Call paperclip_finish with reportedWorkDisposition="yielded" and omit continuation (or use null); Paperclip will select the saved workspace continuation. Preserve the completed work and explain what remains. Do not repeat setup, set a timer, or ask the user to restart.';
// A Paperclip stall or restart leaves the agent's Paperclip tool calls
// unanswered until the server is back; they then return together. The agent can
// read that silence as an outage and report it as a blocker. This report reached
// the server and is being answered, which is the server's own proof that its
// control plane is reachable now. Ask once per run to re-check with fresh
// Paperclip calls instead of resting the task blocked on a past stall.
const paperclipServiceRecheckCorrection = "Paperclip answered this report, so its control plane is reachable now. Paperclip tool calls that did not return earlier were most likely delayed by a Paperclip restart or pause, and their results may have arrived since. Do not report this as a blocker. Re-read the current Paperclip state (for example read_document) before writing again; when a write is still needed, repeat it with the same idempotencyKey so it cannot apply twice. Then continue the task and report its real outcome. Report blocked only if a fresh Paperclip call fails now.";
const PAPERCLIP_SERVICE_RECHECK_LIMIT = 1_000;
const paperclipServiceRechecked = new Set<string>();
function isPaperclipServiceOutageBlocker(result: PrpStructuredRunResult) {
const blocker = result.blocker;
return result.reportedWorkDisposition === "blocked" && blocker?.owner.kind === "system"
&& blocker.reasonCode === "external_service_unavailable" && (blocker.owner.name ?? "").toLowerCase().includes("paperclip");
}
/** Once per run (per server process), so a real persistent failure still rests blocked. */
function claimPaperclipServiceRecheck(runId: string) {
if (paperclipServiceRechecked.has(runId)) return false;
if (paperclipServiceRechecked.size >= PAPERCLIP_SERVICE_RECHECK_LIMIT)
paperclipServiceRechecked.delete(paperclipServiceRechecked.values().next().value!);
paperclipServiceRechecked.add(runId);
return true;
}
function record(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? value as Record<string, unknown> : {};
@@ -323,6 +348,10 @@ export async function nativeCompletionFeedback(
throw new Error(repositoryYieldCorrection);
}
}
if (isPaperclipServiceOutageBlocker(result) && issue.executionRunId === run.id && run.status === "running"
&& claimPaperclipServiceRecheck(run.id)) {
throw new Error(paperclipServiceRecheckCorrection);
}
if (result.reportedWorkDisposition === "yielded" && result.continuation?.kind === "monitor") {
return `Monitor wait accepted. The task remains active and Paperclip will wake its assignee at or after ${issue.monitorNextCheckAt!.toISOString()} with issue_monitor_due. End this turn; do not poll or mark the task done.`;
}