fix(recovery): do not auto-replace a chat Retry under its consumed exact-request authority (#22414)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs that fail before they do provider work can continue
automatically. The native safe-replacement sweep schedules a system
successor run for them.
> - The successor copies the failed run's context. The sweep removes
consumed wake fields, but it keeps the `chatFailedRunRetry` selector and
`taskRetryRequestId`.
> - That selector is the authority of an exact chat Retry. It is bound
to the failed run's own chat action and to the user's Retry intent. The
successor gets a new system wakeup request, so admission can never
authorize it.
> - Thus every automatic successor of a chat Retry fails at setup with
`FailedChatRunRetryAuthorizationError`. The sweep has already retired
the task's recovery hold at that time, so the task can be left with no
owner of the next step.
> - This pull request stops the sweep from replacing a run that carries
`chatFailedRunRetry`.
> - The benefit is that the task keeps its hold. The user's next Retry
or message continues it through the normal authorized path, and no run
fails for a known reason.

## Linked Issues or Issue Description

**What happened?**
On staging, a user pressed Retry on a failed GitHub chat run. The Retry
run failed before its first claim, after a workspace restore failure.
Its own exact-request authority refused it
(`chat_failed_run_retry_not_authorized`). The pre-claim continuation
path then scheduled a `native_safe_replacement` successor. The successor
kept the copied `chatFailedRunRetry` selector under a new wakeup request
ID, and `authorizeFailedChatRunRetryWake` refused it at setup.

**Expected behavior**
An automatic system successor must not be scheduled under chat Retry
authority that it cannot hold. The task must keep its recovery hold for
the user's next Retry or message.

**Steps to reproduce**
1. Make a native run whose context has `chatFailedRunRetry` (an exact
chat Retry).
2. Fail it before its first coordinator claim, or with a provider
terminal failure, so it qualifies for safe replacement.
3. Run `reconcileSafeNativeReplacements`.
4. See that a successor run is scheduled with the copied selector. At
admission it always fails with `FailedChatRunRetryAuthorizationError`.

Related: #22413 changes the same file for a different defect (the
reviewed chat message batch on fresh-session retries). It does not touch
the eligibility predicate or `chatFailedRunRetry`. The two changes are
independent.

## What Changed

- `nativeReplacementPending()` now excludes a failed run whose context
carries `chatFailedRunRetry`. Because the shared eligibility predicate
has the guard, the pre-claim, historical listener, and provider
terminal-failure replacement paths all get it.
- New regression in `native-preclaim-continuation.test.ts`: this is the
staging shape (a pre-claim failure of an exact chat Retry after its
saved work is recovered). It schedules no successor, the hold stays
effective, and the task stays blocked.
- New paired regression in `native-safe-replacement.test.ts` for the
provider terminal-failure path. An ordinary run still gets exactly one
successor. An exact chat Retry gets none.

## Verification

- Red before the fix: the pre-claim chat Retry case failed with
`expected length 0, got 1`. The terminal-failure chat Retry case also
failed. Its ordinary-run control passed.
- Green after the fix, rebased on `fe0c4aa95d`:
- `npx vitest run src/services/native-preclaim-continuation.test.ts
src/services/native-runtime/native-safe-replacement.test.ts`: 89 passed.
- `npx vitest run
src/services/native-runtime/native-safe-replacement.test.ts
src/services/quarantined-task-retry.test.ts
src/__tests__/execution-control-work.test.ts
src/__tests__/recovery-settled-records-census.test.ts
src/__tests__/durable-chat-wakeup.test.ts`: 206 passed.
  - `pnpm --filter @paperclipai/server typecheck`: passed.
- Second-order check: `nativeInterruptionHold()` uses
`not(nativeReplacementPending())`. Thus an excluded run can now reach
the existing fresh-continuation fallback, the same as other pre-claim
runs that do not qualify. That fallback records a continuation decision
and lets normal admission build a fresh context. It does not copy the
predecessor's selector.
- `pnpm verify` selects 2,950 of 3,817 files for this widely imported
module. I ran the focused suites above and rely on required CI for the
broad coverage.

## Risks

- Low risk. The change only removes candidates from automatic
replacement. Before this change, a removed candidate always produced a
successor that failed at setup.
- Behavior shift: after a chat Retry run fails, the user (or a new chat
message) continues the task. A system successor that cannot be
authorized does not start. Ordinary chat-message runs and non-chat runs
are not changed.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in an agent harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
authored and GitHub committed 2026-11-20 07:53:08 -08:00
1 parent 875738a8d4
commit 0e38c80323
3 files changed
+52

No files matched your search

@@ -136,6 +136,36 @@ describe("native run that failed before its first coordinator claim", () => {
expect(await successorsOf(f.runId)).toHaveLength(0);
}, 60_000);
// Staging 2026-11-20: a user's exact chat Retry failed before its first
// claim. Its authority is bound to its own chat action and Retry intent, so
// a system successor carrying the copied selector is always refused at
// setup. The saved hold keeps the task for the user's next Retry or message.
it("does not continue a chat Retry automatically under its consumed exact-request authority", async () => {
const f = await preClaimNativeFailure();
const current = await run(f.runId);
// The failed run the user retried. It precedes the Retry, so it does not
// supersede it.
const [failedSource] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId,
status: "failed", contextSnapshot: { issueId: f.issueId }, createdAt: new Date(current.createdAt.getTime() - 60_000) })
.returning();
const retryOfRunId = failedSource!.id;
await db.update(heartbeatRuns).set({
errorCode: "chat_failed_run_retry_not_authorized", retryOfRunId,
contextSnapshot: { ...current.contextSnapshot, retryOfRunId, forceFreshSession: true, taskRetryRequestId: randomUUID(),
chatFailedRunRetry: { version: 1, actionId: current.wakeupRequestId ?? randomUUID(), failedRunId: retryOfRunId } },
}).where(eq(heartbeatRuns.id, f.runId));
await recoverRetainedWorkspace(f.input);
expect(readWorkspaceRecovery((await action(f.actionId)).evidence)).toMatchObject({ status: "recovered" });
await sweep();
expect(await successorsOf(f.runId)).toHaveLength(0);
const hold = await action(f.actionId);
expect(hold).toMatchObject({ outcome: "blocked", evidence: { automaticRecovery: { replay: "blocked" } } });
const [task] = await db.select().from(issues).where(eq(issues.id, f.issueId));
expect(task.status).toBe("blocked");
}, 60_000);
it.each(["claimed", "provider_event", "tool_call", "uncertain_side_effect", "process", "superseded"] as const)(
"keeps reconciliation when the run may have acted: %s", async gate => {
const f = await preClaimNativeFailure();
@@ -527,6 +527,24 @@ const support = externalDatabaseUrl
const [action] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, source.issueId));
expect(action.status).toBe("active");
});
it.each([["an ordinary run", false, 1], ["an exact chat Retry", true, 0]] as const)(
"replaces a provider terminal failure only without consumed chat Retry authority (%s)", async (_label, chatRetry, expected) => {
const source = await seed();
if (chatRetry) {
const [failed] = await db.insert(heartbeatRuns).values({ companyId: source.companyId, agentId: source.agentId,
status: "failed", contextSnapshot: { issueId: source.issueId }, createdAt: new Date(Date.now() - 60_000) }).returning();
const [current] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, source.runId));
await db.update(heartbeatRuns).set({ retryOfRunId: failed!.id, contextSnapshot: { ...current!.contextSnapshot,
retryOfRunId: failed!.id, taskRetryRequestId: randomUUID(),
chatFailedRunRetry: { version: 1, actionId: randomUUID(), failedRunId: failed!.id } } })
.where(eq(heartbeatRuns.id, source.runId));
}
invalidateNativeReplacementDiscovery(db);
await reconcileSafeNativeReplacements(db);
const successors = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.retryOfRunId, source.runId));
expect(successors).toHaveLength(expected);
for (const successor of successors) expect(successor.contextSnapshot).not.toHaveProperty("chatFailedRunRetry");
});
async function seedUnprovenOwner(finishedAgoMs: number) {
const source = await seed();
await db.update(heartbeatRuns).set({ finishedAt: sql`now() - ${finishedAgoMs}::integer * interval '1 millisecond'`,
@@ -240,6 +240,10 @@ export async function remoteWorkspaceReplacementEvidence(
export function nativeReplacementPending() {
return and(
eq(heartbeatRuns.status, "failed"),
// An exact chat Retry is authorized only through its own chat action and
// the user's Retry intent. A system successor cannot hold that authority,
// so admission would always refuse it. Keep the task's hold for the user.
sql`not (coalesce(${heartbeatRuns.contextSnapshot}, '{}'::jsonb) ? 'chatFailedRunRetry')`,
eq(nativeRunFinalizations.phase, "terminal_failure"),
or(inArray(
nativeRunFinalizations.failureCode,