fix(native): offer Retry at once when the Runner refuses a turn start on a closed session (#20729)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native ACPX run (for example Claude) starts each turn with
`turn.start`. When a start is cut off, the Runner reads the sidecar's
dispatch journal (#20675).
> - If the prompt never left, the Runner keeps the session reopenable,
and the same-run resume sends the prompt once. If the prompt may have
left, the Runner closes the session, because a resend could duplicate
it.
> - A closed session cannot reopen for the same run. Every same-run
resume got `ACPX session is not open`. The server treated that as a
generic interruption and spent three attempts (about 95 s) before the
board offered Retry.
> - This pull request gives that refusal a closed class in the Runner
and a permanent, named failure code in the server.
> - The benefit is that the board offers a working Retry on the first
refusal, while every recoverable case keeps recovering by itself.

## Linked Issues or Issue Description

**What happened?**
Restart-chaos wave `w-20261111T011704-s9107`, task t05 (local, claude,
workload C1, fault TM2). A freeze cut off a turn start, and the Runner
saved the session `closed`. Three same-run resumes each failed with `PRP
command turn.start failed: … "message":"ACPX session is not open"`.
Recovery stopped with `native_session_retry_exhausted` at 01:25:14, and
the board then offered Retry. The Retry run succeeded.

**Expected behavior**
When the Runner proves the session is closed for this run, the run stops
at the first refusal with a named failure, and the board offers a Retry
that works. A start that the Runner proved never reached the provider
still recovers by itself.

**Steps to reproduce**
1. `cd server && npx vitest run
src/services/native-runtime/native-closed-session-retry.integration.test.ts`
2. On master, "offers Retry on the first refusal, and Retry starts a new
run" fails: the coordinator is `retryable_failure` /
`native_session_interrupted` (a same-run retry in 30 s).

Refs #20675, #20720

## What Changed

- Runner (`durable/mod.rs`, `acpx_provider_backend.rs`): new closed
class `provider_session_closed`. The Runner sets it on these refusals:
- a turn start that saves the session `closed` (`ACPX turn start failed
closed: …`), but not the rejected-checkpoint case, which stays
`prepared` and reopens;
- `ACPX session is not open` when the saved lifecycle is `closed` for
this run;
- the saved turn-end-restart refusal for this run (asserted in
`acpx_turn_end_restart`).
A session that waits for provider-exit proof can reopen after a restart.
Its refusal gets no class.
- Server (`native-session-executor.ts`): a `turn.start` failure with
that class maps to the new permanent code
`native_provider_session_closed`. The server reads the class from the
authenticated command result, and from the command-failure text if the
error was rewrapped. `nativeSessionFailureDisposition` treats the code
as permanent, so the first refusal ends the run as `terminal_failure`
with a board owner and a plain next action.
- Docs: `packages/paperclip-runner/docs/durable-recovery.md` states the
class and the older-Runner behavior.
- Tests:
- `native-closed-session-retry.integration.test.ts` (new, embedded
Postgres) drives `executePaperclipNativeSession` with the Runner
refusal. On the first refusal, the board shows `canRetry: true`.
`requestTaskRetry` is accepted, and `resumeTaskRetry` starts exactly one
new run with a fresh provider session. A start proved undispatched stays
`retryable_failure`.
- Unit tests in `native-session-executor.test.ts`: both closed refusals,
a rewrapped error, an undispatched start, an unclassified older-Runner
refusal, and a class on another command.
- Rust `acpx_cut_turn_start`: a start cut off before dispatch has no
class. A start cut off after dispatch, and the later refusal, carry
`provider_session_closed`.

## Verification

- Red on master, green here: step 1 above.
- `npx vitest run
src/services/native-runtime/native-session-executor.test.ts`: 1358/1358
pass.
- `cargo test -p paperclip-runner-core --test acpx_park_preparation` (26
pass), `--lib acpx` (94 pass), `--test acpx_provider_turns --test
acpx_provider_session --test acpx_warm_attachment` (all pass), `cargo
fmt --check` clean.
- `npx tsc --noEmit -p server/tsconfig.json`: clean.
- Not run: a chaos wave.

## Risks

- This changes the Rust Runner. Daytona needs a new runner image before
remote runs get the class. Until then, an older Runner sends the refusal
without the class, and the server keeps today's bounded same-run
retries. That path is unchanged and ends with a working Retry. The
server does not trust the text alone, because an older Runner sends the
same text for a session that can still reopen.
- The class is set only where the Runner's saved state proves that no
later start of this run can reopen the session. A false class would cost
one automatic resume and show Retry earlier. It never replays provider
work.
- No persisted shape changes. The class is an optional field of a failed
command result. Old saved state resumes unchanged. Nothing new runs
before `turn.start`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic), model ID `claude-opus-5-5`, run in Claude Code
with tool use (shell, file edit) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>

server/src/services/native-runtime/native-closed-session-retry.integration.test.ts

@@ -0,0 +1,106 @@
+import { randomUUID } from "node:crypto";
+import { eq } from "drizzle-orm";
+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
+import { agents, companies, companyMemberships, completionContracts, createDb, heartbeatRuns, issues, nativeRunFinalizations } from "@paperclipai/db";
+import type { NativeExecutionInputV1 } from "@paperclipai/paperclip-runner";
+import { startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
+
+const provider = vi.hoisted(() => ({ execute: vi.fn() }));
+vi.mock("../../vendor/paperclip-runner/index.js", async (importOriginal) => ({
+  ...(await importOriginal<typeof import("../../vendor/paperclip-runner/index.js")>()),
+  executeNativeSession: provider.execute,
+}));
+
+import { executePaperclipNativeSession, NATIVE_PROVIDER_SESSION_CLOSED } from "./native-session-executor.js";
+import { buildNativeCompletionContract } from "./completion-contracts.js";
+import { prepareNativeHeartbeatRun } from "./prepare-native-run.js";
+import { heartbeatService } from "../heartbeat.js";
+import { requestTaskRetry } from "../task-retry.js";
+import { taskInterruptionPresentation } from "../task-interruption-presentation.js";
+
+// Chaos w-20261111T011704-s9107/t05 (claude, TM2): a cut-off ACPX turn start
+// left this run's session closed with the prompt possibly sent. Every same-run
+// resume got "ACPX session is not open"; the board offered Retry only after
+// three attempts (about 95 s). The Runner now classifies that refusal, and
+// Retry starts a new run with a fresh provider session.
+describe("native turn start refused on a closed provider session", () => {
+  let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
+  let db: ReturnType<typeof createDb>;
+  const companyId = randomUUID();
+  const agentId = randomUUID();
+
+  beforeAll(async () => {
+    temporary = await startEmbeddedPostgresTestDatabase("native-closed-session-");
+    db = createDb(temporary.connectionString);
+    await db.insert(companies).values({ id: companyId, name: "Closed session", issuePrefix: "NCS" });
+    await db.insert(agents).values({ id: agentId, companyId, name: "Claude agent", status: "active", adapterType: "paperclip_runner",
+      adapterConfig: { provider: "acpx", agent: "claude", model: "claude-test" } });
+    await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: "board-user", status: "active", membershipRole: "member" });
+  });
+  afterAll(async () => { await temporary?.cleanup(); });
+
+  const refusal = (result: Record<string, unknown>) => {
+    const commandResult = { commandId: "command_lab_1", commandType: "turn.start", controllerSeq: 10, result, status: "failed" };
+    return Object.defineProperty(new Error(`PRP command turn.start failed: ${JSON.stringify(commandResult)}`),
+      "runnerCommandResult", { value: commandResult, enumerable: false });
+  };
+
+  async function refuseTurnStart(failure: Error) {
+    const issueId = randomUUID();
+    await db.insert(issues).values({ id: issueId, companyId, title: "Write the brief", status: "in_progress", workMode: "standard",
+      assigneeAgentId: agentId, createdByUserId: "board-user", responsibleUserId: "board-user" });
+    const [issue] = await db.select().from(issues).where(eq(issues.id, issueId));
+    const runId = randomUUID();
+    const [run] = await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "running", runtimeMode: "native",
+      nativeIssueId: issueId, responsibleUserId: "board-user", invocationSource: "automation", triggerDetail: "system",
+      contextSnapshot: { issueId } }).returning();
+    await db.update(issues).set({ executionRunId: runId }).where(eq(issues.id, issueId));
+    const native = await prepareNativeHeartbeatRun({ db, run, issue, environmentLeaseId: randomUUID() });
+    const [contract] = await db.select().from(completionContracts).where(eq(completionContracts.issueId, issueId));
+    await db.insert(nativeRunFinalizations).values({ runId, companyId, issueId, phase: "observed" });
+    const execution: NativeExecutionInputV1 = {
+      schema: "paperclip.native-execution-input.v1",
+      binding: { companyId, runId, issueId, agentId, executionWorkspaceId: runId },
+      provider: { kind: "opencode", model: "openrouter/test-model" },
+      task: { identifier: issueId, title: issue.title, description: null, prompt: "Write the brief.", workMode: "standard" },
+      workspace: { cwd: "/tmp", repoUrl: null, repoRef: null, branchName: null },
+      session: { normalizedSessionId: native.normalizedSessionId, driverKind: "opencode_server", protocolVersion: 1, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } },
+      completionContract: { id: contract.id, sha256: contract.canonicalSha256, schemaVersion: "paperclip.completion-contract.v1",
+        contract: buildNativeCompletionContract(issue, { revision: contract.revision }) },
+      interactionResponses: [], credentialBindings: [],
+    };
+    // An established provider session: same-run resume is the ordinary recovery.
+    await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeExecutionInput: execution,
+      sessionCheckpoint: { backendKind: "mock", providerSessionId: "claude-session" } } }).where(eq(heartbeatRuns.id, runId));
+    provider.execute.mockReset().mockRejectedValue(failure);
+    await executePaperclipNativeSession({ db, execution, runnerInstanceId: native.runnerInstanceId, backend: {
+      descriptor: async () => { throw new Error("Unexpected provider call"); },
+      openSession: async () => { throw new Error("Unexpected provider call"); },
+    } }).catch(() => undefined);
+    const [coordinator] = await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, runId));
+    return { issueId, runId, coordinator };
+  }
+
+  it("offers Retry on the first refusal, and Retry starts a new run", async () => {
+    const { issueId, runId, coordinator } = await refuseTurnStart(refusal({ code: "command_execution_failed",
+      message: "ACPX session is not open", classification: "provider_session_closed" }));
+    expect(coordinator).toMatchObject({ phase: "terminal_failure", failureCode: NATIVE_PROVIDER_SESSION_CLOSED, nextAttemptAt: null });
+    const [issue] = await db.select().from(issues).where(eq(issues.id, issueId));
+    await expect(taskInterruptionPresentation(db, issue)).resolves.toMatchObject({ state: "failed", canRetry: true });
+    const receipt = await requestTaskRetry({ db, companyId, issueId, actorId: "board-user", actorSource: "session",
+      idempotencyKey: randomUUID(), observedRunId: runId });
+    expect(receipt).toMatchObject({ accepted: true, stale: false });
+    const heartbeat = heartbeatService(db);
+    await heartbeat.resumeTaskRetry(companyId, receipt.requestId!);
+    const retried = (await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, agentId)))
+      .filter(r => r.contextSnapshot?.issueId === issueId && r.contextSnapshot?.taskRetryRequestId === receipt.requestId);
+    expect(retried).toHaveLength(1);
+    await heartbeat.drainActiveRunExecutions();
+  }, 120_000);
+
+  it("keeps a same-run retry when the Runner proved the prompt never left", async () => {
+    const { coordinator } = await refuseTurnStart(refusal({ code: "command_execution_failed",
+      message: "ACPX turn start failed before its prompt reached the provider: ACPX sidecar request timed out at turn.start" }));
+    expect(coordinator).toMatchObject({ phase: "retryable_failure", failureCode: "native_session_interrupted" });
+  }, 120_000);
+});

server/src/services/native-runtime/native-session-executor.test.ts

@@ -429,6 +429,7 @@ import {
   retainedNativeCleanupJournalMatches,
   nativeProviderUsageLimitFromEvent,
   nativeSessionFailureSourceCode,
+  NATIVE_PROVIDER_SESSION_CLOSED,
   nativeSessionRecoveryProjection,
   nativeGovernedWaitResult,
   nativeConversationReplyResult,
@@ -13976,6 +13977,52 @@ describe("native session bounded recovery", () => {
     expect(nativeSessionFailureSourceCode(new Error(failure.message))).toBe("native_session_interrupted");
   });
 
+  // Chaos w-20261111T011704-s9107/t05: after a cut-off ACPX turn start the
+  // Runner saved the session closed; three same-run resumes each got the same
+  // refusal before the board offered Retry.
+  describe("a turn start the Runner refused because this run's session is closed", () => {
+    const turnStartFailure = (result: Record<string, unknown>) => {
+      const commandResult = { commandId: "command_lab_1", commandType: "turn.start", controllerSeq: 10, result, status: "failed" };
+      return Object.defineProperty(new Error(`PRP command turn.start failed: ${JSON.stringify(commandResult)}`),
+        "runnerCommandResult", { value: commandResult, enumerable: false });
+    };
+    const closed = (message: string) => turnStartFailure({ code: "command_execution_failed", message,
+      classification: "provider_session_closed" });
+
+    it.each(["ACPX session is not open", "ACPX turn start failed closed: ACPX sidecar request timed out at turn.start"])(
+      "stops on the first refusal and hands the run to the board: %s", (message) => {
+        const code = nativeSessionFailureSourceCode(closed(message));
+        expect(code).toBe(NATIVE_PROVIDER_SESSION_CLOSED);
+        const disposition = nativeSessionFailureDisposition(1, new Date(), code);
+        expect(disposition).toEqual({ phase: "terminal_failure", failureCode: NATIVE_PROVIDER_SESSION_CLOSED, nextAttemptAt: null });
+        expect(nativeSessionRecoveryProjection({ ...disposition, agentId: "agent" })).toMatchObject({
+          issueStatus: "blocked", recoveryOwner: { kind: "board" },
+        });
+        // A rewrapped failure keeps the class through its authenticated text.
+        expect(nativeSessionFailureSourceCode(new Error(closed(message).message))).toBe(NATIVE_PROVIDER_SESSION_CLOSED);
+        expect(nativeSessionFailureSourceCode(Object.assign(new Error("native session failed"), { cause: closed(message) })))
+          .toBe(NATIVE_PROVIDER_SESSION_CLOSED);
+      });
+
+    it("keeps retrying a start the Runner proved never reached the provider", () => {
+      const failure = turnStartFailure({ code: "command_execution_failed",
+        message: "ACPX turn start failed before its prompt reached the provider: ACPX sidecar request timed out at turn.start" });
+      expect(nativeSessionFailureSourceCode(failure)).toBe("native_session_interrupted");
+      expect(nativeSessionFailureDisposition(1, new Date(), "native_session_interrupted").phase).toBe("retryable_failure");
+    });
+
+    it("keeps today's same-run retries for an unclassified refusal from an older Runner", () => {
+      // An older Runner sends the same text for a session that can reopen after
+      // a restart (provider exit not yet proven), so the text alone proves nothing.
+      const failure = turnStartFailure({ code: "command_execution_failed", message: "ACPX session is not open" });
+      expect(nativeSessionFailureSourceCode(failure)).toBe("native_session_interrupted");
+      // Another command's class is not a turn-start refusal.
+      const other = Object.defineProperty(new Error("PRP command session.open failed"), "runnerCommandResult",
+        { value: { commandType: "session.open", result: { classification: "provider_session_closed" } }, enumerable: false });
+      expect(nativeSessionFailureSourceCode(other)).not.toBe(NATIVE_PROVIDER_SESSION_CLOSED);
+    });
+  });
+
   it("stops same-run retries for an operator while a saved side-effecting call has no result", () => {
     const failure = new NativeProviderSideEffectPendingError(
       "Error: native_runner_source_semantic_reconciliation_required",

server/src/services/native-runtime/native-session-executor.ts

@@ -8312,7 +8312,8 @@ export function nativeSessionFailureDisposition(
     sourceFailureCode === NATIVE_CURRENT_WAKE_COMMENTS_CHANGED_AFTER_READ ||
     sourceFailureCode === "native_session_cleanup_quarantined" ||
     sourceFailureCode === "native_adopted_runner_authentication_timeout" ||
-    sourceFailureCode === "native_provider_usage_limit";
+    sourceFailureCode === "native_provider_usage_limit" ||
+    sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED;
   const permissionDeclined = sourceFailureCode === "native_permission_declined";
   const exhausted = permanentFailure || permissionDeclined || attempt >= 3;
   return {
@@ -8353,6 +8354,33 @@ export function nativeSessionRecoveryProjection(input: {
   };
 }
 
+/**
+ * The Runner closed this run's provider session after a turn start whose prompt
+ * may have reached the provider. Every later same-run `turn.start` is refused
+ * the same way, and resending could duplicate the prompt, so the run stops and
+ * the board offers Retry (a new run with a fresh provider session). A start the
+ * Runner proved never dispatched is not classified and still retries.
+ */
+export const NATIVE_PROVIDER_SESSION_CLOSED = "native_provider_session_closed";
+export const NATIVE_PROVIDER_SESSION_CLOSED_ACTION =
+  "The provider session closed after a turn start that may have reached the provider, and this run cannot reopen it. Retry starts a new run with a fresh provider conversation and the saved task context.";
+const RUNNER_PROVIDER_SESSION_CLOSED = "provider_session_closed";
+
+/** Reads the Runner's closed class from an authenticated turn.start failure. */
+function runnerClosedTurnStartSession(error: unknown, depth = 0): boolean {
+  if (!error || typeof error !== "object" || depth > 5) return false;
+  const value = error as Record<string, unknown>;
+  const result = record(value.runnerCommandResult);
+  if (result.commandType === "turn.start" && record(result.result).classification === RUNNER_PROVIDER_SESSION_CLOSED) {
+    return true;
+  }
+  // The structured result can be lost when the failure is rewrapped; the
+  // authenticated command failure text carries the same closed class.
+  const message = error instanceof Error ? error.message : "";
+  if (/\bPRP command turn\.start failed: \{.*"classification":"provider_session_closed"/.test(message)) return true;
+  return runnerClosedTurnStartSession(value.cause, depth + 1);
+}
+
 export function nativeSessionFailureSourceCode(
   error: unknown,
 ):
@@ -8363,6 +8391,7 @@ export function nativeSessionFailureSourceCode(
   | typeof NATIVE_PROVIDER_SIDE_EFFECT_PENDING
   | "native_provider_approval_required"
   | "native_provider_usage_limit"
+  | typeof NATIVE_PROVIDER_SESSION_CLOSED
   | "native_session_cleanup_quarantined"
   | "native_adopted_runner_authentication_timeout"
   | "runner_remote_provider_artifact_incompatible"
@@ -8424,6 +8453,7 @@ export function nativeSessionFailureSourceCode(
     return error.failure.httpStatus === 401 || error.failure.httpStatus === 403
       ? "runner_ingress_access_rejected" : "runner_ingress_acquisition_failed";
   }
+  if (runnerClosedTurnStartSession(error)) return NATIVE_PROVIDER_SESSION_CLOSED;
   const message = error instanceof Error ? error.message : String(error);
   if (/classification=runtime_package_missing\)/.test(message))
     return "native_provider_runtime_missing";
@@ -12261,6 +12291,8 @@ async function executePaperclipNativeSessionAttempt(
                   ? "Provider permission was declined. Inspect the task and give explicit direction before starting further provider work. Automatic recovery is stopped."
                   : sourceFailureCode === "native_provider_approval_required"
                   ? "Approval required. Review the operation and update the agent's permission setting before retrying. This runner has no interactive approval handler."
+                  : sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED
+                  ? NATIVE_PROVIDER_SESSION_CLOSED_ACTION
                   : sourceFailureCode === "native_session_cleanup_quarantined"
                   ? nativeCleanupRefusalAction(completedTurnCleanup)
                   : sourceFailureCode === "runner_remote_provider_verification_unavailable"
@@ -12530,6 +12562,8 @@ async function executePaperclipNativeSessionAttempt(
               ? "Provider permission was declined. Inspect the task and give explicit direction before starting further provider work. Automatic recovery is stopped."
               : sourceFailureCode === "native_provider_approval_required"
               ? "Approval required. Review the operation and update the agent's permission setting before retrying. This runner has no interactive approval handler."
+              : sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED
+              ? NATIVE_PROVIDER_SESSION_CLOSED_ACTION
               : sourceFailureCode === "native_session_cleanup_quarantined"
               ? nativeCleanupRefusalAction(completedTurnCleanup)
               : sourceFailureCode === "runner_ingress_access_rejected"