fix(native): offer Retry at once when the Runner refuses a turn start on a closed session (#20729)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native ACPX run (for example Claude) starts each turn with
`turn.start`. When a start is cut off, the Runner reads the sidecar's
dispatch journal (#20675).
> - If the prompt never left, the Runner keeps the session reopenable,
and the same-run resume sends the prompt once. If the prompt may have
left, the Runner closes the session, because a resend could duplicate
it.
> - A closed session cannot reopen for the same run. Every same-run
resume got `ACPX session is not open`. The server treated that as a
generic interruption and spent three attempts (about 95 s) before the
board offered Retry.
> - This pull request gives that refusal a closed class in the Runner
and a permanent, named failure code in the server.
> - The benefit is that the board offers a working Retry on the first
refusal, while every recoverable case keeps recovering by itself.

## Linked Issues or Issue Description

**What happened?**
Restart-chaos wave `w-20261111T011704-s9107`, task t05 (local, claude,
workload C1, fault TM2). A freeze cut off a turn start, and the Runner
saved the session `closed`. Three same-run resumes each failed with `PRP
command turn.start failed: … "message":"ACPX session is not open"`.
Recovery stopped with `native_session_retry_exhausted` at 01:25:14, and
the board then offered Retry. The Retry run succeeded.

**Expected behavior**
When the Runner proves the session is closed for this run, the run stops
at the first refusal with a named failure, and the board offers a Retry
that works. A start that the Runner proved never reached the provider
still recovers by itself.

**Steps to reproduce**
1. `cd server && npx vitest run
src/services/native-runtime/native-closed-session-retry.integration.test.ts`
2. On master, "offers Retry on the first refusal, and Retry starts a new
run" fails: the coordinator is `retryable_failure` /
`native_session_interrupted` (a same-run retry in 30 s).

Refs #20675, #20720

## What Changed

- Runner (`durable/mod.rs`, `acpx_provider_backend.rs`): new closed
class `provider_session_closed`. The Runner sets it on these refusals:
- a turn start that saves the session `closed` (`ACPX turn start failed
closed: …`), but not the rejected-checkpoint case, which stays
`prepared` and reopens;
- `ACPX session is not open` when the saved lifecycle is `closed` for
this run;
- the saved turn-end-restart refusal for this run (asserted in
`acpx_turn_end_restart`).
A session that waits for provider-exit proof can reopen after a restart.
Its refusal gets no class.
- Server (`native-session-executor.ts`): a `turn.start` failure with
that class maps to the new permanent code
`native_provider_session_closed`. The server reads the class from the
authenticated command result, and from the command-failure text if the
error was rewrapped. `nativeSessionFailureDisposition` treats the code
as permanent, so the first refusal ends the run as `terminal_failure`
with a board owner and a plain next action.
- Docs: `packages/paperclip-runner/docs/durable-recovery.md` states the
class and the older-Runner behavior.
- Tests:
- `native-closed-session-retry.integration.test.ts` (new, embedded
Postgres) drives `executePaperclipNativeSession` with the Runner
refusal. On the first refusal, the board shows `canRetry: true`.
`requestTaskRetry` is accepted, and `resumeTaskRetry` starts exactly one
new run with a fresh provider session. A start proved undispatched stays
`retryable_failure`.
- Unit tests in `native-session-executor.test.ts`: both closed refusals,
a rewrapped error, an undispatched start, an unclassified older-Runner
refusal, and a class on another command.
- Rust `acpx_cut_turn_start`: a start cut off before dispatch has no
class. A start cut off after dispatch, and the later refusal, carry
`provider_session_closed`.

## Verification

- Red on master, green here: step 1 above.
- `npx vitest run
src/services/native-runtime/native-session-executor.test.ts`: 1358/1358
pass.
- `cargo test -p paperclip-runner-core --test acpx_park_preparation` (26
pass), `--lib acpx` (94 pass), `--test acpx_provider_turns --test
acpx_provider_session --test acpx_warm_attachment` (all pass), `cargo
fmt --check` clean.
- `npx tsc --noEmit -p server/tsconfig.json`: clean.
- Not run: a chaos wave.

## Risks

- This changes the Rust Runner. Daytona needs a new runner image before
remote runs get the class. Until then, an older Runner sends the refusal
without the class, and the server keeps today's bounded same-run
retries. That path is unchanged and ends with a working Retry. The
server does not trust the text alone, because an older Runner sends the
same text for a session that can still reopen.
- The class is set only where the Runner's saved state proves that no
later start of this run can reopen the session. A false class would cost
one automatic resume and show Retry earlier. It never replays provider
work.
- No persisted shape changes. The class is an optional field of a failed
command result. Old saved state resumes unchanged. Nothing new runs
before `turn.start`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic), model ID `claude-opus-5-5`, run in Claude Code
with tool use (shell, file edit) and extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
DottaandClaude authored and GitHub committed 2026-11-11 00:12:41 -06:00
1 parent 1143abaf92
commit 08ff0828d6
8 files changed
+255 -20

No files matched your search

@@ -8312,7 +8312,8 @@ export function nativeSessionFailureDisposition(
sourceFailureCode === NATIVE_CURRENT_WAKE_COMMENTS_CHANGED_AFTER_READ ||
sourceFailureCode === "native_session_cleanup_quarantined" ||
sourceFailureCode === "native_adopted_runner_authentication_timeout" ||
sourceFailureCode === "native_provider_usage_limit";
sourceFailureCode === "native_provider_usage_limit" ||
sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED;
const permissionDeclined = sourceFailureCode === "native_permission_declined";
const exhausted = permanentFailure || permissionDeclined || attempt >= 3;
return {
@@ -8353,6 +8354,33 @@ export function nativeSessionRecoveryProjection(input: {
};
}
/**
* The Runner closed this run's provider session after a turn start whose prompt
* may have reached the provider. Every later same-run `turn.start` is refused
* the same way, and resending could duplicate the prompt, so the run stops and
* the board offers Retry (a new run with a fresh provider session). A start the
* Runner proved never dispatched is not classified and still retries.
*/
export const NATIVE_PROVIDER_SESSION_CLOSED = "native_provider_session_closed";
export const NATIVE_PROVIDER_SESSION_CLOSED_ACTION =
"The provider session closed after a turn start that may have reached the provider, and this run cannot reopen it. Retry starts a new run with a fresh provider conversation and the saved task context.";
const RUNNER_PROVIDER_SESSION_CLOSED = "provider_session_closed";
/** Reads the Runner's closed class from an authenticated turn.start failure. */
function runnerClosedTurnStartSession(error: unknown, depth = 0): boolean {
if (!error || typeof error !== "object" || depth > 5) return false;
const value = error as Record<string, unknown>;
const result = record(value.runnerCommandResult);
if (result.commandType === "turn.start" && record(result.result).classification === RUNNER_PROVIDER_SESSION_CLOSED) {
return true;
}
// The structured result can be lost when the failure is rewrapped; the
// authenticated command failure text carries the same closed class.
const message = error instanceof Error ? error.message : "";
if (/\bPRP command turn\.start failed: \{.*"classification":"provider_session_closed"/.test(message)) return true;
return runnerClosedTurnStartSession(value.cause, depth + 1);
}
export function nativeSessionFailureSourceCode(
error: unknown,
):
@@ -8363,6 +8391,7 @@ export function nativeSessionFailureSourceCode(
| typeof NATIVE_PROVIDER_SIDE_EFFECT_PENDING
| "native_provider_approval_required"
| "native_provider_usage_limit"
| typeof NATIVE_PROVIDER_SESSION_CLOSED
| "native_session_cleanup_quarantined"
| "native_adopted_runner_authentication_timeout"
| "runner_remote_provider_artifact_incompatible"
@@ -8424,6 +8453,7 @@ export function nativeSessionFailureSourceCode(
return error.failure.httpStatus === 401 || error.failure.httpStatus === 403
? "runner_ingress_access_rejected" : "runner_ingress_acquisition_failed";
}
if (runnerClosedTurnStartSession(error)) return NATIVE_PROVIDER_SESSION_CLOSED;
const message = error instanceof Error ? error.message : String(error);
if (/classification=runtime_package_missing\)/.test(message))
return "native_provider_runtime_missing";
@@ -12261,6 +12291,8 @@ async function executePaperclipNativeSessionAttempt(
? "Provider permission was declined. Inspect the task and give explicit direction before starting further provider work. Automatic recovery is stopped."
: sourceFailureCode === "native_provider_approval_required"
? "Approval required. Review the operation and update the agent's permission setting before retrying. This runner has no interactive approval handler."
: sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED
? NATIVE_PROVIDER_SESSION_CLOSED_ACTION
: sourceFailureCode === "native_session_cleanup_quarantined"
? nativeCleanupRefusalAction(completedTurnCleanup)
: sourceFailureCode === "runner_remote_provider_verification_unavailable"
@@ -12530,6 +12562,8 @@ async function executePaperclipNativeSessionAttempt(
? "Provider permission was declined. Inspect the task and give explicit direction before starting further provider work. Automatic recovery is stopped."
: sourceFailureCode === "native_provider_approval_required"
? "Approval required. Review the operation and update the agent's permission setting before retrying. This runner has no interactive approval handler."
: sourceFailureCode === NATIVE_PROVIDER_SESSION_CLOSED
? NATIVE_PROVIDER_SESSION_CLOSED_ACTION
: sourceFailureCode === "native_session_cleanup_quarantined"
? nativeCleanupRefusalAction(completedTurnCleanup)
: sourceFailureCode === "runner_ingress_access_rejected"