fix(native): stop a session.open model rejection instead of a doomed bootstrap retry (#19190)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native Runner runs start a provider session with the `session.open`
command before any prompt
> - A definite `session.open` rejection is treated as a pre-session
failure and gets a bounded same-run bootstrap retry
> - The runner also reports a closed classification. Two classifications
say that the provider cannot run the selected model:
`requested_model_unsupported` and `grok_model_mismatch`
> - Paperclip ignored that classification. It scheduled a retry that
must fail the same way. On a remote sandbox with a retained runner PID,
the due retry entered restart recovery. Restart recovery has no
checkpoint for a bootstrap retry, so it failed closed as
`native_restart_recovery_blocked` / `ambiguous_remote_recovery_evidence`
> - The task then stayed `blocked`. The recovery actions settled as
no-replay holds, so no active action told anyone to change the model,
and the parent task stalled too
> - This pull request classifies those two definite rejections as the
existing `native_provider_model_rejected` failure
> - The benefit is a terminal failure on the first attempt, an accurate
error, and an active board recovery action that names the next step:
choose a supported model

## Linked Issues or Issue Description

**What happened?**
On a hosted staging instance, a Grok agent run failed at `session.open`.
The runner log said `ACPX sidecar command session.open was rejected
(retryable=false, classification=grok_model_mismatch)`. Paperclip
recorded `recoveryMode: "bootstrap_retry"` and scheduled a same-run
retry 30 seconds later. The run had a remote sandbox and a recorded
sandbox PID, so the due retry went to restart recovery. Restart recovery
found no checkpoint and no provider events and failed closed with
`ambiguous_remote_recovery_evidence`. The run ended as
`native_restart_recovery_blocked`. The task stayed `blocked` with no
active recovery action for more than one hour. Its parent task stayed
blocked on it.

**What did you expect to happen?**
The selected model cannot start, so a retry cannot succeed. The run
should fail once with the model-rejection error. The task should show
the existing "choose a model" action, owned by the board, until someone
changes the model and retries.

**Steps to reproduce**
1. Start a native run whose `session.open` command fails with
`classification=grok_model_mismatch` (or `requested_model_unsupported`).
2. Read the `native_run_finalizations` row. Before this change it is
`retryable_failure` with `recoveryMode: "bootstrap_retry"` and a
`nextAttemptAt`.
3. On a remote sandbox run with a recorded PID, the due retry enters
restart recovery and fails as `native_restart_recovery_blocked`.

**Paperclip version or commit**
`master` at `f5f5184f23`.

Related: #18910 routes due remote retries with a retained PID to restart
recovery. #18908 continues verified failed native preparation. Open PRs
#19177, #19180, #19182 and #19184 change no-replay hold settlement and
workspace hold release. They do not change this classification. This PR
does not duplicate them.

## What Changed

- `nativeSessionFailureSourceCode`
(`server/src/services/native-runtime/native-session-executor.ts`)
returns `native_provider_model_rejected` for a definite `session.open`
failure or rejection whose runner classification is
`requested_model_unsupported` or `grok_model_mismatch`. Other definite
`session.open` rejections still return `provider_session_open_rejected`
and keep the bounded startup retry. An indeterminate `session.open`
outcome stays `native_session_interrupted`.
- `native_provider_model_rejected` is already a permanent failure. The
run stops on the first attempt with no `nextAttemptAt`, so it never
enters the restart-recovery retry path.
- When no provider session exists, the coordinator `nextAction` and the
board recovery action `nextAction` say: choose a supported model for
this agent, then retry the task.
- The existing product paths for `native_provider_model_rejected` apply:
the task presentation shows `choose_model`, and the recovery service
treats it as `configuration_incomplete`.
- `doc/execution-semantics.md` describes the rule under "Native provider
model capacity".

## Verification

Red before the fix (new test, unchanged production code):

```sh
cd server && npx vitest run src/services/native-runtime/native-startup-retry.integration.test.ts
# Tests 2 failed | 21 passed
# failureCode: native_session_interrupted, phase: retryable_failure, recoveryMode: bootstrap_retry,
# nextAction "Retry provider bootstrap on this same run ..." - the same record seen on staging
```

Green after the fix:

```sh
cd server && npx vitest run src/services/native-runtime/native-startup-retry.integration.test.ts
# Tests 23 passed

cd server && npx vitest run src/services/native-runtime/native-session-executor.test.ts \
  src/services/native-runtime/native-startup-retry.integration.test.ts src/services/task-interruption-presentation.test.ts
# Test Files 3 passed, Tests 1186 passed

pnpm --filter @paperclipai/server typecheck   # exit 0
```

The new integration cases use embedded PostgreSQL and the real executor
failure path. For both classifications they check:

- coordinator `terminal_failure`, `failureCode:
native_provider_model_rejected`, `nextAttemptAt: null`, attempt 1
- run `failed` with `errorCode: native_provider_model_rejected`
- task `blocked`, execution lock released
- an `active` board recovery action with cause
`native_provider_model_rejected`, return owner set to the agent, and the
choose-model next action
- `settleUnrecoverableExecutions` leaves the action `active`, so it does
not become a silent no-replay hold
- `claimNativeSessionResumptions` claims nothing, so there is no second
executor

The unit test also checks that `session_ensure_failed` keeps its bounded
retry and that an indeterminate model rejection stays unclassified.

A live Grok run with a mismatched model was not run. The failure record
in the test matches the staging record field by field.

## Risks

- Low risk. The change adds two closed runner classifications to an
existing permanent failure code. Other `session.open` rejections keep
their current retry.
- A model mismatch that a retry could fix would now need an explicit
retry. The runner already marks these rejections `retryable=false`.
- This PR does not change restart recovery. A remote bootstrap retry
with a retained sandbox PID for other `session.open` causes can still
fail closed as `ambiguous_remote_recovery_evidence`. That needs a
separate change with remote stop evidence.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used in
Claude Code with tool use (shell, file edits, browser reads of the
staging API). It did the investigation, code change, tests and PR text.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-04 23:24:34 -06:00
1 parent 46a50a3bdd
commit f1516e8009
4 files changed
+80 -1

No files matched your search

+8
View File
@@ -959,6 +959,14 @@ workspace finalization, and environment cleanup to settle. Model incompatibility
usage-limit exhaustion, unknown failures, and unbound diagnostic text do not
qualify as capacity failures.
A definite `session.open` rejection whose runner classification says the
provider cannot run the selected model (`requested_model_unsupported`,
`grok_model_mismatch`) is a model rejection, not a startup interruption. It
stops on its first attempt as `native_provider_model_rejected`, with no
same-run bootstrap retry, and leaves an active board recovery action that asks
for a supported model. Other `session.open` rejections keep the bounded
startup retry. An indeterminate `session.open` outcome stays unclassified.
### Workspace scan failures before provider startup
Workspace source preparation keeps its dedicated advisory-lock connection active
@@ -17979,6 +17979,15 @@ it("classifies only a definite session.open outcome as a bounded pre-session fai
.toBe("native_session_interrupted");
expect(nativeSessionFailureSourceCode(new Error(`PRP command turn.start failed: ${detail}`)))
.toBe("native_session_interrupted");
// A definite model rejection needs a model choice; an indeterminate one stays unknown.
const modelDetail = detail.replace("classification=session_ensure_failed", "classification=grok_model_mismatch");
const modelCode = nativeSessionFailureSourceCode(new Error(`PRP command session.open failed: ${modelDetail}`));
expect(modelCode).toBe("native_provider_model_rejected");
expect(nativeSessionFailureDisposition(1, new Date(), modelCode)).toEqual({
phase: "terminal_failure", failureCode: "native_provider_model_rejected", nextAttemptAt: null,
});
expect(nativeSessionFailureSourceCode(new Error(`PRP command session.open indeterminate: ${modelDetail}`)))
.toBe("native_session_interrupted");
});
describe("HTTP Stop during remote runner staging", () => {
it("does not dispatch after Stop and bounded bootstrap cleanup", async () => {
@@ -7305,6 +7305,12 @@ export function nativeSessionFailureSourceCode(
// before any prompt, so provider work cannot have started. An indeterminate
// outcome stays generic.
if (/\bPRP command session\.open (?:failed|rejected): /.test(message)) {
// The sidecar's closed classification says the provider cannot run the
// selected model. A same-run bootstrap retry would open the same session
// and fail the same way, so this needs a model choice, not a retry.
if (/classification=(?:requested_model_unsupported|grok_model_mismatch)\)/.test(message)) {
return "native_provider_model_rejected";
}
return "provider_session_open_rejected";
}
return "native_session_interrupted";
@@ -7313,6 +7319,9 @@ export function nativeSessionFailureSourceCode(
const NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION =
"The current runner cannot restore this task's saved provider session, so retrying this run cannot succeed. Retry the task to continue in a fresh provider session with the same workspace. The earlier provider conversation is not carried over.";
const NATIVE_STARTUP_MODEL_REJECTED_ACTION =
"The provider could not start the selected model before any provider work began. Choose a supported model for this agent, then retry the task. Automatic retries are stopped.";
const NATIVE_CLEANUP_OPERATOR_RECOVERY_MESSAGE =
"Send a new message to continue after Paperclip verifies that the previous provider and its tools have stopped. If cleanup cannot be verified, inspect the run and its environment. Clearing a task session does not resolve this quarantine. Automatic retries are stopped.";
@@ -10746,6 +10755,8 @@ async function executePaperclipNativeSessionWithinScope(
? "The computer provider rejected access. Restore the environment's credentials or permissions before retrying. Automatic retries are stopped."
: sourceFailureCode === "native_provider_terminal_failed"
? "The provider session is permanently unusable. Verify stopped execution, completed actions, and task context before starting a linked continuation."
: sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished
? NATIVE_STARTUP_MODEL_REJECTED_ACTION
: sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE
? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION
: recoveryEvidence.recoveryMode === "ambiguous_state"
@@ -10997,6 +11008,8 @@ async function executePaperclipNativeSessionWithinScope(
? "The computer provider rejected access. Restore the environment's credentials or permissions before retrying. Automatic retries are stopped."
: sourceFailureCode === "native_provider_terminal_failed"
? "Verify that the failed provider stopped and reconcile its action outcomes. A linked continuation can proceed only after these checks succeed."
: sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished
? NATIVE_STARTUP_MODEL_REJECTED_ACTION
: sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE
? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION
: recoveryEvidence.recoveryMode === "ambiguous_state"
@@ -38,6 +38,7 @@ import { currentNativeControllerIdentity } from "./native-restart-recovery.js";
import { loadNativeRunnerCheckpoint } from "./native-runner-checkpoint.js";
import { environmentRunnerIngressErrorData, JsonRpcCallError, PluginEnvironmentRunnerIngressError } from "@paperclipai/plugin-sdk";
import { accountRunCost } from "../run-cost-accounting.js";
import { settleUnrecoverableExecutions } from "../execution-recovery-resolution.js";
type SessionOpenStatus = "failed" | "rejected" | "indeterminate";
@@ -45,6 +46,7 @@ type SessionOpenStatus = "failed" | "rejected" | "indeterminate";
function sessionOpenFailure(options: {
status?: SessionOpenStatus;
providerFailure?: Record<string, unknown>;
classification?: string;
} = {}) {
const status = options.status ?? "failed";
const result = {
@@ -52,7 +54,7 @@ function sessionOpenFailure(options: {
code: "command_execution_failed",
message: options.providerFailure
? "Provider request failed"
: "failed to start ACPX provider: ACPX sidecar command session.open was rejected (retryable=false, classification=session_ensure_failed)",
: `failed to start ACPX provider: ACPX sidecar command session.open was rejected (retryable=false, classification=${options.classification ?? "session_ensure_failed"})`,
...(options.providerFailure ? { providerFailure: options.providerFailure } : {}),
},
status,
@@ -476,6 +478,53 @@ describe("native provider startup retry", () => {
expect(state.issue.status).toBe("in_progress");
});
it.each(["grok_model_mismatch", "requested_model_unsupported"])(
"stops at a session.open %s rejection with a choose-model action instead of a doomed retry",
async (classification) => {
const seeded = await seedRun(await createAgent());
await failStartup(seeded, sessionOpenFailure({ classification }));
const state = await read(seeded);
// No automatic same-run bootstrap retry: the selected model cannot start.
expect(state.coordinator).toMatchObject({
phase: "terminal_failure",
recoveryState: "blocked",
failureCode: "native_provider_model_rejected",
attempt: 1,
nextAttemptAt: null,
failureDetail: expect.objectContaining({
originalFailureCode: "native_provider_model_rejected",
providerSessionEstablished: false,
providerEventsExist: false,
checkpointExists: false,
}),
});
expect(String(state.coordinator.failureDetail?.nextAction)).toContain("Choose a supported model");
expect(state.run).toMatchObject({ status: "failed", errorCode: "native_provider_model_rejected" });
expect(state.issue.status).toBe("blocked");
expect(state.issue.executionRunId).toBeNull();
// A named, durable board action owns the next step.
expect(state.action).toMatchObject({
status: "active",
ownerType: "board",
cause: "native_provider_model_rejected",
returnOwnerAgentId: seeded.agentId,
wakePolicy: null,
});
expect(state.action!.nextAction).toContain("Choose a supported model");
// The no-replay settler must not fold this owned action into a silent hold.
await settleUnrecoverableExecutions(db);
expect((await read(seeded)).action).toMatchObject({ id: state.action!.id, status: "active" });
expect(await claimNativeSessionResumptions({
db,
runnerInstanceId: "startup-retry-test",
runIds: [seeded.runId],
now: new Date(Date.now() + 60 * 60_000),
deferIneligible: false,
})).toEqual([]);
},
);
it("blocks with a clear reason once the bounded startup budget is exhausted", async () => {
const seeded = await seedRun(await createAgent());
await failStartup(seeded, sessionOpenFailure());