From 8b3348c6bade125ed2f79014ac661cfc79e48176 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 10 Nov 2026 04:41:02 -0600 Subject: [PATCH] fix(runner): classify Claude's rejected selected model as an unavailable model (#20513) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runner agents on the Claude (ACPX) provider run with the model the user saves on the agent > - When the provider cannot run that model, Paperclip should classify the run as `native_provider_model_rejected`, ask the user to choose a model, and let Retry finish the task > - Claude Code accepts an unknown model at session start and reports it only on the first turn, as an AIR `request` failure with a fixed title. The runner reported it as an unknown provider failure > - So the run failed as `native_provider_terminal_failed`, and the task stopped with `provider_effect_inventory_unavailable` ("Automatic recovery stopped. Recorded work is preserved…") without naming the model > - This pull request classifies that Claude title as an unavailable model, the category the native runtime already turns into `native_provider_model_rejected` > - The benefit is that an unknown Claude model asks for a model choice and names it, as Codex does after #20488 ## Linked Issues or Issue Description **What happened?** Config wave `w-20261110T094430-s207-cf` on `master` (82abb3e), `--adapters claude`: CF3 `model_id` failed both times (t01 K1, t04 K2) as `CF3/model_id:actionable_message`. The task showed `Stopped`, cause `provider_effect_inventory_unavailable`, "Automatic recovery stopped. Recorded work is preserved; actions with unverified outcomes will not be repeated." The run was `native_provider_terminal_failed`, "ACP agent reported a terminal request failure." Root cause, from a direct ACP probe of the bundled `@agentclientprotocol/claude-agent-acp` 0.73.0 with the AIR `sessionFailure` capability and the model in `_meta.claudeCode.options.model` (as ACPX passes it): `session/new` lists `chaos-missing-model-abc12345` among the model options, so admission's model check passes. `session/prompt` then returns `stopReason: "end_turn"` with `_meta.jetbrains.air.sessionFailure` = `{"category":"request","severity":"error","title":"There's an issue with the selected model (chaos-missing-model-abc12345). It may not exist or you may not have access to it. Run --model to pick a different model.","actions":[]}`. ACPX turns that into "ACP agent reported a terminal request failure.", and `acpxProviderFailure` classified only AIR `limit` titles, so the provider failure stayed `unknown`. (Selecting the same model later through `session/set_config_option` instead returns JSON-RPC `-32603` "Internal error", `data.details` "Invalid value for config option model: ". The runner does not take that path for Claude, so this PR does not change it.) **Expected behavior** The run is `native_provider_model_rejected`, the task shows `choose_model` with the rejected model, no recovery incident opens, and Retry finishes the task after the model is repaired. **Steps to reproduce** `node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/config-wave.ts --faults CF3 --adapters claude --references model_id --tasks 2 --concurrency 2 --restarts K1,K2 --seed 207` on `master`. **Paperclip version or commit** `master` at 82abb3e. ## What Changed - `packages/paperclip-runner/src/drivers/acpx/provider-failure.ts`: for the `claude` agent, an AIR `request` session failure whose title is Claude Code's fixed "There's an issue with the selected model (…). It may not exist or you may not have access to it." is classified `model_unavailable` (`requires_action`). The native runtime already marks such a failed turn `native_provider_model_rejected`, and the server already projects that as `choose_model` with `rejectedModel`. - Other AIR categories, other agents and other request titles stay `unknown`. - `server/src/services/native-runtime/native-session-executor.ts`: a `native_provider_model_rejected` failure after the provider session opened now gets a choose-model next action ("The provider rejected the selected model. Choose a supported model for this agent, then retry the task. Automatic retries are stopped."). Before, only a rejection before the session opened got that action; the Claude case fell through to "Inspect the original provider failure and explicitly resolve the ambiguous session state; do not open a replacement provider session." Recovery mode, blocking and retry behavior are unchanged; only the action text changes. ## Verification - Red before, green after: `src/drivers/provider-failure.test.ts` — "classifies Claude's rejection of the selected model as an unavailable model", with the exact title captured from `claude-agent-acp` 0.73.0, through both `acpxTerminalResult` and the thrown form, plus negative cases. - Red before, green after: `native-startup-retry.integration.test.ts` — "asks for a model choice when the provider rejects the model on an opened session's turn (Claude ACPX)": a durable `turn.started` and the `NativeProviderTerminalFailure` the runtime throws; the coordinator and the board recovery action name the model choice, not ambiguous state. - Live: on `master` the wave failed (see above). On this branch: `w-20261110T103809-s208-cf` on head 3881650 (CF3 `model_id`, claude, K1 and K2): **2/2 pass**. Each task rests as `waiting_for_user` / `choose_model` with `rejectedModel: "chaos-missing-model-…"`, cause `native_provider_model_rejected`, next action "The provider rejected the selected model. Choose a supported model for this agent, then retry the task. Automatic retries are stopped.", rests again after the restart, and is done after the model is restored and Retry. An earlier wave on e552b24 (`w-20261110T100526-s207-cf`, 2/2 pass) still showed the ambiguous-state text that the second commit replaces. ## Risks - Low risk. The match is limited to the Claude agent, the AIR `request` category and Claude Code's fixed title. If Claude Code changes the text, the run falls back to today's generic failure. - No profile, digest or pack contract change (`rules/acpx-profile-identity.mdc`): ordinary Runner source. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs, a direct ACP probe of `claude-agent-acp`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude --- .../src/drivers/acpx/provider-failure.ts | 15 +++++++- .../src/drivers/provider-failure.test.ts | 17 +++++++++ .../native-runtime/native-session-executor.ts | 9 +++++ .../native-startup-retry.integration.test.ts | 37 ++++++++++++++++++- 4 files changed, 76 insertions(+), 2 deletions(-) diff --git a/packages/paperclip-runner/src/drivers/acpx/provider-failure.ts b/packages/paperclip-runner/src/drivers/acpx/provider-failure.ts index 51cd3ae111e..d66fb3c27cf 100644 --- a/packages/paperclip-runner/src/drivers/acpx/provider-failure.ts +++ b/packages/paperclip-runner/src/drivers/acpx/provider-failure.ts @@ -48,6 +48,18 @@ function sessionLimitCategory(failure: AcpxTerminalSessionFailure | undefined): return undefined; } +/** + * Claude Code reports a model it cannot run (unknown, retired, or not on the + * account) as an AIR `request` failure with this fixed title. The user must + * choose another model, so it is an unavailable model, not a generic request. + */ +const CLAUDE_SELECTED_MODEL_REJECTED = + /^There's an issue with the selected model \([^()\n]{1,200}\)\. It may not exist or you may not have access to it\./; +function sessionModelCategory(agent: string, failure: AcpxTerminalSessionFailure | undefined): ProviderFailure["category"] | undefined { + return agent === "claude" && failure?.category === "request" && typeof failure.title === "string" && + CLAUDE_SELECTED_MODEL_REJECTED.test(failure.title.trim()) ? "model_unavailable" : undefined; +} + /** * `ownMessage` keeps an error's own message when it carries no ACP error. Use * it only where every rejection belongs to the provider: a failed terminal @@ -72,7 +84,8 @@ export function acpxProviderFailure(error: unknown, agent: string, phase: Provid const code = detail.type ?? detail.code ?? data.errorType ?? data.errorKind; const category = (typeof code === "string" ? CATEGORIES.get(code) : undefined) ?? (acp?.code === ACP_AUTH_REQUIRED ? "authentication" : undefined) - ?? sessionLimitCategory(sessionFailure); + ?? sessionLimitCategory(sessionFailure) + ?? sessionModelCategory(agent, sessionFailure); return createProviderFailure({ provider: agent === "claude" ? "anthropic" : agent === "codex" ? "openai" : agent, phase, message, httpStatus: status, providerCode: code, requestId: data.request_id, category, environment }); } diff --git a/packages/paperclip-runner/src/drivers/provider-failure.test.ts b/packages/paperclip-runner/src/drivers/provider-failure.test.ts index f0e48ca19a4..3f832b3a1cd 100644 --- a/packages/paperclip-runner/src/drivers/provider-failure.test.ts +++ b/packages/paperclip-runner/src/drivers/provider-failure.test.ts @@ -79,6 +79,23 @@ describe("provider failure boundary", () => { .toMatchObject({ category: "rate_limit", message: `${generic}\nClaude is temporarily rate limited.` }); }); + it("classifies Claude's rejection of the selected model as an unavailable model", () => { + const generic = "ACP agent reported a terminal request failure."; + const failed = { status: "failed", error: { code: "RUNTIME", message: generic } }; + // The AIR sessionFailure claude-agent-acp 0.73.0 returns for an unknown model (live probe, 2026-11-10). + const title = "There's an issue with the selected model (chaos-missing-model-abc12345). It may not exist or you may not have access to it. Run --model to pick a different model."; + expect(acpxTerminalResult(failed, "claude", {}, { category: "request", title })).toMatchObject({ status: "failed", error: { providerFailure: { + provider: "anthropic", phase: "turn", category: "model_unavailable", retryability: "requires_action", message: `${generic}\n${title}` } } }); + // The thrown form of the same terminal is classified the same way. + expect(acpxProviderFailure(new Error(generic), "claude", "turn", {}, false, { category: "request", title })) + .toMatchObject({ category: "model_unavailable", retryability: "requires_action" }); + // Another category, another agent, or another request failure stays unknown. + for (const [agent, category, text] of [["claude", "service", title], ["codex", "request", title], + ["claude", "request", "Claude could not process this request."]] as const) { + expect(acpxTerminalResult(failed, agent, {}, { category, title: text })).toMatchObject({ error: { providerFailure: { category: "unknown" } } }); + } + }); + it("keeps the ACPX terminal cause when the raw result exceeds its payload limit", () => { const terminal = acpxTerminalResult({ status: "failed", error: { ...acp, data: { ...acp.data, responseBody: "private-body".repeat(7000) } } }, "claude"); diff --git a/server/src/services/native-runtime/native-session-executor.ts b/server/src/services/native-runtime/native-session-executor.ts index 78f65266595..be4537caf08 100644 --- a/server/src/services/native-runtime/native-session-executor.ts +++ b/server/src/services/native-runtime/native-session-executor.ts @@ -8210,6 +8210,11 @@ const NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION = const NATIVE_STARTUP_MODEL_REJECTED_ACTION = "The provider could not start the selected model before any provider work began. Choose a supported model for this agent, then retry the task. Automatic retries are stopped."; +// The provider session opened, then the provider refused the selected model on +// the turn. The task needs a model choice, not a reconciliation of session state. +const NATIVE_TURN_MODEL_REJECTED_ACTION = + "The provider rejected the selected model. Choose a supported model for this agent, then retry the task. Automatic retries are stopped."; + const NATIVE_CLEANUP_OPERATOR_RECOVERY_MESSAGE = "Send a new message to continue after Paperclip verifies that the previous provider and its tools have stopped. Clearing a task session does not resolve this quarantine. Automatic retries are stopped."; @@ -11913,6 +11918,8 @@ async function executePaperclipNativeSessionAttempt( ? "The work and waiting state are saved. Confirm shutdown of the retained execution and settle that same handoff without repeating provider work." : sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished ? NATIVE_STARTUP_MODEL_REJECTED_ACTION + : sourceFailureCode === "native_provider_model_rejected" + ? NATIVE_TURN_MODEL_REJECTED_ACTION : sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE ? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION : sourceFailureCode === NATIVE_PROVIDER_SIDE_EFFECT_PENDING @@ -12170,6 +12177,8 @@ async function executePaperclipNativeSessionAttempt( ? "Verify that the failed provider stopped and reconcile its action outcomes. A linked continuation can proceed only after these checks succeed." : sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished ? NATIVE_STARTUP_MODEL_REJECTED_ACTION + : sourceFailureCode === "native_provider_model_rejected" + ? NATIVE_TURN_MODEL_REJECTED_ACTION : sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE ? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION : sourceFailureCode === NATIVE_PROVIDER_SIDE_EFFECT_PENDING diff --git a/server/src/services/native-runtime/native-startup-retry.integration.test.ts b/server/src/services/native-runtime/native-startup-retry.integration.test.ts index 936a705a4c3..4d75db6db41 100644 --- a/server/src/services/native-runtime/native-startup-retry.integration.test.ts +++ b/server/src/services/native-runtime/native-startup-retry.integration.test.ts @@ -33,7 +33,7 @@ import { executePaperclipNativeSession, prepareNativeSessionOpenTimeoutRetry, sc import { buildNativeCompletionContract } from "./completion-contracts.js"; import { prepareNativeHeartbeatRun } from "./prepare-native-run.js"; import { claimNativeSessionResumptions } from "./native-finalization-reconciler.js"; -import { DurablePrpControlPlane } from "../../vendor/paperclip-runner/index.js"; +import { DurablePrpControlPlane, NativeProviderTerminalFailure } from "../../vendor/paperclip-runner/index.js"; import { createNativeRunnerExecutionDriver } from "./native-runner-execution-driver.js"; import { currentNativeControllerIdentity } from "./native-restart-recovery.js"; import { loadNativeRunnerCheckpoint } from "./native-runner-checkpoint.js"; @@ -785,6 +785,41 @@ describe("native provider startup retry", () => { }, ); + it("asks for a model choice when the provider rejects the model on an opened session's turn (Claude ACPX)", async () => { + const seeded = await seedRun(await createAgent()); + provider.execute.mockReset().mockImplementation(async () => { + await db.insert(heartbeatRunEvents).values({ + companyId, runId: seeded.runId, agentId: seeded.agentId, seq: 900, + eventType: "turn.started", stream: "system", + }); + // What the native runtime throws for Claude's AIR "selected model" request failure. + throw new NativeProviderTerminalFailure("RUNTIME", false, + "native_provider_model_rejected: ACP agent reported a terminal request failure."); + }); + await expect(executePaperclipNativeSession({ + db, execution: seeded.execution, runnerInstanceId: seeded.runnerInstanceId, + backend: { + descriptor: async () => { throw new Error("Unexpected provider descriptor call"); }, + openSession: async () => { throw new Error("Unexpected provider session call"); }, + }, + })).rejects.toThrow("native_provider_model_rejected"); + + const state = await read(seeded); + expect(state.coordinator).toMatchObject({ + phase: "terminal_failure", + failureCode: "native_provider_model_rejected", + nextAttemptAt: null, + failureDetail: expect.objectContaining({ providerSessionEstablished: true }), + }); + expect(String(state.coordinator.failureDetail?.nextAction)).toContain("Choose a supported model"); + expect(String(state.coordinator.failureDetail?.nextAction)).not.toContain("ambiguous"); + expect(state.run).toMatchObject({ status: "failed", errorCode: "native_provider_model_rejected" }); + expect(state.issue.status).toBe("blocked"); + expect(state.action).toMatchObject({ ownerType: "board", cause: "native_provider_model_rejected", wakePolicy: null }); + expect(state.action!.nextAction).toContain("Choose a supported model"); + expect(state.action!.nextAction).not.toContain("ambiguous"); + }); + it("blocks with a clear reason once the bounded startup budget is exhausted", async () => { const seeded = await seedRun(await createAgent()); await failStartup(seeded, sessionOpenFailure());