mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 05:11:49 +01:00
fix(runner): classify Claude's rejected selected model as an unavailable model (#20513)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner agents on the Claude (ACPX) provider run with the
model the user saves on the agent
> - When the provider cannot run that model, Paperclip should classify
the run as `native_provider_model_rejected`, ask the user to choose a
model, and let Retry finish the task
> - Claude Code accepts an unknown model at session start and reports it
only on the first turn, as an AIR `request` failure with a fixed title.
The runner reported it as an unknown provider failure
> - So the run failed as `native_provider_terminal_failed`, and the task
stopped with `provider_effect_inventory_unavailable` ("Automatic
recovery stopped. Recorded work is preserved…") without naming the model
> - This pull request classifies that Claude title as an unavailable
model, the category the native runtime already turns into
`native_provider_model_rejected`
> - The benefit is that an unknown Claude model asks for a model choice
and names it, as Codex does after #20488
## Linked Issues or Issue Description
**What happened?**
Config wave `w-20261110T094430-s207-cf` on `master` (82abb3e),
`--adapters claude`: CF3 `model_id` failed both times (t01 K1, t04 K2)
as `CF3/model_id:actionable_message`. The task showed `Stopped`, cause
`provider_effect_inventory_unavailable`, "Automatic recovery stopped.
Recorded work is preserved; actions with unverified outcomes will not be
repeated." The run was `native_provider_terminal_failed`, "ACP agent
reported a terminal request failure."
Root cause, from a direct ACP probe of the bundled
`@agentclientprotocol/claude-agent-acp` 0.73.0 with the AIR
`sessionFailure` capability and the model in
`_meta.claudeCode.options.model` (as ACPX passes it): `session/new`
lists `chaos-missing-model-abc12345` among the model options, so
admission's model check passes. `session/prompt` then returns
`stopReason: "end_turn"` with `_meta.jetbrains.air.sessionFailure` =
`{"category":"request","severity":"error","title":"There's an issue with
the selected model (chaos-missing-model-abc12345). It may not exist or
you may not have access to it. Run --model to pick a different
model.","actions":[]}`. ACPX turns that into "ACP agent reported a
terminal request failure.", and `acpxProviderFailure` classified only
AIR `limit` titles, so the provider failure stayed `unknown`.
(Selecting the same model later through `session/set_config_option`
instead returns JSON-RPC `-32603` "Internal error", `data.details`
"Invalid value for config option model: <id>". The runner does not take
that path for Claude, so this PR does not change it.)
**Expected behavior**
The run is `native_provider_model_rejected`, the task shows
`choose_model` with the rejected model, no recovery incident opens, and
Retry finishes the task after the model is repaired.
**Steps to reproduce**
`node cli/node_modules/tsx/dist/cli.mjs
tests/restart-chaos/config-wave.ts --faults CF3 --adapters claude
--references model_id --tasks 2 --concurrency 2 --restarts K1,K2 --seed
207` on `master`.
**Paperclip version or commit**
`master` at 82abb3e.
## What Changed
- `packages/paperclip-runner/src/drivers/acpx/provider-failure.ts`: for
the `claude` agent, an AIR `request` session failure whose title is
Claude Code's fixed "There's an issue with the selected model (…). It
may not exist or you may not have access to it." is classified
`model_unavailable` (`requires_action`). The native runtime already
marks such a failed turn `native_provider_model_rejected`, and the
server already projects that as `choose_model` with `rejectedModel`.
- Other AIR categories, other agents and other request titles stay
`unknown`.
- `server/src/services/native-runtime/native-session-executor.ts`: a
`native_provider_model_rejected` failure after the provider session
opened now gets a choose-model next action ("The provider rejected the
selected model. Choose a supported model for this agent, then retry the
task. Automatic retries are stopped."). Before, only a rejection before
the session opened got that action; the Claude case fell through to
"Inspect the original provider failure and explicitly resolve the
ambiguous session state; do not open a replacement provider session."
Recovery mode, blocking and retry behavior are unchanged; only the
action text changes.
## Verification
- Red before, green after: `src/drivers/provider-failure.test.ts` —
"classifies Claude's rejection of the selected model as an unavailable
model", with the exact title captured from `claude-agent-acp` 0.73.0,
through both `acpxTerminalResult` and the thrown form, plus negative
cases.
- Red before, green after: `native-startup-retry.integration.test.ts` —
"asks for a model choice when the provider rejects the model on an
opened session's turn (Claude ACPX)": a durable `turn.started` and the
`NativeProviderTerminalFailure` the runtime throws; the coordinator and
the board recovery action name the model choice, not ambiguous state.
- Live: on `master` the wave failed (see above). On this branch:
`w-20261110T103809-s208-cf` on head 3881650 (CF3 `model_id`, claude, K1
and K2): **2/2 pass**. Each task rests as `waiting_for_user` /
`choose_model` with `rejectedModel: "chaos-missing-model-…"`, cause
`native_provider_model_rejected`, next action "The provider rejected the
selected model. Choose a supported model for this agent, then retry the
task. Automatic retries are stopped.", rests again after the restart,
and is done after the model is restored and Retry. An earlier wave on
e552b24 (`w-20261110T100526-s207-cf`, 2/2 pass) still showed the
ambiguous-state text that the second commit replaces.
## Risks
- Low risk. The match is limited to the Claude agent, the AIR `request`
category and Claude Code's fixed title. If Claude Code changes the text,
the run falls back to today's generic failure.
- No profile, digest or pack contract change
(`rules/acpx-profile-identity.mdc`): ordinary Runner source.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs, a direct ACP probe of
`claude-agent-acp`).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
57def75d01
commit
8b3348c6ba
4 files changed
+76
-2
No files matched your search
@@ -48,6 +48,18 @@ function sessionLimitCategory(failure: AcpxTerminalSessionFailure | undefined):
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Claude Code reports a model it cannot run (unknown, retired, or not on the
|
||||
* account) as an AIR `request` failure with this fixed title. The user must
|
||||
* choose another model, so it is an unavailable model, not a generic request.
|
||||
*/
|
||||
const CLAUDE_SELECTED_MODEL_REJECTED =
|
||||
/^There's an issue with the selected model \([^()\n]{1,200}\)\. It may not exist or you may not have access to it\./;
|
||||
function sessionModelCategory(agent: string, failure: AcpxTerminalSessionFailure | undefined): ProviderFailure["category"] | undefined {
|
||||
return agent === "claude" && failure?.category === "request" && typeof failure.title === "string" &&
|
||||
CLAUDE_SELECTED_MODEL_REJECTED.test(failure.title.trim()) ? "model_unavailable" : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* `ownMessage` keeps an error's own message when it carries no ACP error. Use
|
||||
* it only where every rejection belongs to the provider: a failed terminal
|
||||
@@ -72,7 +84,8 @@ export function acpxProviderFailure(error: unknown, agent: string, phase: Provid
|
||||
const code = detail.type ?? detail.code ?? data.errorType ?? data.errorKind;
|
||||
const category = (typeof code === "string" ? CATEGORIES.get(code) : undefined)
|
||||
?? (acp?.code === ACP_AUTH_REQUIRED ? "authentication" : undefined)
|
||||
?? sessionLimitCategory(sessionFailure);
|
||||
?? sessionLimitCategory(sessionFailure)
|
||||
?? sessionModelCategory(agent, sessionFailure);
|
||||
return createProviderFailure({ provider: agent === "claude" ? "anthropic" : agent === "codex" ? "openai" : agent,
|
||||
phase, message, httpStatus: status, providerCode: code, requestId: data.request_id, category, environment });
|
||||
}
|
||||
|
||||
@@ -79,6 +79,23 @@ describe("provider failure boundary", () => {
|
||||
.toMatchObject({ category: "rate_limit", message: `${generic}\nClaude is temporarily rate limited.` });
|
||||
});
|
||||
|
||||
it("classifies Claude's rejection of the selected model as an unavailable model", () => {
|
||||
const generic = "ACP agent reported a terminal request failure.";
|
||||
const failed = { status: "failed", error: { code: "RUNTIME", message: generic } };
|
||||
// The AIR sessionFailure claude-agent-acp 0.73.0 returns for an unknown model (live probe, 2026-11-10).
|
||||
const title = "There's an issue with the selected model (chaos-missing-model-abc12345). It may not exist or you may not have access to it. Run --model to pick a different model.";
|
||||
expect(acpxTerminalResult(failed, "claude", {}, { category: "request", title })).toMatchObject({ status: "failed", error: { providerFailure: {
|
||||
provider: "anthropic", phase: "turn", category: "model_unavailable", retryability: "requires_action", message: `${generic}\n${title}` } } });
|
||||
// The thrown form of the same terminal is classified the same way.
|
||||
expect(acpxProviderFailure(new Error(generic), "claude", "turn", {}, false, { category: "request", title }))
|
||||
.toMatchObject({ category: "model_unavailable", retryability: "requires_action" });
|
||||
// Another category, another agent, or another request failure stays unknown.
|
||||
for (const [agent, category, text] of [["claude", "service", title], ["codex", "request", title],
|
||||
["claude", "request", "Claude could not process this request."]] as const) {
|
||||
expect(acpxTerminalResult(failed, agent, {}, { category, title: text })).toMatchObject({ error: { providerFailure: { category: "unknown" } } });
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the ACPX terminal cause when the raw result exceeds its payload limit", () => {
|
||||
const terminal = acpxTerminalResult({ status: "failed", error: { ...acp,
|
||||
data: { ...acp.data, responseBody: "private-body".repeat(7000) } } }, "claude");
|
||||
|
||||
@@ -8210,6 +8210,11 @@ const NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION =
|
||||
const NATIVE_STARTUP_MODEL_REJECTED_ACTION =
|
||||
"The provider could not start the selected model before any provider work began. Choose a supported model for this agent, then retry the task. Automatic retries are stopped.";
|
||||
|
||||
// The provider session opened, then the provider refused the selected model on
|
||||
// the turn. The task needs a model choice, not a reconciliation of session state.
|
||||
const NATIVE_TURN_MODEL_REJECTED_ACTION =
|
||||
"The provider rejected the selected model. Choose a supported model for this agent, then retry the task. Automatic retries are stopped.";
|
||||
|
||||
const NATIVE_CLEANUP_OPERATOR_RECOVERY_MESSAGE =
|
||||
"Send a new message to continue after Paperclip verifies that the previous provider and its tools have stopped. Clearing a task session does not resolve this quarantine. Automatic retries are stopped.";
|
||||
|
||||
@@ -11913,6 +11918,8 @@ async function executePaperclipNativeSessionAttempt(
|
||||
? "The work and waiting state are saved. Confirm shutdown of the retained execution and settle that same handoff without repeating provider work."
|
||||
: sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished
|
||||
? NATIVE_STARTUP_MODEL_REJECTED_ACTION
|
||||
: sourceFailureCode === "native_provider_model_rejected"
|
||||
? NATIVE_TURN_MODEL_REJECTED_ACTION
|
||||
: sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE
|
||||
? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION
|
||||
: sourceFailureCode === NATIVE_PROVIDER_SIDE_EFFECT_PENDING
|
||||
@@ -12170,6 +12177,8 @@ async function executePaperclipNativeSessionAttempt(
|
||||
? "Verify that the failed provider stopped and reconcile its action outcomes. A linked continuation can proceed only after these checks succeed."
|
||||
: sourceFailureCode === "native_provider_model_rejected" && !recoveryEvidence.providerSessionEstablished
|
||||
? NATIVE_STARTUP_MODEL_REJECTED_ACTION
|
||||
: sourceFailureCode === "native_provider_model_rejected"
|
||||
? NATIVE_TURN_MODEL_REJECTED_ACTION
|
||||
: sourceFailureCode === NATIVE_PROVIDER_STATE_INCOMPATIBLE
|
||||
? NATIVE_PROVIDER_STATE_INCOMPATIBLE_ACTION
|
||||
: sourceFailureCode === NATIVE_PROVIDER_SIDE_EFFECT_PENDING
|
||||
|
||||
@@ -33,7 +33,7 @@ import { executePaperclipNativeSession, prepareNativeSessionOpenTimeoutRetry, sc
|
||||
import { buildNativeCompletionContract } from "./completion-contracts.js";
|
||||
import { prepareNativeHeartbeatRun } from "./prepare-native-run.js";
|
||||
import { claimNativeSessionResumptions } from "./native-finalization-reconciler.js";
|
||||
import { DurablePrpControlPlane } from "../../vendor/paperclip-runner/index.js";
|
||||
import { DurablePrpControlPlane, NativeProviderTerminalFailure } from "../../vendor/paperclip-runner/index.js";
|
||||
import { createNativeRunnerExecutionDriver } from "./native-runner-execution-driver.js";
|
||||
import { currentNativeControllerIdentity } from "./native-restart-recovery.js";
|
||||
import { loadNativeRunnerCheckpoint } from "./native-runner-checkpoint.js";
|
||||
@@ -785,6 +785,41 @@ describe("native provider startup retry", () => {
|
||||
},
|
||||
);
|
||||
|
||||
it("asks for a model choice when the provider rejects the model on an opened session's turn (Claude ACPX)", async () => {
|
||||
const seeded = await seedRun(await createAgent());
|
||||
provider.execute.mockReset().mockImplementation(async () => {
|
||||
await db.insert(heartbeatRunEvents).values({
|
||||
companyId, runId: seeded.runId, agentId: seeded.agentId, seq: 900,
|
||||
eventType: "turn.started", stream: "system",
|
||||
});
|
||||
// What the native runtime throws for Claude's AIR "selected model" request failure.
|
||||
throw new NativeProviderTerminalFailure("RUNTIME", false,
|
||||
"native_provider_model_rejected: ACP agent reported a terminal request failure.");
|
||||
});
|
||||
await expect(executePaperclipNativeSession({
|
||||
db, execution: seeded.execution, runnerInstanceId: seeded.runnerInstanceId,
|
||||
backend: {
|
||||
descriptor: async () => { throw new Error("Unexpected provider descriptor call"); },
|
||||
openSession: async () => { throw new Error("Unexpected provider session call"); },
|
||||
},
|
||||
})).rejects.toThrow("native_provider_model_rejected");
|
||||
|
||||
const state = await read(seeded);
|
||||
expect(state.coordinator).toMatchObject({
|
||||
phase: "terminal_failure",
|
||||
failureCode: "native_provider_model_rejected",
|
||||
nextAttemptAt: null,
|
||||
failureDetail: expect.objectContaining({ providerSessionEstablished: true }),
|
||||
});
|
||||
expect(String(state.coordinator.failureDetail?.nextAction)).toContain("Choose a supported model");
|
||||
expect(String(state.coordinator.failureDetail?.nextAction)).not.toContain("ambiguous");
|
||||
expect(state.run).toMatchObject({ status: "failed", errorCode: "native_provider_model_rejected" });
|
||||
expect(state.issue.status).toBe("blocked");
|
||||
expect(state.action).toMatchObject({ ownerType: "board", cause: "native_provider_model_rejected", wakePolicy: null });
|
||||
expect(state.action!.nextAction).toContain("Choose a supported model");
|
||||
expect(state.action!.nextAction).not.toContain("ambiguous");
|
||||
});
|
||||
|
||||
it("blocks with a clear reason once the bounded startup budget is exhausted", async () => {
|
||||
const seeded = await seedRun(await createAgent());
|
||||
await failStartup(seeded, sessionOpenFailure());
|
||||
|
||||
Reference in new issue
Block a user