mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 15:55:46 +01:00
DottaandClaude
82abb3ea04
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner agents run on a provider (here Codex) with a model the
user saves on the agent
> - When the provider rejects that model, Paperclip should classify the
run as `native_provider_model_rejected`, ask the user to choose a model,
and let Retry finish the task
> - Paperclip recognized only the ChatGPT-account rejection (a 400 JSON
envelope). With API-key auth, Codex returns a different terminal error
for an unknown model
> - So the run failed as a generic `adapter_failed` ("Native session
failed"), and recovery opened a reconciliation incident ("Recovery
needed") for a turn that never did any work
> - This pull request recognizes the real API-key shape, bound to the
configured model, and names the rejected model on the task
> - The benefit is that the task asks for a model choice instead of a
reconciliation, and Retry completes the task after the user picks a
valid model
## Linked Issues or Issue Description
**What happened?**
The restart-chaos config fault CF3 `model_id` saves a Codex agent with
the model `chaos-missing-model-<nonce>`. The run stays "running" for
about 60 seconds, then fails with `errorCode: adapter_failed`, "Native
session failed". The task shows "Recovery needed", cause
`native_continuation_requires_reconciliation`. The grader fails as
`CF3/model_id:actionable_message` (waves `w-20261110T061914-s206-cf` t01
and `w-20261110T053831-s205-cf` t02).
Root cause: the wave runs Codex with an OpenAI API key. I drove the
bundled `codex app-server` (0.160.0) directly with an unknown model:
- With API-key auth, Codex retries the Responses API 404 ten times
(about 59 s), then fails the turn with `{"message":"unexpected status
404 Not Found: The model \`<id>\` does not exist or you do not have
access to it., url: https://api.openai.com/v1/responses, cf-ray: …,
request id:
req_…","codexErrorInfo":{"httpConnectionFailed":{"httpStatusCode":404}}}`.
- With ChatGPT auth, the turn fails in about 3 s with `message` =
`{"type":"error","status":400,"error":{"type":"invalid_request_error","message":"The
'<id>' model is not supported when using Codex with a ChatGPT
account.","code":"model_not_found"}}`. Paperclip already classified this
one.
`createNativeProviderFailureObservation` parsed only the JSON envelope,
and the runner's provider-failure category for an HTTP 404 is `unknown`.
The terminal therefore stayed unclassified.
**Expected behavior**
The run is `native_provider_model_rejected`, the task shows `userAction:
"choose_model"` and names the model, no reconciliation incident opens,
and Retry completes the task after the user picks a valid model.
**Steps to reproduce**
`node cli/node_modules/tsx/dist/cli.mjs
tests/restart-chaos/config-wave.ts --faults CF3 --adapters codex
--references model_id --tasks 1 --concurrency 1 --restarts K2 --seed
206` on `master` (with `OPENAI_API_KEY` set).
**Paperclip version or commit**
`master` at 5ab98b0.
## What Changed
- `server/src/services/native-runtime/native-provider-failure.ts`: the
Codex observation recognizes the API-key terminal shape
(`codexErrorInfo` `httpConnectionFailed` or `responseStreamDisconnected`
with `httpStatusCode: 404`, and the message prefix for the configured
model only). It classifies it as `native_provider_model_rejected` with a
new closed diagnostic `NATIVE_MODEL_NOT_FOUND_DIAGNOSTIC` (`codex`,
`model_not_found`, `404`, `api_key`). No provider text or model name is
stored in the diagnostic. A recoverable turn, another model's 404, or
another status is not classified.
- `readNativeModelRejectionDiagnostic` accepts both closed diagnostics,
so run failure diagnostics show it.
- `TaskInterruptionPresentation.rejectedModel` (shared type): with
`userAction: "choose_model"`, the task query carries the model id from
the run's native execution input. Only a
`native_provider_model_rejected` run discloses it.
- `TaskChatThread` names the model in the "Model unavailable" marker:
`The provider rejected the model "<id>".`
- The restart-chaos config wave reads `interruption.rejectedModel` as
task-level text. For `choose_model`, its repair step uses the Retry that
the "Model unavailable" marker offers (`POST /agents/{id}/wakeup` with
`reason: "retry_failed_run"`, the same request as
`agentsApi.retryFailedRun`), because a model rejection does not offer
the task-level Retry.
## Verification
- Red before, green after (the classification line disabled makes both
fail):
- `native-provider-failure.test.ts` — "classifies the API-key 404 for
the configured model as a model rejection (codex 0.160.0 shape)", with
the exact terminal error captured from `codex app-server` 0.160.0, plus
negative cases.
- `native-provider-failure.integration.test.ts` — new `api-key` scenario
through `executePaperclipNativeSession`, the persisted evidence reader
and `finalizeNativeRun`: the run ends `native_provider_model_rejected`,
the issue is `blocked` with that decision, and no issue recovery action
is created.
- `task-interruption-query.test.ts` — the real task query carries
`rejectedModel` only for a model rejection. `TaskChatThread.test.tsx` —
the marker names the model.
- `pnpm --filter @paperclipai/shared typecheck`, `pnpm --filter
@paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui
typecheck` pass.
- Live proof: config wave `w-20261110T090801-s206-cf` (CF3 `model_id`,
codex with an OpenAI API key, K2, seed 206) on commit 05e8938: **1/1
pass**. The run ends `native_provider_model_rejected`; the task rests as
`waiting_for_user` / `choose_model` with `rejectedModel:
"chaos-missing-model-472e1851"` and execution cause
`native_provider_model_rejected` (no reconciliation incident), rests
again after the K2 restart, and finishes 27 s after the model is
restored and the marker's Retry is used. On `master` the same wave
failed as `actionable_message` with "Recovery needed" /
`native_continuation_requires_reconciliation`
(`w-20261110T061914-s206-cf`).
## Risks
- Low risk. The match is bound to the configured model, a 404 status and
a non-recoverable turn, so other provider failures keep their current
behavior.
- The match depends on the Codex terminal text. If a later Codex release
changes it, the run falls back to the old generic failure, not to a
wrong classification.
- Claude (ACPX) unknown models are not changed here. Claude Code reports
an unknown model as a synthetic assistant message ("There's an issue
with the selected model …") rather than a provider error; this PR does
not change or live-verify that path.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test runs, a direct `codex
app-server` probe).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>