mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-20 15:09:57 +02:00
275189d8d6
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs use the Paperclip runner to drive a provider session, for example the Codex app server > - When a Codex turn fails, two classifiers look for known failures: a model rejection and a server capacity failure > - When no classifier matches, the run fails with the generic message `Native session failed` and the error code `adapter_failed` > - The run failure report to Sentry then has no data about the cause, because the server drops the Codex `codexErrorInfo` value > - This pull request records a closed-vocabulary diagnostic for that failed turn and sends it to the Sentry `provider_failure` context > - The benefit is that operators can see which Codex failure kind causes these generic failures, and then write a targeted fix ## Linked Issues or Issue Description **What happened?** Hosted staging reports many run failures with the title `Error: Native session failed`. The events have `agent_adapter=paperclip_runner`, `error_code=adapter_failed`, `driverKind=codex_app_server`, and `exitCode=1`. The events do not show why the Codex turn failed. **Expected behavior** A run failure report for a failed Codex turn shows the failure kind, so that an operator can find the cause. The report must not contain provider response text, prompts, or model names. **Steps to reproduce** 1. Start a native Codex run. 2. Make the Codex app server end the turn with `status: "failed"` and an error that is not a model rejection or `serverOverloaded`. For example, use `codexErrorInfo: "contextWindowExceeded"`. 3. Look at the Sentry event for the run failure. Before this change, the `provider_failure` context is empty. **Agent adapter(s) involved** Codex (native runner, `codex_app_server` driver). ## What Changed - `native-provider-failure.ts`: The provider failure observation now records a diagnostic for each failed Codex turn. The diagnostic has only closed values: `provider: "codex"`, `category: "turn_failed"`, a known `CodexErrorInfo` kind (or `unrecognized` / `absent`), an HTTP status from 100 to 599 for the HTTP variants, and a runner failure code from an explicit allowlist of the codes the runner itself writes on a failed Codex turn (any other string becomes `unrecognized`, because a provider error object can carry its own `code`). It never copies the provider message. - New `unclassifiedTurnFailure()` gives this diagnostic only for a failed terminal turn that `forTerminal()` does not classify. `forTerminal()`, error codes, error messages, retries, and finalizer decisions do not change. - New `readNativeTurnFailureDiagnostic()` rebuilds a persisted value from the closed vocabulary and rejects all other values. - `native-session-executor.ts`: The adapter result keeps the generic message. It adds `resultJson.nativeTurnFailure` when the turn failure is unclassified. - `run-failure-diagnostics.ts`: The run failure report copies the validated diagnostic into the `provider` context. Sentry sends this as `provider_failure`. - Tests: unit tests for the vocabulary, HTTP status bounds, unknown kinds, runner codes, and privacy; a diagnostics test; and an update to the end-to-end executor test for the `unknown-error` scenario. ## Verification - `pnpm exec vitest run src/services/native-runtime/native-provider-failure src/services/__tests__/run-failure-diagnostics.test.ts src/services/__tests__/run-failure-report*.test.ts src/__tests__/sentry*.test.ts` (in `server/`): 6 files, 432 passed, 5 skipped. - `pnpm exec vitest run src/services/native-runtime/native-session-executor.test.ts` (in `server/`): 698 passed. - I could not run the full `tsc --noEmit` in my sandbox: the memory limit stopped the process, and the runner vendor build needs `cargo`. CI runs the full typecheck. - To check by hand: make a Codex turn fail with an unclassified error. Then make sure that the Sentry event `provider_failure` context shows `category: turn_failed` and the `codexErrorInfo` kind, and no provider text. ## Risks - Low risk. The change adds diagnostic data only. Run status, error code, error message, retry, and finalizer logic do not change. - Privacy: the diagnostic uses an allowlist of Codex enum names, a bounded integer, and an allowlist of runner-issued failure codes. Tests make sure that provider text does not get through. - If Codex adds a new `CodexErrorInfo` kind, the diagnostic shows `unrecognized` until the allowlist gets the new kind. - This change does not fix the root cause. It gives the data that is necessary to find the cause. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #15347 (capacity classifier), #15304 (model rejection diagnostics). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>