fix(native): surface model capacity errors and retry automatically (#15347)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs preserve provider events and results, then decide task
state.
> - Codex can fail a turn because the selected model is temporarily at
capacity.
> - Paperclip displayed a generic native-session failure and left the
task in recovery.
> - A known capacity failure needs a clear message and a durable delayed
retry.
> - This pull request uses committed terminal evidence, the
status-effect ledger, and existing dispatch gates.
> - The task can continue automatically without an unbounded retry loop
or a model switch.

## Linked Issues or Issue Description

Related: #13993 adds launcher capacity deferrals before provider
startup. This change handles a committed native Codex `serverOverloaded`
terminal after provider work has started.

**What happened?**
A native Codex turn ended with `codexErrorInfo: serverOverloaded` and
“Selected model is at capacity. Please try a different model.” Paperclip
saved the result, showed a generic failure, and required recovery
instead of waiting and retrying.

**Expected behavior**
Show the capacity error directly. Schedule bounded retries after a short
delay. Preserve saved work and honor current execution gates.

**Steps to reproduce**
1. Run a native Codex task.
2. Commit a `turn.failed` event with `serverOverloaded`, then accept a
failed result for the same turn.
3. Inspect the task error and its recovery state. The regression test
reproduces this without a paid provider call.

**Paperclip version or commit**
Reproduced against master `16b7db35ffa0f9a95913c8cbdeea3d595435691f`.

## What Changed

- Classify capacity failures from committed runner events and the pinned
execution identity. Preserve accepted results and display the specific
capacity error.
- Atomically persist one scheduled successor with the status decision.
Retry after one minute, then two minutes. Share the existing failure
budget and stop after two automatic retries.
- Reuse dispatch gates for ownership, task holds, dependencies, budget,
and locks. Wait for predecessor execution, finalization, and cleanup
before claiming a retry.
- Preserve pending reviewer authority and suppress retries after
reassignment or a successor claim.
- Consume the failed run's resume receipt and delivered wake input.
Rebuild ordinary continuation from the failed run so explicitly resumed
tasks can retry without borrowing one-run authorization.
- Label scheduled retries “Model at capacity.” Add a Storybook example
and avoid duplicate punctuation in the existing retry card.
- Add regression coverage and document the runtime contract.

## Verification

- Targeted recovery and UI suites: 125 tests passed. Additional final
cleanup and lock regressions passed.
- Latest-head continuation and authorization regressions: 57 tests
passed, including all four capacity integration tests against an
isolated PostgreSQL database.
- Rechecked all four capacity integration tests with the full runner's
isolated `PAPERCLIP_HOME`, config, temporary directory, and serial fork
settings: passed.
- `pnpm -r typecheck`: passed. Final server typecheck passed.
- `pnpm build`: passed.
- `pnpm check:token-gates`: passed.
- Browser: checked the real Storybook card. It shows the capacity
message, automatic retry time, and existing Retry now action.
- `pnpm test:run`: attempted and restarted after an interruption. The
resumed run started before the final review correction and was stopped
after recorded workspace/native test failures and the five-minute Git
streaming timeout already documented on master. Exit 130; no complete
local full-suite pass is claimed. Current-head isolated capacity tests
and the complete CI suite pass.
- A combined local recovery-suite attempt also hit PostgreSQL
initialization failures at this macOS host's global shared-memory limit
(32 slots). The focused recovery suites passed separately; final-head
capacity tests also pass with the full runner's isolated environment
settings.
- Latest-head GitHub checks: all 56 checks green, including general and
serialized test shards, browser E2E, typecheck, build, Runner
verification, and canary dry run. No merge conflicts.
- Greptile: 5/5 on `813a470b8c18c05aeb7e31e63e573c3a3a2f5cac`, with zero
unresolved findings after fixing the resumed-task continuation issue.

## Risks

- Capacity retries can repeat a task turn after partial work. They start
a fresh provider session with task history and wait for predecessor
cleanup. They do not resume the failed turn.
- Retries retain the configured model unless an operator changes
configuration. Persistent overload consumes the existing failure budget
and then needs an explicit retry or model change.
- Only run-bound native Codex `serverOverloaded` failures qualify.
Usage-limit exhaustion, model/account incompatibility, unbound text, and
unknown provider failures retain their current recovery behavior.
- No schema migration is required.

## Model Used

- OpenAI GPT-6 through Codex. The exact model ID and context-window size
are not exposed to this session. Used reasoning, repository tools, code
execution, and browser verification. No subagents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

server/src/services/native-runtime/native-run-finalizer.ts

@@ -1,5 +1,6 @@
 import { settleSlackConversation } from "../slack-conversation-lifecycle.js";
-import { readPersistedNativeModelRejection } from "./native-provider-failure-evidence.js";
+import { executionFailureRetryCount } from "../execution-recovery-attempt.js";
+import { readPersistedNativeProviderFailure } from "./native-provider-failure-evidence.js";
 import { dismissAutomaticCompletionReviews } from "./automatic-completion-reviews.js";
 import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-review-participant.js";
 import { conversationNativeDecision, isConversation } from "../agent-conversations.js";
@@ -1263,9 +1264,9 @@ export async function finalizeNativeRun(input: {
         issueId: authoritativeIssue.id,
         runId: run.id,
       }),
-      assessment.reportedDisposition === "yielded" &&
+      (terminalState === "failed" || (assessment.reportedDisposition === "yielded" &&
       assessment.continuation?.kind === "response_wake" &&
-      assessment.hasBlockingRemainingWork
+      assessment.hasBlockingRemainingWork))
         ? issueTreeControlService(input.db).getActivePauseHoldGate(
             run.companyId, authoritativeIssue.id,
           )
@@ -1276,15 +1277,18 @@ export async function finalizeNativeRun(input: {
       companyId: run.companyId, issueId: authoritativeIssue.id, agentId: run.agentId,
       contextSnapshot: reviewContext, allowResolvedByRunId: run.id,
     }) : null;
-    const providerModelRejection = await readPersistedNativeModelRejection(
+    const providerFailure = await readPersistedNativeProviderFailure(
       input.db, run, resultRow.turnId, envelope.terminal as PrpTerminalState,
     );
-    const ownsModelRejectionDecision =
+    const ownsProviderFailureDecision =
       (!authoritativeIssue.executionRunId || authoritativeIssue.executionRunId === run.id) &&
       (reviewContext ? nativeReview?.interaction.status === "pending"
         : authoritativeIssue.assigneeAgentId === run.agentId && !authoritativeIssue.assigneeUserId);
     const proposedDecision = resolveNativeFinalizerStatus({
-      providerModelRejected: providerModelRejection !== null && ownsModelRejectionDecision,
+      providerModelRejected: providerFailure?.errorCode === "native_provider_model_rejected" && ownsProviderFailureDecision,
+      providerOverloaded: providerFailure?.errorCode === "native_provider_overloaded" && ownsProviderFailureDecision,
+      providerFailureSuperseded: providerFailure?.errorCode === "native_provider_overloaded" && !ownsProviderFailureDecision,
+      failureRetryCount: executionFailureRetryCount(run),
       ...(reviewContext ? { nativeReviewOutcome: nativeReview
         ? nativeReview.interaction.status === "pending" ? "pending" as const : "resolved" as const
         : "stale" as const } : {}),
@@ -1370,6 +1374,8 @@ export async function finalizeNativeRun(input: {
         requireModelRejectionOwner: decision.reasonCode === "native_provider_model_rejected"
           ? { agentId: run.agentId, reviewContext }
           : undefined,
+        requireProviderFailureOwner: decision.reasonCode?.startsWith("native_provider_overloaded")
+          ? { agentId: run.agentId, reviewContext } : undefined,
         requireBoardResponseWaitSource:
           decision.reasonCode === "board_response_waiting" || repairBoardResponseWait
             ? boardResponseWait?.source
@@ -1412,7 +1418,7 @@ export async function finalizeNativeRun(input: {
       );
       const clearExecutionFailure = input.projectRunStatus && finalizationPhase === "committed" && terminalState === "succeeded";
       const finalizationMetadata = {
-        ...(providerModelRejection ? { nativeProviderFailure: providerModelRejection.diagnostic } : {}),
+        ...(providerFailure ? { nativeProviderFailure: providerFailure.diagnostic } : {}),
         finalizationPhase,
         ...(finalizationPhase === "committed" ? { failureCode: null, originalFailureCode: null, nextAttemptAt: null } : {}),
         assessmentId: assessmentRow.id,
@@ -1463,11 +1469,11 @@ export async function finalizeNativeRun(input: {
           nativePhase: finalizationPhase,
           nativePhaseUpdatedAt: now,
           ...(clearExecutionFailure ? { error: null, errorCode: null } : {}),
-          ...(providerModelRejection ? {
+          ...(providerFailure ? {
             // Recovery can finalize before heartbeat saves the adapter result.
             // Retain any independently recorded execution/cleanup failure.
-            error: sql`coalesce(${heartbeatRuns.error}, ${providerModelRejection.errorMessage})`,
-            errorCode: sql`coalesce(${heartbeatRuns.errorCode}, ${providerModelRejection.errorCode})`,
+            error: sql`coalesce(${heartbeatRuns.error}, ${providerFailure.errorMessage})`,
+            errorCode: sql`coalesce(${heartbeatRuns.errorCode}, ${providerFailure.errorCode})`,
           } : {}),
           resultJson: sql`coalesce(${heartbeatRuns.resultJson}, '{}'::jsonb)
             || ${JSON.stringify(finalizationMetadata)}::jsonb