mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 14:19:00 +01:00
DottaandClaude Opus 5.5
c55af26821
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native Codex run must survive a control-plane restart and continue its work > - #20057 made recovery adopt an in-flight turn, or start a new turn when the restored runner reports the old turn lost > - One case was not covered. A graceful drain can detach the controller after the runner accepted `turn.start` but before the provider started the turn > - The runner then records the provider's `turn.started` while no controller is attached. After a restart it replays that start, and the Codex session rejects it as an unexpected turn > - This pull request treats that replayed start as history from the earlier generation > - The benefit is that the run starts its continuation turn instead of stopping with `native_event_replay_conflict` ## Linked Issues or Issue Description Refs #20057 **What happened?** A restart chaos wave ended with one failed task (t48, Codex). A graceful SIGTERM suspended the controller 300 ms after the runner accepted `turn.start`. The provider's `turn.started` came after the controller detached. Recovery classified the run as `resume_dead_runner`. The restored runner reported the turn as `provider_turn_lost_on_restore`, which is correct. But it also replayed the undelivered `turn.started`. The Codex session failed with "Provider started an unexpected turn." (`turn_binding_mismatch`), and recovery stopped with `native_event_replay_conflict`. **Expected behavior** Recovery starts a new continuation turn. The replayed start of the lost turn is recorded as history, and the run completes. **Steps to reproduce** 1. Start a Codex native run. 2. Detach the controller gracefully after the runner accepts `turn.start` and before the provider emits `turn/started`. 3. Let the runner record `turn.started` while it is unattached, then kill the runner and the provider. 4. Restore the run. Before this change, the session fails with `turn_binding_mismatch`. **Paperclip version or commit** master ata9cf4ee731(the wave ran on66a2cbab98). ## What Changed - `codex-app-server-driver-impl.ts`: recovery identifies the turn that this run's latest `turn.start` had the runner accept, when recovery did not adopt it and provider history does not show it live. Recovery passes it to the session as `lostAcceptedTurnId`. - `codex-session-state.ts`: the session keeps `lostAcceptedTurnId` as an unconfirmed lost turn. The stale-event rule treats its replayed start and items as history. - `codex-session-notifications.ts`: `turn/started` applies the stale-event rule to earlier-generation and unconfirmed lost turns. The runner's restart-loss terminal confirms the lost turn. Any other terminal, or a semantic result, from that turn fails closed, so a real outcome is never dropped. A start for a turn that this generation settled, or for a turn the session never saw, still fails closed. - `runnerd-restart-continuation.test.ts`: a new connected test with the real runner and the fake Codex server reproduces the t48 shape. A new fake option, `--turn-started-gate`, holds the provider's start until the test detaches the controller, so the test does not depend on timing. - `codex-app-server-driver.terminal-replay.test.ts`: driver tests for the replayed start (history), and for an ordinary failure or a result from the lost turn (fail closed). ## Verification - Red: before the fix, the new test failed with `NativeProviderTerminalFailure: Provider started an unexpected turn.` at `codex-harness-session.ts:211`. This is the same error and stack as the chaos run. - Green: `npx vitest run src/drivers/codex src/live/runnerd-restart-continuation.test.ts` (in `packages/paperclip-runner`) passed 392 of 392. - The wider Codex, restart and recovery test files (45 files, 1153 tests) passed on the final revision. The two fail-closed driver tests fail on the first revision, where the lost turn's outcome was dropped as history. The one failure in an earlier run ("still rejects history for a turn that became terminal in the live session") showed that the first version was too broad. The rule now applies only to earlier-generation turns, and that test passes. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `packages/paperclip-runner` passed. ## Risks - Low risk. The change only applies after a restore, to a turn that the runner accepted for this run's latest `turn.start`, and only when recovery did not adopt that turn and provider history does not show it in progress. - A turn that the session never saw still fails closed. A re-start of a turn this generation settled still fails closed. A real failure or a result from the lost turn fails closed instead of being dropped. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>