mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Codex agents run through the Paperclip runner, and the server-side transport starts each provider turn > - After Codex accepts a turn, the transport waits a fixed 30 s for that turn's `turn/started` before it reports the turn identity > - Codex starts its MCP servers before it starts the turn, so a healthy cold start in a loaded sandbox often takes longer than 30 s > - The transport then failed a live turn with `runnerd did not report the provider turn identity`, and the abandoned turn's late notifications broke the reattachment (`did not name its execution owner`, `runner_harness_state_mismatch: resumed_not_suspended`) > - This pull request sizes the start window to the measured cold start and extends it while the provider keeps reporting startup progress > - The benefit is that slow but healthy Codex starts succeed on the first attempt, and a dead start still fails at a bound ## Linked Issues or Issue Description **What happened?** On staging, Codex runs on Daytona failed at turn start with `native_session_interrupted: runnerd did not report the provider turn identity`. The run log shows the same sequence each time. The runner accepts the turn. Codex then reports startup progress (a bubblewrap config warning, `account/updated`, and `mcpServer/startupStatus/updated` failures for an MCP server that is not logged in) for about 30 s. The transport gives up 30 s after acceptance. A few seconds later, Codex sends the turn's `turn/started`, which now arrives at a closed turn (`protocol_failure:turn_binding_mismatch`). The same-run reattachment then hits that orphaned turn. It fails with `Provider authoritative notification item/started did not name its execution owner.` or `runner_harness_state_mismatch: resumed_not_suspended`. Of 198 recent runs in the staging company, 52 contain this turn identity failure. In 68 successful starts, submit to `turn/started` took p50 25 s, p75 37 s, p90 50 s, p95 54 s and max 76 s. The fixed 30 s wait sits near the median. **Expected behavior** A slow but healthy Codex start succeeds. A start that makes no progress, whose Runner exits, or that the runner fails still fails at a bound and recovers through the normal same-run resume. **Steps to reproduce** 1. Start a Codex turn whose provider accepts the turn but sends `turn/started` after the 30 s default window (on staging: several MCP servers and a loaded sandbox). 2. Let the provider report startup notices during the wait, and let the runner not prove the accepted turn live in time (on staging no turn was adopted). 3. See `runnerd did not report the provider turn identity`, then the reattachment conflict. The new tests reproduce this with the fake Codex app server (`--turn-started-delay-ms` with `--turn-started-progress-ms`) and a runner snapshot that cannot prove liveness. **Paperclip version or commit** Staging Core `c730872974` (`2026.1106.0+2253`). Base of this PR: `origin/master`. **Deployment mode** Authenticated, public, managed cloud (staging). Remote runner on Daytona. **Agent adapter(s) involved** `paperclip_runner` with the Codex provider (`gpt-6.1-sol`). ## What Changed - `runnerd-codex-transport.ts`: the default wait for an accepted turn's `turn/started` is now 90 s (measured max 76 s), not 30 s. An explicit `turnStartTimeoutMs` keeps its old bound. - The wait restarts its window each time the runner commits a provider notice, or a runner diagnostic that names the accepted turn. The total wait stops at 180 s (option `turnStartProgressCeilingMs`, so tests can lower it). A provider relaunch after the accepted `turn.start` does not extend it. - The adoption check (runner snapshot proves the accepted turn live) now runs before each window end, not only before the first one. A Runner exit and a runner failure of the pending turn still fail at once. - `fake-codex-app-server`: new `--turn-started-progress-ms` flag sends `mcpServer/startupStatus/updated` failures while it delays `turn/started`. - `doc/execution-semantics.md`: documents the start window, the progress rule and the staging measurements. ## Verification - Red on `origin/master` transport (with the new fake flag): `keeps waiting for an accepted Codex turn while its startup keeps reporting progress` failed after 4.6 s, and `gives a silent Codex cold start the measured default start window` failed after 30.9 s. Both failed with `runnerd did not report the provider turn identity`. - Green: `npx vitest run src/live/runnerd-codex-transport-deadlines.test.ts` passes 9/9. This includes a ceiling test: provider notices every 1 s and no `turn/started` fail at an 8 s test ceiling, and the test goes red with the ceiling cap removed. It also includes the existing adoption, no-adoption-within-bound and runner-failure tests, and a new test that a silent start still fails within its bound. - `npx vitest run` on the adoption, cut-turn-start, stall, turn-start-timeout-retention, remote-retention, restart-continuation and controller-outage transport suites passes 48/48. - `npx vitest run src/live/runnerd-codex-transport.test.ts src/live/live-session.test.ts src/live/turn-stream.test.ts`: 391/392 pass. The one failure (`stops a lost runnerd's provider and escaped tool job from its lineage journal (journal=true)`, 1 s, process-lineage timing under full-suite load) passes when run alone, both on this branch and on `origin/master`. - `tsc --noEmit` for `packages/paperclip-runner` and `cargo fmt --check` pass. ## Risks - A silent dead start (no Runner exit, no provider output, no runner failure) now fails after 90 s, not 30 s. The Runner exit and runner-failure paths still fail at once, so this applies only to a provider that stops without a trace. - A provider that keeps sending notices but never starts the turn waits at most 180 s. - No schema, API or UI change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, browser reads of staging run data). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge Refs #21272, #21120 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>