mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
DottaandClaude Opus 5.5
beaa9a622d
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs start a local `paperclip-runnerd` process. The control plane keeps a durable state root for each session. > - When the control plane restarts, restart recovery classifies each live native run. A run whose runner died before any provider identity existed gets the `bootstrap_incomplete` disposition. Recovery then quarantines the old state root and starts the bootstrap again. > - The executor accepts that disposition only when the durable state root proves that no provider work started. The old proof required `connectionCount === 0` and every command `pending`. > - A restart chaos wave killed the server about 2 s after runnerd started. The runner had connected and completed `run.prepare`, but `session.open` was still pending. No event was committed, so no provider session or turn existed. > - The proof rejected that root. The run failed with `runner_state_identity_mismatch: bootstrap_not_proven_incomplete`, and the task stopped with `native_continuation_requires_reconciliation`. Nothing continued automatically. > - This pull request widens the proof to the exact window where provider work cannot have started, and keeps every other shape fail-closed. > - The benefit is that a restart during native runner bootstrap continues automatically for every native provider. ## Linked Issues or Issue Description **What happened?** A control plane restart that lands after the native runner connected and acknowledged `run.prepare`, but before `session.open` completed, fails the run with `adapter_failed` and `runner_state_identity_mismatch: bootstrap_not_proven_incomplete`. Startup recovery chose `bootstrap_incomplete` (controller generations 2 and 3), but the executor proof in `migrateRunnerdStateRootForExecution` refused it. The durable control-plane state had `connectionCount: 1`, `run.prepare` completed, `session.open` pending, no committed events, and `ackedSourceSeq: 0`. The runner journal had `nextSourceSeq: 1`, an empty outbox, and the provider state was `prepared` with no provider session ID. **Steps to reproduce** 1. Start a native task (seen with OpenCode on a local runner; the window applies to every native provider). 2. Kill the control plane about 2 s after runnerd starts, after `run.prepare` completes and before `session.open` completes. 3. Let the control plane restart. **Expected behavior** Recovery quarantines the state root and starts the bootstrap again. The task continues without a user action. **Actual behavior** The run fails and the task is blocked with `native_continuation_requires_reconciliation`. ## What Changed - `runnerdStateIsIncompleteBootstrap` (native-session-executor.ts) now also accepts a connected root when all of these are true: no committed event, `ackedSourceSeq === 0`, only `run.prepare` and `session.open` commands, `session.open` not completed, `run.prepare` pending or completed, no warm transition or staged attachment, and the bootstrap was not adopted. - When `runner/runner-state.json` exists, the runner journal must agree: empty outbox, `nextSourceSeq === 1`, `ackedSourceSeq === 0`, and no completed `session.open` or other processed command. An unreadable runner journal stays fail-closed. This covers events that the runner emitted but the controller never committed. - The remote "claim an untouched session" path keeps the old never-connected proof (`runnerdStateIsUnconnectedBootstrap`), so its behavior does not change. - Tests: a red test with the production state shape (connected, `run.prepare` completed, `session.open` pending), plus fail-closed counter-tests for a completed `session.open`, committed or acknowledged events, a turn command, a failed `run.prepare`, an adopted bootstrap, a warm transition, an uncommitted runner event, a runner-completed `session.open`, a runner-processed turn, and a torn runner journal. ## Verification - Red before the fix: `pnpm exec vitest run src/services/native-runtime/native-session-executor.test.ts -t "native incomplete-bootstrap evidence"` failed on "allows a fresh bootstrap after connect and run.prepare when session.open never completed" (`expected false to be true`). Green after the fix: 13/13. - The proof run against the kept durable state root from the failing wave (read only) returns `true` after the fix. - Related suites: `native-session-executor.test.ts` and `native-restart-recovery.test.ts` filtered to `bootstrap|restart|remote|unstarted|untouched`: 301 passed. - `tsc --noEmit -p server/tsconfig.json` passes. - Live restart chaos proof: see the comment on this PR. ## Risks - The proof now accepts a root where the runner received `session.open` but did not complete it. The runner can have started a provider process or created an empty provider session in that window. Recovery only proceeds when restart classification already proved that the runner process and its group are dead and no known provider process is alive. No provider turn can exist, because the controller never issued a turn command. The worst case is an orphaned empty provider session, not repeated work. - Every state with committed or acknowledged events, a completed `session.open`, a turn, a warm transition, an adopted bootstrap, or an unreadable runner journal still fails closed. - Out of scope: when the proof fails legitimately, the task offers the existing execution-blocker Retry. In the failing wave the issue interruption also showed a budget wait (`userAction: budget`). That is the RC-06 accounting issue and belongs to a separate fix. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use (shell, file edits, test execution). Default context window. Extended reasoning was available. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>