mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
DottaandClaude Opus 5.5
07ddf9a5c2
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs persist an accepted result first. The Runner then closes and suspends its session. > - Under heavy host load, the Runner close can fail to prove a durable suspension after the result is accepted. > - The executor then kept its finalization lease. The heartbeat finalizer got `native_finalization_lease_busy`, and the run ended `failed/adapter_failed`. > - The provider work was complete. Only Runner bookkeeping failed, so the run must not fail. > - This pull request hands the exact lease to finalization and completes the run from its accepted result. > - The benefit is that an accepted result stays accepted, independent of host load. ## Linked Issues or Issue Description **What happened?** A native ACPX Claude run asked a structured question. The control plane accepted its result (`Waiting for…`) and recorded `run.terminal` with `runTerminalState=succeeded`. The Runner close then took about 8.6 s. Its `runner.suspend` command ended `failed`, but the durable Runner state was `suspended` with a matching identity. The close threw `NativeSessionCloseUnrecoverableError`. The executor emitted `native.post_result_failure` (`category: runner_settlement_incomplete`, `resultAlreadyAccepted: true`) and rethrew `native_session_lease_lost`. Its finalization lease stayed held (its TTL is 20 minutes). `finalizeNativeRun` got `native_finalization_lease_busy`, and the run ended `failed/adapter_failed`. No restart was in progress. Host load average was about 40. **Expected behavior** After the control plane accepts a result, finalization bookkeeping must not fail the run. The run settles from its accepted result. Repair of the Runner checkpoint stays with the existing next-run admission rules. **Steps to reproduce** Run `pnpm exec vitest run src/services/heartbeat/runtime-execution.test.ts -t "accepted local result"` in `server/`. The test uses the real executor, the real control-plane result commit, and embedded PostgreSQL. The provider boundary accepts a result and then throws `NativeSessionCloseUnrecoverableError`. On the base revision, the finalization row keeps the executor lease (`phase: workspace_finalizing`) and the finalizer reports `lease_busy`. **Paperclip version or commit** Base `ff1d1dea5d`. The incident was on `819420a60a`. ## What Changed - `native-session-executor.ts`: after the rejected failure write proves an accepted result under the exact, unexpired lease, the executor releases that lease to finalization. It uses the same exact-authority predicates as the normal release. Before, the lease stayed held until its TTL expired. - `runtime-execution.ts`: for `runner_settlement_incomplete` on a local workspace (no workspace sync and no remote workspace authority), Paperclip records `workspace_finalize` as succeeded, finalizes the accepted result, and returns it to the ordinary completion path. - New `native-accepted-result-settlement.ts`: rebuilds the adapter result from the durable accepted result row, its terminal, the run's native session, and the accepted usage checkpoint. The result carries the closed `nativePostResultFailure` diagnostic. It contains no provider text. - Other post-result categories (for example `io_not_found`) keep their workspace-failure handling. They now also reach finalization at once, instead of after the lease TTL. - `doc/execution-semantics.md`: describes the settlement. ## Verification - Red, then green: `settles an accepted local result when runner close cannot prove suspension` and `keeps workspace failure handling for other accepted-result failures on a local workspace` in `server/src/services/heartbeat/runtime-execution.test.ts`. Both failed on the base revision with the executor lease still held. Both pass with this change. - Safety case `settles an accepted local result ... (workspace-check-failed)`: when the `workspace_finalize` check fails, the run keeps the existing failure path. Case `terminal-unreadable`: when the accepted result cannot be rebuilt, only a failed `workspace_finalize` record is written. - `pnpm exec vitest run src/services/heartbeat/runtime-execution.test.ts`: all pass (some cases time out at 30 s only under host load above 40; they pass on a narrow rerun). This includes the 14 existing post-result diagnostic and authority cases (expired, replaced owner, generation, attempt, boot, pid, process start, stopped, lock busy, and others). - `pnpm --filter @paperclipai/server typecheck`: pass. - Live cell `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` (real ACPX Claude, local runner): 3 runs at host load average about 40. Runs 1 and 2 passed. Run 3 hit the fault naturally: the first run emitted `native.post_result_failure` (`runner_settlement_incomplete`) and now ended `run succeeded` (before this change it ended `failed/adapter_failed`). The cell then failed later, in the resume run after the answer: `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate; retirement_refused=target_not_sandbox_runner`. That is a separate follow-up (see Risks). ## Risks - A changed lease handoff. The executor releases the lease only after the rejected failure write proves an accepted result under the exact owner, attempt, controller generation, controller identity, and an unexpired lease. A stale or replaced owner does not release anything. A failed release is bounded and best effort. The old behavior then applies. - The run now completes as `succeeded` while the local Runner checkpoint is not proven suspended. The next run on the same session then meets the existing fence: the continuity-break retirement (`retireFinishedPerTurnRunnerdOwner`) accepts only sandbox Runners, so a local Runner is refused with `target_not_sandbox_runner` and its state root is quarantined. This PR does not change that fence. Extending the retirement to local Runners needs its own proof that the local Runner process stopped, and is a follow-up. - Settlement happens only when the accepted result and its terminal are rebuilt first, and the `workspace_finalize` record then succeeds. A failed branch or plugin workspace check keeps the failure path. - Synced and remote workspaces are not changed. They keep their copy-back retry path. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs). Extended reasoning was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #19126 added the closed post-result diagnostic that this change uses. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>