Files
PaperClipAI/packages
DottaandClaude Opus 5.5 3f2a940d5b fix(runner): answer held Paperclip tool calls after a local restart before the next turn (#20227)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - When a restart interrupts a turn, the Runner keeps each Paperclip
tool call the model made as "held" until Paperclip answers it, and it
refuses to start the next turn while any call is held
> - Paperclip answers held calls after a reattach through its tool
recovery path. That path was wired only for remote runners
> - In restart chaos wave `w-20261109T084023-s9403`, t101 (local codex,
K2 then K1) held a `read_document` call that nothing answered, so the
continuation's `turn.start` failed and the task needed a Retry
> - This pull request wires held-call recovery for local runners too,
and makes `turn.start` answer or wait for held calls first
> - The benefit is that a turn lost to a restart no longer strands the
run behind one unanswered call

## Linked Issues or Issue Description

Refs #19831. Restart chaos wave `w-20261109T084023-s9403` (master
`fc5c79a327`), task t101 (codex C1, K2, local).

**What happened:**
- Run `4b5810d5` called `read_document` (`exec-ac8fa9b4`) around the K2
drain at 09:08:18. A K1 then killed the next server start.
- At 09:08:36 the third controller reattached. The Runner restored the
conversation and reported the old turn lost
(`provider_turn_lost_on_restore`, recoverable). The held call stayed
pending, by design: a held call accepts a late result.
- The controller had the call: the quarantine detail lists
`exec-ac8fa9b4` under `semanticTools.pending`. But
`recoverPendingSemanticInputs()` returned at once: for a local run
`controllerPersistence` is unset, so `recoverSemanticToolInput` was
`undefined`.
- At 09:08:46 the continuation's `turn.start` failed with "cannot
prepare the next provider turn while tool calls are active (pending: 1
[read_document:exec-ac8fa9b4…])". The run ended
`native_session_interrupted`.

## What Changed

- `server/src/services/native-runtime/native-session-executor.ts`: the
runnerd transport now always gets `recoverSemanticToolInput`, not only
when `controllerPersistence` (remote) is set. It uses the same epoch
check and `SessionAuthorityEpoch.recover`. Recovery re-authorizes and
re-runs reads, and replays receipts for mutations; it never repeats an
uncertain mutation.
- `packages/paperclip-runner/src/live/runnerd-codex-transport.ts`
(`#startTurnWork`): when the control plane has unsettled semantic
inputs, it runs `recoverPendingSemanticInputs()` before queueing
`turn.start`, and waits for that pass, bounded by the turn-start
deadline. Recovered results are queued as commands ahead of
`turn.start`, and the Runner applies commands in order. If recovery has
not finished by the deadline, the turn start fails with
`native_turn_start_held_tool_calls_unresolved` instead of sending
`turn.start` behind it. When everything is settled there is no wait, so
a normal turn start is unchanged. Live calls of a turn being replaced
are not waited on; the replaced start still rejects them.
-
`packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts`:
`recoverPendingSemanticInputs()` passes are serialized. A call made
while the startup pass is still answering a held call waits for that
pass, then runs its own, so the turn-start gate cannot finish early by
skipping a call that is already being recovered.

## Verification

Red/green:
- `native-session-executor.test.ts` "answers a local run's held
Paperclip tool calls after a restart (restart chaos t101)". On master
`recoverSemanticToolInput` is `undefined` for a local run; with this PR
it is wired.
- `runnerd-codex-transport.test.ts` "answers a held call before starting
the next turn (restart chaos t101)", against real runnerd and fake
Codex, with a slow startup recovery pass already in flight when
`turn/start` arrives. On master `turn/start` never ran recovery, and
without the serialization `turn.start` is queued before the held call's
result. With this PR the held call is recovered once and its result is
queued before `turn.start`.

Regression and static checks:
- `runnerd-codex-transport.test.ts`,
`durable-prp-control-plane.test.ts`,
`runnerd-codex-transport-adoption.test.ts`: 510 passed.
- `native-session-executor.test.ts` and `native-session-resume.test.ts`:
1368 passed, 1 failed. The failing case, "archives a damaged prior epoch
… with real runnerd", fails identically on master on this Mac (`ENOENT …
authority-epochs`), so CI is its reference.
- `tsc --noEmit` passes for `packages/paperclip-runner` and `server`.

## Risks

- Local runs now recover held calls after a reattach, as remote runs
already did. Reads re-run under current authority. A mutation without a
saved receipt still returns no result and stays held, as before.
- `turn.start` may now wait up to its existing deadline while a recovery
pass runs, then fails clearly if it has not finished. It never waits
when nothing is unsettled, so normal turns get no added latency before
`turn.start`.
- No schema change and no change to persisted run state formats.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 05:26:12 -06:00
..