Files
PaperClipAI/packages/paperclip-runner/runner
DottaandClaude Opus 5.5 f755361ffe fix(recovery): deliver a steered message the restart lost before the model read it (#19962)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A user can steer a running Codex or Pi turn: a comment goes into the
live turn instead of waiting for the next run
> - The provider acknowledges the steer at once, but the model reads it
only at its next step, after the in-flight tool call
> - A control-plane restart in that window kills the provider with the
message in memory. Restart recovery starts a continuation turn that does
not carry the message. The message is lost and no error shows
> - The restart chaos harness showed this in 6 of 6 steered runs (Codex,
K1, K2 and K6, 2 samples each): steer acknowledged, run succeeded, no
reply to the steered message
> - This pull request makes restart recovery deliver every acknowledged
steer that the model did not read, exactly once
> - The benefit is that manual Steer no longer loses messages on
restart, and the planned automatic steering can ship on top of it

## Linked Issues or Issue Description

**What happened?**
A comment steered into a running Codex turn got no reply when a
control-plane restart (SIGKILL, SIGTERM or crash loop) happened during
the agent's tool call. The run succeeded, and the steered comment had no
effect.

**Steps to reproduce**
1. Start a Codex task that runs a 40-second command.
2. During the command, post a comment and click Steer. The provider
acknowledges it.
3. Restart the control plane before the command ends.
4. Recovery resumes the turn with a continuation. The agent finishes the
task and ignores the comment.

**Expected behavior**
The agent reads and handles the steered comment once, after recovery.

**Root cause**
The provider keeps acknowledged steering input in memory until its next
model step. The restart continuation
(`paperclip.native-continuation.v1`) only told the model to continue the
current request, so the acknowledged input was gone.

## What Changed

- `unreadSteeredMessageIds` (runner package): derives
acknowledged-but-unread steers from run-log events in order. A
`steering_acknowledgement` item adds the message id (once per id). A
`userMessage` item in the same turn reads the oldest unread steer. Codex
records each steered input as one `userMessage` item when it enters the
conversation. A `userMessage` item in a later turn reads all earlier
steers, because the continuation carries them. `turn.completed` reads
all steers.
- `readUnreadSteeredMessages` (server): reads those facts from
`heartbeat_run_events` (only transport-validated events with a
`sourceEventId`). It also reads the comment or interaction text for the
same company and issue. The run log is the durable record of both
"acknowledged" and "read", so no new column is needed.
- The restart continuation now includes `steeredMessages` and one
instruction to handle them, oldest first. The runtime reads them through
a lazy `getUnreadSteeredMessages` option, only when a restart
continuation starts. Native recovery wires it for every run.
- `queuedInteractionSteeringBody` is extracted, so recovery rebuilds a
steered interaction response with the same text the steer sent.
- Fake Codex gets `--consume-steer`, which emits the `userMessage` item
for an accepted steer, like real Codex.

## Verification

- Red/green live test with real runnerd and fake Codex
(`runnerd-restart-continuation.test.ts`, "delivers a steered message the
restart lost exactly once"): start a held turn, steer it, SIGKILL runner
and provider, restore and recover.
- "unread": the continuation carries the message exactly once. Without
the runtime change this test fails.
- "read" (`--consume-steer`): the continuation does not carry it again.
- Both: after the continuation completes, a later restart has nothing to
deliver.
- `steering-delivery.test.ts`: ordering, dedupe, initial-input,
later-turn and completed-turn rules.
- `unread-steered-messages.test.ts` (embedded Postgres): ignores events
without a source id and messages from other issues. It reads interaction
text, ignores model-output items as read evidence, and returns nothing
after the continuation input.
- `pnpm vitest run src/steering-delivery.test.ts
src/live/runnerd-restart-continuation.test.ts` (runner) and `pnpm vitest
run src/services/native-runtime/unread-steered-messages.test.ts`
(server): pass on the rebased revision.
- `issue-queued-comments-routes.test.ts` and
`native-session-runtime.test.ts -t restart`: pass.
`native-session-resume.test.ts` "archives a damaged prior epoch…" fails
locally on plain `origin/master` too (local runnerd binary); CI runs it
with a fresh build.
- Restart chaos (Codex steer mode, K1/K2/K6) runs after this lands in
the harness slot. Before: 0 of 6 steered messages answered.

## Risks

- Duplicate delivery: a provider that records steered input without a
`userMessage` item, or a `userMessage` event lost in the crash window
before persistence, makes recovery send the message again. The model
then sees it twice. That is safer than losing it, and the instruction
asks the model to handle each one once.
- Pi (ACPX) steering has no `userMessage` read evidence today, so a Pi
steer that the model already read before a restart may arrive again in
the continuation.
- Only the restart continuation carries the messages. Other recovery
paths start after a completed turn, which already read every steer.
- No schema or migration change. A session saved by an earlier release
resumes normally; its run log already holds the acknowledgement events.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file editing, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 22:22:52 -06:00
..