Files
PaperClipAI/doc
Nicky LeachandClaude Opus 5.5 e107a75755 fix(native): decide session settlement from live state (#18992)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native Runner sessions keep one provider conversation across task
turns, so a later run attaches to a session that an earlier run settled.
> - Three gates decide if a session is settled: Core's journal check,
the Runner's attach check, and the Runner's drain of provider messages
after a turn.
> - Each gate kept a list of harmless events and refused anything else.
Every new harmless event became an outage.
> - An example: Codex replays a settled turn's usage on `thread/resume`,
so `run.attach` refused every session that restarted after its idle
timeout.
> - Events after a terminal describe the past. Refusing the session
cannot undo them, and it hides them.
> - This pull request asks a different question: is anything live or
waiting right now? Past events are recorded and no longer block.
> - The benefit is that harmless events stop breaking session
continuity, with no event list for Core and the Runner to keep in step.

## Linked Issues or Issue Description

No issue exists. I searched open and closed pull requests and issues for
`run.attach requires a settled Codex provider session` and
`prior_owner_terminal_state_indeterminate`. Related work: #18890 added
one more accepted event to Core's list. This change replaces that list.
#18991 handles late usage and is independent of this change.

**What happened?**

On a test deployment, 11 runs failed in 21 days when `run.attach`
refused with `requires a settled Codex provider session with no pending
events`. In all 11, the four pending events were two startup audit
events, `session.resumed`, and a `codex_resume_usage_snapshot`
diagnostic. The diagnostic is what the attach check refused. The retry
then failed with `prior_owner_terminal_state_indeterminate` in 6 of
them, which hid the first failure.

**Expected behavior**

A session with nothing live or waiting accepts the next run. If
attachment fails, the run reports the real cause.

**Steps to reproduce**

1. Finish a Codex turn on a native Runner session.
2. Wait for the idle timeout so that the provider process stops.
3. Start the next run. Codex replays the usage for the settled turn on
`thread/resume`.
4. The attach fails.
`durable_backend_attaches_after_a_settled_resume_usage_snapshot`
reproduces this on the unfixed code.

**Paperclip version or commit**

Observed on `8fd917aedf`. The same check is on `master` before this
change.

**Deployment mode**

Cloud deployment, with a native Runner in a plugin sandbox environment.

## What Changed

- **Runner attach:** `attach_run` refuses only for live or waiting
state: no thread, a closed session, a session that needs reconciliation,
an active or ambiguous turn, an active goal, a pending tool call, or an
unanswered provider request. The error now names the reason.
- **Pending events:** they no longer block attachment. They stay in the
retained queue, and the Runner commits them under the prior authority
before it rotates. `session.resumed` is still discarded because it is
superseded.
- **Runner drain:** `drain_completed_turn_tail_for_warm_attachment` no
longer needs a list of known messages. It still fails on a message that
names another turn, on `turn/started`, on a provider request, and on a
process exit.
- **Core:** `isSettledRemoteWarmControlState` accepts any authenticated
event after the terminal unless it starts work (`turn.started`,
`turn.accepted`, `run.attached`, `tool.execution.started`,
`semantic_tool.input`, `runtime_request.created`), or a goal snapshot or
update shows an active goal or work in progress. All identity, source
and protocol checks are unchanged.
- **Visibility:** the next run's log names the event types that were
recorded after the previous run finished.
- **Real cause:** a same-run retry after a failed attachment now fails
with `prior_attach_failed: <original failure>`, instead of
`prior_owner_terminal_state_indeterminate`.
- **Docs:** added "Session settlement after a run" to
`doc/execution-semantics.md`.

## Verification

- `cd packages/paperclip-runner/runner && cargo test --locked -p
paperclip-runner-core --test codex_provider`: 102 passed, 2 ignored.
`--lib`: 478 passed, 2 ignored. `--test durable_recovery`: 3 passed.
- **Failure before the fix:**
`durable_backend_attaches_after_a_settled_resume_usage_snapshot` failed
on the unfixed code with the same error as production.
- `cd server && pnpm exec vitest run
src/services/native-runtime/remote-warm-session-recovery.test.ts
src/services/native-runtime/native-session-executor.test.ts -t "settled
remote warm control evidence|recovers a blocked|automatically recovers
only a proven settled local session|retained|warm"`: 476 passed, 680
skipped by the filter.
- `pnpm exec vitest run
src/services/native-runtime/native-session-resume.test.ts`: 57 passed.
- `npx tsc --noEmit -p .` in `server` reports no errors.
- **Real data, read-only:** I decrypted the 11 failed runs' stored
events on a test deployment and printed only event types, field names,
and thread and turn IDs. All 11 show the same four events.
- **Audit of live state:** I read the Runner's provider state and
request tracking to build the attach check. It is a code audit. I have
not proved that every kind of live work sets a flag. The Runner holds
active turns, ambiguous starts, tool calls and provider requests in
state today.
- **Not covered:** the wiring that throws `prior_attach_failed` has unit
coverage for its message only, not an end-to-end test.
- **Review:** Greptile Apex scored 4/5, then 5/5 at the current head.
- **Not run locally:** the full test suite. CI provides that coverage.
- **Not verified live:** no live run has used this change.

## Risks

- **Fail-open for unknown live work:** an event type that starts work,
but is not in the short list and sets no state flag, would now pass.
This is the main tradeoff. The previous list failed closed, but it
caused real outages.
- **More events reach the journal after a terminal.** The usage replay
diagnostic is now delivered rather than refused. These events are
informational, and they are named in the next run's log.
- **A failed attachment is not retried on the same run.** Some refusals
could clear on their own, such as a turn that finishes, and some attach
errors are not about live work, such as a failed provider restore. The
run now reports the cause, and it needs another run to continue. On a
test deployment, the next successful run after each failure used a new
provider thread and a new execution workspace. A later change could add
same-run retry.
- **Rollout order:** deploy Core before the Runner. An older Core would
refuse the new events that a new Runner commits after a terminal.
- **Migrations:** this change has no migration.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be reworked.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code,
with tool use and code execution. A human directed the design and
approved the scope.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 12:04:29 -08:00
..
…
…
…
…
…