Files
PaperClipAI/packages
Nicky LeachandClaude Fable 5.2 0ebe6d846d perf(runner): keep a quiet Claude provider between turns of a run (#21293)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar. The
Runner stops and starts the sidecar and Claude at each turn boundary.
> - The reason was that a late tool call from an earlier turn could run
in a later turn. #21278 removes that reason for Claude: the sidecar
refuses a call that the active turn did not announce.
> - A second reason remains. Claude Code works by itself between
prompts. When a background task finishes, Claude starts a cycle of its
own. If the process stayed alive with a live background task, that cycle
could start between turns or mix with the next turn.
> - So the Runner can keep the process only when it knows that Claude
has no background work.
> - This pull request adds that knowledge and uses it. After a Claude
turn that completed with no background work, the Runner keeps the
process for the next turn of the same run.
> - The benefit is that a later turn of the same run starts with no
provider restart. This is also the base for the next step, which lets a
run continue when background work finishes.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is stacked on #21278. Its base branch is
`fix/bind-claude-tool-calls-to-turn`. Refs #19912 (design), #12419
(restart at turn start), #20145 (restart at park) and #20190 (restart at
turn end).

**What existing behavior does this improve?**

The provider restart between turns of the same run, for Claude through
ACPX.

**Subsystem affected**

Paperclip Runner: the ACPX sidecar (TypeScript) and the ACPX provider
session in `runner-core` (Rust). Claude only.

**Current behavior**

- When a completed turn settles, the Runner restarts the sidecar and
Claude (`restart_session_after_turn`, #20190).
- If that restart did not run, the next turn restarts them at its start
(`start_turn`).
- The Runner does not know if Claude has background tasks.

**Proposed behavior**

- The sidecar asks `claude-agent-acp` for its async task reports. This
is an optional capability of the pinned `claude-agent-acp` 0.73.0. The
patch does not change.
- A tracker reads those reports and the id of each background Bash
command. It reports `{ live: 0 }` only when it knows that this process
never had a background task and that Claude did not act between prompts.
In all other cases the state is unknown.
- The sidecar adds `backgroundTasks` to the terminal event of a
completed Claude turn, and to `session.snapshot`.
- The Runner keeps the process between two turns only when all of these
are true:
  - the agent is Claude;
  - the turn completed (not failed, cancelled or interrupted);
  - the terminal event reported `backgroundTasks: { "live": 0 }`;
  - each existing condition of the turn-end restart is true;
- before the next turn, a new `session.snapshot` reports the same
identity and run, no active turn, no pending work, and `{ "live": 0 }`;
  - the settled-turn ledger and the tool replay ledgers are not full.
- In each other case the Runner restarts as before.
- A kept process keeps its tool call-id tombstones and its settled turn
identities.

**Reason and benefit**

- A turn that follows a quiet turn in the same run starts without a
restart.
- The decision is safe by default: unknown state means a restart.

## What Changed

TypeScript (`packages/paperclip-runner/src`):

- `drivers/acpx/claude-background-tasks.ts` (new):
`ClaudeBackgroundTaskTracker`. It reads raw ACP messages. It reports
unknown when the capability is not negotiated, when a task finished,
when a live task is not a shell command, when an update is malformed,
when a limit is reached, when Claude acts with no client request in
flight, or when Claude calls `CronCreate` or `ScheduleWakeup`.
- `drivers/acpx/provider-failure.ts` and `acpx-runtime-adapter.ts`:
Claude sessions also advertise the `asyncTasks` capability. The raw ACP
observer gives each message to the tracker. ACPX 0.22 drops the
`async_task_*` updates before the turn event stream, so only the raw
observer sees them.
- `drivers/acpx/runtime-host.ts`: `backgroundTaskState()`.
- `cli/acpx-runtime-sidecar.ts`: the terminal event of a completed
Claude turn carries `backgroundTasks` in the same payload as
`goalAtTerminal`, with the same size limit. `session.snapshot` carries
it when no turn is active.
- `drivers/acpx/claude-tool-use-admission.ts`:
`hasRoomForAnotherTurn()`. When the admitted-id ledger cannot hold one
more full turn, the sidecar reports unknown, so the Runner restarts
before the ledger is full.

Rust (`packages/paperclip-runner/runner/crates/runner-core`):

- `acpx_provider_session.rs`: the session records a quiet turn from the
terminal event. `start_turn` continues in the same process only under
the conditions above, and then uses `prepare_turn`, which keeps the
tombstones.
- `acpx_provider_generation.rs`: `can_continue_after_turn`,
`continue_after_quiet_turn`, and the snapshot check
`background_work_is_quiet`.
- `acpx_park_preparation.rs`: the turn-end restart does not run for a
quiet Claude turn.
- `acpx_provider_backend.rs`: `session.reconciled` reports a replaced
process only when the turn start replaced it.
- `bin/fake-acpx-sidecar.rs`: new option `--background-tasks` for tests.
- `runner/DURABLE_TRANSPORT.md`: the rule for a kept sidecar.

Contract (`packages/paperclip-runner/protocol`):

- The `backgroundTasks` report is `$defs.backgroundTasks` in
`provider-schemas/acpx-sidecar.schema.json`: an object with only `live`,
a non-negative integer. The schema validates it on
`runtime.turn_terminal`.
- `scripts/generate-acpx-sidecar-contract.mjs` generates the TypeScript
type and reader and the Rust reader from it. The tracker, the sidecar
and the Rust session use the generated code.
- `fixtures/provider-boundary/acpx-background-tasks.json` has shared
test documents. The TypeScript reader, the Rust reader and the schema
are tested with the same accepted and refused shapes.
- This changes the contract digest of the provider pack
(`prp.acpx-sidecar` and `rust`). Older image packs are not reused. No
provider `commandDigest` and no profile version changes.

Not changed: park and run attachment. A process that ran a turn is still
restarted at park, and only a process that ran no turn is bound to the
next run. Other ACPX agents and Codex do not change. The
`claude-agent-acp` patch does not change.

## State saved by the previous release

- This pull request adds no persisted state. The three new fields of the
session are in memory only.
- A terminal event or a snapshot without `backgroundTasks` means
unknown. An older sidecar therefore gets the restart, as before.
- An older Runner with a new pack ignores the field and restarts at each
turn, as before.
- After a Runner restart, the in-memory state is gone. The next turn
restarts the provider, as before. A test covers this.

## Verification

From `packages/paperclip-runner`:

- `npx vitest run src/drivers/acpx/claude-background-tasks.test.ts
src/drivers/acpx/acpx-runtime-adapter.claude-background-tasks.test.ts
src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts
src/drivers/acpx/claude-tool-use-admission.test.ts
src/cli/acpx-runtime-sidecar.test.ts
src/drivers/acpx/acpx-runtime-adapter.test.ts
src/drivers/acpx/runtime-host.test.ts`: `7 passed (7)` files, `363
passed` tests.
- `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p
tsconfig.test.json`: no errors.
- `pnpm run typecheck:typescript` (it includes the contract checks): no
errors. `node --test test/acpx-sidecar-contract.test.mjs`: 17 passed.
- `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no
changes.
- `cargo test -p paperclip-runner-core --test
acpx_quiet_turn_continuation --test acpx_park_preparation --test
acpx_park_readiness --test acpx_provider_turns --test
acpx_provider_session --test acpx_warm_attachment --test
native_provider_backend`: 4, 33, 8, 31, 12, 6 and 47 passed. No test
failed.
- `cargo test -p paperclip-runner-core --lib`: 637 passed, 4 ignored.
- Mutation check: when the Rust code accepts each live count as quiet,
two tests in `acpx_park_preparation` fail.

New tests:

- **Real ACP runtime**
(`acpx-runtime-adapter.claude-background-tasks.test.ts`), with the
pinned `claude-agent-acp`, the pinned Claude Code client and a local
fake Anthropic API. A foreground Bash turn reports `{ live: 0 }`. A
background Bash in the same process reports `{ live: 1 }` at the
terminal event, and unknown after Claude ran its own cycle for the
finished task. Each turn has its own usage receipt. One process serves
all turns.
- **Tracker unit tests** (13): spawn, completion, failure, stop, pause,
a task that is not a shell, a subagent, no negotiation, malformed
updates, activity between prompts, scheduling tools, session load
replay, limits.
- **Sidecar tests**: the terminal event carries the state for Claude and
not for Codex. A full ledger reports unknown.
- **Rust** `tests/acpx_quiet_turn_continuation.rs` (4) and
`tests/support/acpx_quiet_turn_restart.rs` (7):
- a quiet Claude turn continues in the same process, and a replay of a
settled call id is refused;
  - live, unknown and malformed reports restart;
- background work that appears after the terminal event restarts at turn
start;
  - failed, cancelled and interrupted turns restart;
  - Codex with a quiet report restarts;
  - an active goal restarts;
  - 4,096 kept turns, then the next turn restarts;
- park after a kept process still prepares a new process, and the next
run attaches to it;
  - a Runner restart after a kept process resumes in a new process.
- The existing restart tests did not change. The fake sidecar sends no
report by default, so they are the tests for the unknown case.

Live run with a real Claude account, not in CI: the real ACP runtime
(ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code
client), a real tool bridge, and the real Anthropic API with model
`claude-sonnet-5-5`, on the code at the top of this stack. A turn that
started one background shell reported `{ live: 1 }`. A turn that started
two reported `{ live: 2 }`, then `{ live: 1 }` and `{ live: 0 }` as each
shell finished. A turn in which Claude waited for its background shell
in the foreground reported the unknown state.

Not run: server integration tests (the fake sidecar there sends no
report, so their process counts do not change), Linux, and a live run
through the real sidecar process and runnerd.

## Risks

- **All Claude sessions now negotiate the `asyncTasks` capability.**
`claude-agent-acp` then sends `async_task_*` updates and adds a marker
to a background Bash update. ACPX drops the updates. No other behavior
of `claude-agent-acp` changes, from a read of its code.
- **Park after a kept process pays the restart again**, as it did before
#20190. With real Claude that restart was about 1.5 s.
- **A small window remains** between the snapshot and the prompt of the
next turn. Claude can start a cycle of its own in that window only
without a background task, because a process that had a background task
is not kept.
- **Activity between prompts is found only when no client request is in
flight.** A request that never gets a response would hide such activity.
- **Background watchers cannot be told apart from other tasks**, so each
task that is not a shell command makes the state unknown.
- **Each kept turn adds one `session.snapshot` request.**
- A multi-turn run with real Claude now uses one provider process. This
path has tests with the fake sidecar and with the real ACP runtime. It
has no test with the real sidecar host.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change, reviewed the diff, and ran the TypeScript and Rust verification
again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), investigated the signals
and wrote the change and the tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 12:54:43 -08:00
..