mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 11:59:26 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude Code through the ACPX sidecar. The Runner stops and starts the sidecar and Claude at each turn boundary. > - The reason was that a late tool call from an earlier turn could run in a later turn. #21278 removes that reason for Claude: the sidecar refuses a call that the active turn did not announce. > - A second reason remains. Claude Code works by itself between prompts. When a background task finishes, Claude starts a cycle of its own. If the process stayed alive with a live background task, that cycle could start between turns or mix with the next turn. > - So the Runner can keep the process only when it knows that Claude has no background work. > - This pull request adds that knowledge and uses it. After a Claude turn that completed with no background work, the Runner keeps the process for the next turn of the same run. > - The benefit is that a later turn of the same run starts with no provider restart. This is also the base for the next step, which lets a run continue when background work finishes. ## Linked Issues or Issue Description No public issue exists. The issue is described here. This pull request is stacked on #21278. Its base branch is `fix/bind-claude-tool-calls-to-turn`. Refs #19912 (design), #12419 (restart at turn start), #20145 (restart at park) and #20190 (restart at turn end). **What existing behavior does this improve?** The provider restart between turns of the same run, for Claude through ACPX. **Subsystem affected** Paperclip Runner: the ACPX sidecar (TypeScript) and the ACPX provider session in `runner-core` (Rust). Claude only. **Current behavior** - When a completed turn settles, the Runner restarts the sidecar and Claude (`restart_session_after_turn`, #20190). - If that restart did not run, the next turn restarts them at its start (`start_turn`). - The Runner does not know if Claude has background tasks. **Proposed behavior** - The sidecar asks `claude-agent-acp` for its async task reports. This is an optional capability of the pinned `claude-agent-acp` 0.73.0. The patch does not change. - A tracker reads those reports and the id of each background Bash command. It reports `{ live: 0 }` only when it knows that this process never had a background task and that Claude did not act between prompts. In all other cases the state is unknown. - The sidecar adds `backgroundTasks` to the terminal event of a completed Claude turn, and to `session.snapshot`. - The Runner keeps the process between two turns only when all of these are true: - the agent is Claude; - the turn completed (not failed, cancelled or interrupted); - the terminal event reported `backgroundTasks: { "live": 0 }`; - each existing condition of the turn-end restart is true; - before the next turn, a new `session.snapshot` reports the same identity and run, no active turn, no pending work, and `{ "live": 0 }`; - the settled-turn ledger and the tool replay ledgers are not full. - In each other case the Runner restarts as before. - A kept process keeps its tool call-id tombstones and its settled turn identities. **Reason and benefit** - A turn that follows a quiet turn in the same run starts without a restart. - The decision is safe by default: unknown state means a restart. ## What Changed TypeScript (`packages/paperclip-runner/src`): - `drivers/acpx/claude-background-tasks.ts` (new): `ClaudeBackgroundTaskTracker`. It reads raw ACP messages. It reports unknown when the capability is not negotiated, when a task finished, when a live task is not a shell command, when an update is malformed, when a limit is reached, when Claude acts with no client request in flight, or when Claude calls `CronCreate` or `ScheduleWakeup`. - `drivers/acpx/provider-failure.ts` and `acpx-runtime-adapter.ts`: Claude sessions also advertise the `asyncTasks` capability. The raw ACP observer gives each message to the tracker. ACPX 0.22 drops the `async_task_*` updates before the turn event stream, so only the raw observer sees them. - `drivers/acpx/runtime-host.ts`: `backgroundTaskState()`. - `cli/acpx-runtime-sidecar.ts`: the terminal event of a completed Claude turn carries `backgroundTasks` in the same payload as `goalAtTerminal`, with the same size limit. `session.snapshot` carries it when no turn is active. - `drivers/acpx/claude-tool-use-admission.ts`: `hasRoomForAnotherTurn()`. When the admitted-id ledger cannot hold one more full turn, the sidecar reports unknown, so the Runner restarts before the ledger is full. Rust (`packages/paperclip-runner/runner/crates/runner-core`): - `acpx_provider_session.rs`: the session records a quiet turn from the terminal event. `start_turn` continues in the same process only under the conditions above, and then uses `prepare_turn`, which keeps the tombstones. - `acpx_provider_generation.rs`: `can_continue_after_turn`, `continue_after_quiet_turn`, and the snapshot check `background_work_is_quiet`. - `acpx_park_preparation.rs`: the turn-end restart does not run for a quiet Claude turn. - `acpx_provider_backend.rs`: `session.reconciled` reports a replaced process only when the turn start replaced it. - `bin/fake-acpx-sidecar.rs`: new option `--background-tasks` for tests. - `runner/DURABLE_TRANSPORT.md`: the rule for a kept sidecar. Contract (`packages/paperclip-runner/protocol`): - The `backgroundTasks` report is `$defs.backgroundTasks` in `provider-schemas/acpx-sidecar.schema.json`: an object with only `live`, a non-negative integer. The schema validates it on `runtime.turn_terminal`. - `scripts/generate-acpx-sidecar-contract.mjs` generates the TypeScript type and reader and the Rust reader from it. The tracker, the sidecar and the Rust session use the generated code. - `fixtures/provider-boundary/acpx-background-tasks.json` has shared test documents. The TypeScript reader, the Rust reader and the schema are tested with the same accepted and refused shapes. - This changes the contract digest of the provider pack (`prp.acpx-sidecar` and `rust`). Older image packs are not reused. No provider `commandDigest` and no profile version changes. Not changed: park and run attachment. A process that ran a turn is still restarted at park, and only a process that ran no turn is bound to the next run. Other ACPX agents and Codex do not change. The `claude-agent-acp` patch does not change. ## State saved by the previous release - This pull request adds no persisted state. The three new fields of the session are in memory only. - A terminal event or a snapshot without `backgroundTasks` means unknown. An older sidecar therefore gets the restart, as before. - An older Runner with a new pack ignores the field and restarts at each turn, as before. - After a Runner restart, the in-memory state is gone. The next turn restarts the provider, as before. A test covers this. ## Verification From `packages/paperclip-runner`: - `npx vitest run src/drivers/acpx/claude-background-tasks.test.ts src/drivers/acpx/acpx-runtime-adapter.claude-background-tasks.test.ts src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts src/drivers/acpx/claude-tool-use-admission.test.ts src/cli/acpx-runtime-sidecar.test.ts src/drivers/acpx/acpx-runtime-adapter.test.ts src/drivers/acpx/runtime-host.test.ts`: `7 passed (7)` files, `363 passed` tests. - `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p tsconfig.test.json`: no errors. - `pnpm run typecheck:typescript` (it includes the contract checks): no errors. `node --test test/acpx-sidecar-contract.test.mjs`: 17 passed. - `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no changes. - `cargo test -p paperclip-runner-core --test acpx_quiet_turn_continuation --test acpx_park_preparation --test acpx_park_readiness --test acpx_provider_turns --test acpx_provider_session --test acpx_warm_attachment --test native_provider_backend`: 4, 33, 8, 31, 12, 6 and 47 passed. No test failed. - `cargo test -p paperclip-runner-core --lib`: 637 passed, 4 ignored. - Mutation check: when the Rust code accepts each live count as quiet, two tests in `acpx_park_preparation` fail. New tests: - **Real ACP runtime** (`acpx-runtime-adapter.claude-background-tasks.test.ts`), with the pinned `claude-agent-acp`, the pinned Claude Code client and a local fake Anthropic API. A foreground Bash turn reports `{ live: 0 }`. A background Bash in the same process reports `{ live: 1 }` at the terminal event, and unknown after Claude ran its own cycle for the finished task. Each turn has its own usage receipt. One process serves all turns. - **Tracker unit tests** (13): spawn, completion, failure, stop, pause, a task that is not a shell, a subagent, no negotiation, malformed updates, activity between prompts, scheduling tools, session load replay, limits. - **Sidecar tests**: the terminal event carries the state for Claude and not for Codex. A full ledger reports unknown. - **Rust** `tests/acpx_quiet_turn_continuation.rs` (4) and `tests/support/acpx_quiet_turn_restart.rs` (7): - a quiet Claude turn continues in the same process, and a replay of a settled call id is refused; - live, unknown and malformed reports restart; - background work that appears after the terminal event restarts at turn start; - failed, cancelled and interrupted turns restart; - Codex with a quiet report restarts; - an active goal restarts; - 4,096 kept turns, then the next turn restarts; - park after a kept process still prepares a new process, and the next run attaches to it; - a Runner restart after a kept process resumes in a new process. - The existing restart tests did not change. The fake sidecar sends no report by default, so they are the tests for the unknown case. Live run with a real Claude account, not in CI: the real ACP runtime (ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code client), a real tool bridge, and the real Anthropic API with model `claude-sonnet-5-5`, on the code at the top of this stack. A turn that started one background shell reported `{ live: 1 }`. A turn that started two reported `{ live: 2 }`, then `{ live: 1 }` and `{ live: 0 }` as each shell finished. A turn in which Claude waited for its background shell in the foreground reported the unknown state. Not run: server integration tests (the fake sidecar there sends no report, so their process counts do not change), Linux, and a live run through the real sidecar process and runnerd. ## Risks - **All Claude sessions now negotiate the `asyncTasks` capability.** `claude-agent-acp` then sends `async_task_*` updates and adds a marker to a background Bash update. ACPX drops the updates. No other behavior of `claude-agent-acp` changes, from a read of its code. - **Park after a kept process pays the restart again**, as it did before #20190. With real Claude that restart was about 1.5 s. - **A small window remains** between the snapshot and the prompt of the next turn. Claude can start a cycle of its own in that window only without a background task, because a process that had a background task is not kept. - **Activity between prompts is found only when no client request is in flight.** A request that never gets a response would hide such activity. - **Background watchers cannot be told apart from other tasks**, so each task that is not a shell command makes the state unknown. - **Each kept turn adds one `session.snapshot` request.** - A multi-turn run with real Claude now uses one provider process. This path has tests with the fake sidecar and with the real ACP runtime. It has no test with the real sidecar host. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change, reviewed the diff, and ran the TypeScript and Rust verification again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), investigated the signals and wrote the change and the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>