Commit Graph
567 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 28c11084ad test(runner): wait for the background task notification turn before the next Claude turn (#22522)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner converts raw Claude ACP session updates the same
way ACPX does. One test proves this against the real Claude ACP bridge.
> - The test runs four turns. The third turn starts a background command
(`sleep 1`). When that command ends, Claude starts its own
task-notification turn.
> - The test starts to record raw updates before ACPX sends the next
prompt. If the notification turn arrives in that window, the test
records it. ACPX correctly does not give it to the prompt. The two lists
then differ.
> - This race stopped the Release for a722935dbc in "Verify Paperclip
Runner (protocol)".
> - This pull request makes the test wait until the notification turn
ends before it starts the next turn.
> - The benefit is a deterministic test and a Release lane that does not
fail on runner speed.

## Linked Issues or Issue Description

**What happened?**
The Cloud readiness run 44970524141 (Release for a722935dbc) failed in
`verify / Verify Paperclip Runner (protocol)`. The failed test is
`acpx-runtime-adapter.claude-session-update-events.test.ts > converts
the real Claude ACP stream exactly as ACPX does for every update kind it
emits`. The assertion on the `turn-plan` turn received `msg_10` with
`_claude/origin.kind: "task-notification"`. ACPX gave `msg_12` with
`kind: "human"`. The log line "Copilot metadata verification failed …
shutdown-incomplete" is the expected stderr of a different test that
passed. It is not related.

**Expected behavior**
The test compares only the updates of the prompt that ACPX runs. A
notification turn from an earlier background command must not get into
the recorded list of the next prompt.

**Steps to reproduce**
1. On master, add `await new Promise(resolve => setTimeout(resolve,
3_000));` in `runTurn` after `const yielded = []` for `turnId ===
"turn-plan"`. This holds the window between "start to record" and "ACPX
sends the prompt" open while the background command ends.
2. Run `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts`.
3. The test fails with the same diff as CI (`msg_10` /
`task-notification` against `msg_12` / `human`).

**Paperclip version or commit**
master at a722935dbc (the race exists since #21420).

Classification: timing flake in the test design. It is not a product
regression and not a dependency change. The task-notification turn is
known product behavior (see `claude-background-tasks.ts`). The test
passed in every earlier Cloud readiness run on master.

## What Changed

- The test counts task-notification turn-end markers (`usage_update`
with `_claude/origin.kind === "task-notification"`) in its stdout
reader.
- After the background turn, the test waits (up to 30 s) until that
notification turn ends. Then it starts the plan turn.
- No Runner source, ACPX profile, digest, pin or image changes.

## Verification

- Red: the original test with the 3 s hold in the window fails with the
exact CI diff (3 of 3 runs).
- Green: the fixed test with the same 3 s hold passes. The fixed test
without the hold passes 5 of 5 local runs.
- `pnpm --filter @paperclipai/paperclip-runner run check:acpx-profiles`
and `check:protocol-manifest` pass. This is a test-only change, so no
new runner image or attestation pin is necessary.

## Risks

- Low risk. Test-only change. If a future Claude Code release stops
sending a task-notification turn, the wait fails after 30 s with a clear
assertion instead of a silent pass.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, GitHub CLI) for investigation, the
fix and local verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-21 20:41:56 -06:00
611dc68581 fix(runner): continue a lost turn on a resumed warm session (#22447)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents in a managed sandbox run under the Paperclip runner
(`runnerd`). When `runnerd` exits mid-run, the transport relaunches it,
and the runner restores the same conversation
> - #22235 lets the execution that started a turn send one continuation
turn when the restored runner reports that turn as
`provider_turn_lost_on_restore`
> - That rule applies only to a session that the execution opened fresh
(`!recovered`). A run that resumes the warm conversation of its previous
run is a recovered session. A chat follow-up does this on every new
message
> - So a resumed session that loses its own task turn gets no
continuation. The run fails, and the user must press Retry
> - This pull request makes the task turn that this execution started
its own turn, whether the session was opened fresh or resumed
> - The benefit is that a chat follow-up keeps its conversation and
workspace after a mid-run `runnerd` restore, as #22235 intended

## Linked Issues or Issue Description

Refs #22235 (continue a turn lost to a mid-run runnerd restore). Refs
#22068 (disk reserve). Refs #22239 (stop retrying a run that filled its
sandbox disk).

**What happened?**
On staging, an agent in a chat task filled its 10 GiB managed sandbox
disk. It fetched the full history of a large Git repository and searched
it. `runnerd` exited with `failed to create private durable state
temporary file: No space left on device (os error 28)`. The disk reserve
let the transport relaunch `runnerd`, and the runner restored the same
conversation. Then the runner reported `turn.failed {code:
provider_turn_lost_on_restore, recoverable: true}` for the turn that
this run had started. The runtime did not send the continuation turn.
The run failed as `sandbox_disk_full` with the message "The runner
restored the conversation after process loss, but the previous turn is
no longer active." A manual Retry on a fresh provider session finished
the work.

The run started from the warm checkpoint of the previous run in the same
conversation (`startup reuse: provider_session (reused)`, then
`session.resumed` with the same provider session id).
`loadWarmNativeCheckpoint` rebinds that checkpoint to the new run and
clears its turns. `executeNativeSession` recovers it, so `recovered` is
true. The run then starts its own task turn. `restoredInThisExecution`
required `!recovered`, so that turn could not be continued.

**Expected behavior**
When a mid-run `runnerd` restore loses the task turn that this execution
started, the execution sends one continuation turn in the restored
conversation. This applies to a fresh session and to a resumed warm
session.

**Steps to reproduce**
1. Run a native agent through `paperclip_runner` in a managed sandbox,
as a follow-up on an existing conversation (a warm resumed session).
2. During the turn, make `runnerd` exit, for example by filling the
sandbox disk.
3. The transport relaunches `runnerd`, and the runner reports the turn
as lost on restore.
4. The run fails instead of continuing the turn.

## What Changed

- `packages/paperclip-runner/src/native-session-runtime.ts`:
`restoredInThisExecution` no longer requires a fresh session. The turn
id that this execution records when it starts its task turn is now
enough.
- The execution records that turn id only for a turn that carries the
task work. A disposition-only recovery turn (the turn that asks only for
`paperclip_finish` after a completed turn) is still not continued, as
before.
- All other guards stay: one continuation per run, the provider
allowlist, no compaction or goal runs, no pending runtime requests, no
saved semantic result, and the snapshot must show that same turn as
restart-interrupted. The control-plane restart claim path
(`restoredAtStartup`) does not change.
- `native-session-runtime.test.ts`: three new cases for a recovered warm
session:
  - it continues its own task turn once after a mid-run restore;
  - it does not continue a lost turn that it never started;
  - it does not continue a lost disposition-only recovery turn.

## Verification

- Red before the fix: `npx vitest run
src/native-session-runtime.test.ts` in `packages/paperclip-runner`
failed `continues its own turn on a resumed warm session once when
runnerd is restored mid-execution` with `NativeProviderTerminalFailure`
(`code: native_provider_terminal_failed`, `providerCode:
provider_turn_lost_on_restore`). This is the same failure as on staging.
- Green after the fix: the same file has 303 passed, 1 failed. The one
failure is `settles a governed wait through real Runner shutdown after a
missing Codex terminal`. It needs the Rust `paperclip-runnerd` binary
(`ENOENT runner/target/debug/paperclip-runnerd`). It fails the same way
before and after the change.
- The existing `requires the control plane restart claim` and `does not
continue a lost turn this execution never started` cases still pass.
- To prove that the disposition-only guard is necessary, I removed it
locally. The new disposition-only case then failed (the run continued).
I put the guard back.
- `native-session-runtime.provider-turn.test.ts`: 36/36 passed. The
Codex driver provider-turn, recovery, stale-generation and
terminal-replay suites and `terminal-call-freshness.test.ts` passed.
- Not run locally: `src/live/runnerd-restart-continuation.test.ts`. It
needs the real Rust `runnerd` binary, and this sandbox has no Rust
toolchain. CI builds the runner and runs these tests.
- Live acceptance: after the automatic staging deploy, a chat follow-up
whose `runnerd` restarts mid-turn must show one continuation turn in the
same sandbox instead of a failed run.

## Risks

- **Second executor.** The continuation starts only for the turn id that
this execution received from its own `startTurn` call. A turn from an
earlier process, or a turn that the execution adopted, is not this
execution's turn. That path still needs the control-plane restart claim.
The continuation runs at most once per run
(`restartContinuationStarted`).
- **Disk still full.** If the continuation fills the disk again, the
second lost turn finds `restartContinuationStarted` set. The run then
fails as `sandbox_disk_full`, and automatic retries stay off (#22239).
The worst case is the current behavior.
- **Disposition-only turns.** A recovered session that only asks for the
disposition does not change. This change does not add a continuation for
that turn.
- Low risk otherwise. The change is 1 condition and 1 assignment in the
runner runtime.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, Paperclip API and Sentry connectors).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (except one test that needs
the Rust runner binary; see Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
doc change is necessary: the runtime comment states the rule)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: paperclipbugbot <356415228+paperclipbugbot@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 16:52:11 -08:00
Nicky LeachandClaude Opus 5.5 5371c83152 fix(runner): bundle the Copilot metadata probe as a standalone entrypoint (#22436)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner ships a provider pack with pinned provider
runtimes and small entrypoints
> - Most provider entrypoints, such as the Kimi metadata probe and the
ACPX sidecar, are esbuild bundles that run as one file
> - The Copilot metadata probe is plain `tsc` output that imports
sibling driver modules
> - So an environment that installs only selected provider-pack files
cannot run the Copilot probe
> - This pull request makes the Copilot metadata probe a standalone
verified entrypoint, like the Kimi probe
> - The benefit is that every metadata probe has the same self-contained
shape, and a consumer can ship the probe as one file

## Linked Issues or Issue Description

No public issue exists. This is an enhancement.

**What existing behavior does this improve?**
The Copilot metadata probe entrypoint
(`dist/cli/copilot-metadata-probe.js`, the
`paperclip-runner-copilot-metadata-probe` bin and provider-pack shim).

**Subsystem affected**
Paperclip Runner provider pack build (`packages/paperclip-runner`).

**Current behavior**
The probe is plain `tsc` output. It imports
`../drivers/copilot-metadata-probe.js` and more modules. It fails when a
consumer installs only selected provider-pack artifacts. The Kimi probe
does not have this problem because
`scripts/build-verified-provider-entrypoints.mjs` bundles it.

**Proposed behavior**
The build bundles the Copilot probe into one self-contained file, like
the Kimi probe. The bundle still finds the pinned Copilot assets when it
runs from `dist/cli/`.

**Reason and benefit**
Every metadata probe has the same standalone shape. A consumer can ship
the probe as one file. An alternative was to ship the full
`dist/drivers` tree with the probe, but that keeps the multi-file shape.

**Breaking changes**
None. The bin name, the output path and the probe output do not change.

## What Changed

- `scripts/build-verified-provider-entrypoints.mjs`: add
`copilot-metadata-probe` (source `src/cli/copilot-metadata-probe.ts`,
outputs `dist/cli/copilot-metadata-probe.js` and `.cjs`) with the same
format and flags as `kimi-metadata-probe`. The bundler asserts one
output and only `node:` imports.
- `src/cli/copilot-metadata-probe.ts`: move the top-level `await` into
`main()`. esbuild cannot emit top-level `await` in the CommonJS bundle.
Output and exit codes do not change.
- `src/drivers/acpx/provider-assets-root.ts`: when no package root is
bound, accept the bundled `dist/cli/copilot-metadata-probe.(js|cjs)`
module URL for the `copilot` provider only. This is the same rule the
bundled ACPX sidecar uses. Before this change, the bundle's
`import.meta.url` is the `dist/cli` file, not
`dist/drivers/acpx/copilot-installation.js`, so the module-URL lookup
failed. The bound `PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT` path does not
change.
- Tests: a bundler test that runs both bundles alone in a temporary pack
layout, and an asset-root test for the bundled probe path that also
proves other providers do not get the new path.

## Verification

- `node --test scripts/build-verified-provider-entrypoints.test.mjs` (44
tests, includes the new standalone-run test)
- `pnpm exec vitest run src/drivers/acpx/provider-assets-root.test.ts
src/drivers/copilot-metadata-probe.test.ts
src/drivers/acpx/copilot-installation.test.ts`
- `node --test scripts/kimi-metadata-probe.test.mjs
scripts/provider-pack-executable-shims.test.mjs
test/acpx-codex-package-contract.test.mjs`
- `pnpm exec vitest run
server/src/__tests__/copilot-connection-probe.test.ts`
- `pnpm --filter @paperclipai/paperclip-runner run typecheck:typescript`
- Manual check on darwin-arm64 with the pinned Copilot closure
installed, child process spawn denied by a preload:
- Bundled `.js` and `.cjs`, no bound root: installation verification
passes and the probe reaches the spawn step.
- Same run without the `provider-assets-root.ts` change:
`COPILOT_INSTALLATION_INVALID`.
- Bundled `.js` with `PAPERCLIP_ACPX_PROVIDER_PACKAGE_ROOT` bound:
installation verification passes.
- `pnpm verify` requires broader coverage (the bundler is a broad
input), so CI supplies it.

## Risks

- Low risk. The bin name, output path and shim do not change. The probe
output and exit codes do not change.
- The new module-URL rule accepts only
`/dist/cli/copilot-metadata-probe.(js|cjs)` and only for `copilot`. It
resolves to the package that holds `dist/cli`, the same as the sidecar
rule.
- The build now also writes `dist/cli/copilot-metadata-probe.cjs`.
Nothing reads it yet.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Provider: Anthropic Claude
- Model: Claude Opus 5.5 (`claude-opus-5-5`)
- Mode: agentic coding with tool use (shell, file edits, test runs)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 16:09:31 -08:00
Nicky LeachandClaude Opus 5.5 dce4d47f60 feat(plugins): carry Grok, Copilot and OpenCode credentials to plugin Runners (#22359)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environment-driver plugins can run `paperclip_runner` tasks on hosts
that the plugin owns
> - Core transfers the selected managed AI credential to the plugin as a
typed task credential
> - The credential contract has only two types: a Codex auth file and a
direct Claude credential
> - Grok, GitHub Copilot and OpenCode agents in a task-capable plugin
environment therefore fail closed, although the Runner already reads
their credentials from its environment
> - This pull request adds one generic, allowlisted environment
credential for these three harnesses and accepts the Runner's Grok
launch settings
> - The benefit is that plugin-provided Runner execution can run Grok,
Copilot and OpenCode agents with their selected connection

## Linked Issues or Issue Description

**Subsystem affected**

Plugin environment tasks (`packages/plugins/sdk` task contract) and
plugin-provided Runner execution in `server`.

**Problem or motivation**

An agent with the Grok, Copilot or OpenCode harness cannot run in a
plugin environment that runs Runner tasks. Core stops at credential
preparation with "Plugin Runner does not support the selected credential
provider". A Grok agent fails before that point too. The Runner emits
`grokEffort` and `grokDisableWebSearch` in its launch requirements (Core
turns off Grok web search by default), but the strict `settings` schema
of `environmentRunnerLaunchRequirementsSchema` rejects both keys.

**Proposed solution**

Core transfers the selected connection's credential variables for these
harnesses through one typed `provider-env` credential. Each provider has
a fixed allowlist of complete variable sets. The plugin sets the
variables in the Runner process environment.

**Alternatives considered**

One credential type per harness (`grok-env`, `copilot-env`,
`opencode-env`). This adds a type for each new harness. One generic type
with an exported allowlist keeps the same strictness with less surface.

**Roadmap alignment**

The roadmap lists Grok and OpenCode as supported harnesses. This change
lets plugin environments run them. It adds no new product concept.

**Additional context**

Grok subscription login (an auth file with refresh and writeback) is out
of scope. It still fails closed, with an error that names the xAI API
key as the supported option.

## What Changed

- `packages/plugins/sdk`: Added the `provider-env` task credential: `{
type: "provider-env", provider: "grok" | "copilot" | "opencode",
environment }`. The environment must contain exactly one complete
variable set for the provider and no other variable. Values are
non-empty, NUL-free and at most 65,536 characters, as for the other
credential types.
  - `grok`: `XAI_API_KEY`
  - `copilot`: `COPILOT_GITHUB_TOKEN`
- `opencode`: `OPENROUTER_API_KEY`, or `PAPERCLIP_AI_PROVIDER_URL` with
`PAPERCLIP_AI_PROVIDER_KEY`
- `packages/plugins/sdk`: Exported the allowlist as
`ENVIRONMENT_TASK_PROVIDER_CREDENTIAL_KEY_SETS` and the
`PluginEnvironmentTaskCredentialProvider` type. `claude-env` and
`codex-auth-json` do not change.
- `packages/plugins/sdk`: Added `grokEffort` (`minimal` to `xhigh`) and
`grokDisableWebSearch` (boolean) to the launch requirements `settings`,
with the types that the Runner uses.
- `server`: `preparePluginRunnerCredentials` now transfers the selected
connection's variables for the `grok`, `copilot` and `opencode`
harnesses, with the same account identity semantics as the Claude
branch. It refuses a Grok subscription, a Grok base URL override, and an
incomplete or ambiguous set. The error never contains a credential
value.
- `server`: The submit check now matches a `provider-env` credential to
the task harness. Before, it treated every non-Codex credential as
Anthropic.
- Drift test:
`server/src/__tests__/plugin-runner-credential-names.test.ts` checks the
SDK key sets against the Runner. The SDK list stays written by hand, and
the Runner keeps its own lists of forwarded names. For `grok` and
`copilot`, each name must be in `ACPX_CREDENTIAL_NAMES`, survive sidecar
admission with the session binding that the Runner mints, and reach the
agent process. The binding requirement must match the Rust sidecar's
candidate agents. For `opencode`, each name must be in the OpenCode
proxy environment set. Each name must also be in the Rust sidecar
credential keys or the Rust OpenCode provider keys. The test reads those
from the Runner source, as the Codex security test does. A provider
without a Runner route fails the test.
`OPEN_CODE_RUNNER_ENVIRONMENT_KEYS` is now exported for this test.
- Docs: Added a "Typed credentials" section to
`doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md`. It also documents the ACPX
credential binding that the Runner requires with a Copilot token.
Updated `doc/plugins/PLUGIN_RUNNER_EXECUTION.md`.

## Verification

- `npx vitest run tests/environment-task-release-resume.test.ts
tests/environment-tasks.test.ts` in `packages/plugins/sdk`: 96 passed.
The tests accept each valid shape. They reject mixed sets, two sets,
extra variables, another provider's variable, empty values, NUL bytes,
oversized values, unknown providers and extra fields, on `submit` and on
`resume`. They also check that every published allowlist set parses.
- `npx vitest run
src/services/native-runtime/plugin-runner-credentials.test.ts
src/__tests__/environment-task-runtime.test.ts` in `server`: 79 passed.
The new tests cover each provider's transfer, Grok subscription refusal,
Grok routing refusal, the fail-closed cases without value leaks, harness
matching at submit, and Runner-built Grok requirements
(`runnerProviderLaunchRequirements`) that parse through the task
contract and produce a Grok launch.
- `npx vitest run src/__tests__/plugin-runner-credential-names.test.ts`
in `server`: 4 passed. I broke each list locally to confirm that the
test fails: a renamed SDK key, an SDK provider without a Runner route,
and a dropped name in the Rust sidecar keys, the Rust candidate agents,
the Rust OpenCode keys and the TypeScript OpenCode keys. Each change
failed the test.
- `node scripts/typecheck-layers.mjs` in `server` (includes
`src/__tests__`): no errors.
- `npx vitest run
server/src/services/native-runtime/plugin-runner-probe.test.ts` (from
the root): 15 passed.
`server/src/services/native-runtime/plugin-runner-task.test.ts`: passed.
- `tsc --noEmit` for `packages/plugins/sdk` and `server`: no errors.
- I did not run the full suite locally. CI supplies that coverage.

## Risks

- Low risk for existing plugins. The change only adds a credential type
and two optional settings. A plugin that does not handle `provider-env`
gets a typed value that it does not recognize. Before this change, Core
failed these runs before it called the plugin.
- A plugin must set `PAPERCLIP_ACPX_CREDENTIAL_BINDING` for a Copilot
task. Otherwise the Runner refuses the token. The new doc section shows
the value.
- An OpenCode gateway connection without a key (`auth: none`) still
fails closed, because every value must be non-empty.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, through Claude Code
with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 08:47:18 -08:00
575ef12575 feat(responsibilities): file company skills under responsibilities (#22275)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A responsibility shows the skills of the agents doing its work.
People cannot say which skills a responsibility needs.
> - The maintainer chose to file skills under responsibilities like
projects, tasks and routines.
> - Agents also need to set a skill's catalog details without rewriting
its SKILL.md.
> - This pull request lets people and agents file, move and remove
company skills under a responsibility, and adds `update_skill_details`.
> - The benefit is that a responsibility lists the skills it needs,
beside the skills its agents already hold.

## Linked Issues or Issue Description

Refs #22253, #22261 and #22274; this pull request is stacked on them.

**Subsystem affected**

Database (`responsibility_links`), shared types and validators, server
responsibility service and routes, Dashboard versions, native runner
tools, and the responsibility page UI.

**Problem or motivation**

Skills under a responsibility are only computed from the agents doing
its work. A person cannot record that "Hiring needs interviewing", and
an agent cannot file a skill when it maps work for a person.

**Proposed solution**

Add a nullable `skill_id` to `responsibility_links` and allow it as the
one target of a link. Show filed and computed skills together. Let the
file, unfile, lookup and link routes and tools take skills. Keep filed
skills in Dashboard versions so undo covers them. Add a small live tool
for a skill's tagline and categories.

**Alternatives considered**

Keeping skills computed only, and giving the skill to the agent that
runs the responsibility, was the other option. It was not chosen.
Extending `update_skill` would change its scenario contract, whose mock
needs `markdown`, so the details tool is separate.

## What Changed

- Review fixes: the default responsibility view's Skills "+" adds an
existing skill, removes a filed one, or opens an agent's skills page. A
newest Dashboard version without a skill list is upgraded in place, so
the first skill filed after it is undone. Undo files again, or removes,
only skills the restoring person can see. A read scope without a skill
condition sees no filed skill. `update_skill_details` with only
`skillId` is a no-op, as its schema allows.

- Migration `0499`: add `responsibility_links.skill_id` (FK to
`company_skills`, on delete cascade). Widen the single-target check to
four targets. Add a unique `(responsibility_id, skill_id)` index and a
`skill_id` index. The migration is idempotent and online-compatible (NOT
VALID then VALIDATE, and the indexes are built CONCURRENTLY). It is
listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`.
- Shared: `ResponsibilityLink.skillId`;
`ResponsibilitySkillSummary.linkId` and `computed`; `skillId` in the
link and lookup schemas; `skill` in `RESPONSIBILITY_FILE_ITEM_KINDS`;
optional `skillIds` in Dashboard snapshot links.
- Server: filing checks the reader's skill visibility
(`skillAccessService`). The detail merges filed and computed skills into
one card each. Lookup accepts `skillId`. Dashboard capture, restore and
summary include filed skills. A version without `skillIds` leaves filed
skills as they are when it is restored.
- Tools: `file_under_responsibility`, `remove_from_responsibility` and
`find_responsibilities_for` accept `kind: "skill"`. `get_responsibility`
returns `linkId` and `computed` for each skill. Add
`update_skill_details` (tagline and categories).
- UI: the responsibility page has "Add existing skill". A filed skill
shows "Filed here" and a remove button. Computed skills keep their
holders.

## Verification

- Server: the `responsibilit*` tests (new: file, find, move and unfile a
skill; one card for a skill that is both filed and computed; undo of a
filed skill from the Dashboard history), `skill-tools.test.ts` (new
`update_skill_details` cases), `responsibility-tools`, `routine-tools`,
`routines-routes`, `paperclip-runner-tool-authority`, `runner-api` and
`openapi-routes`: 24 files, 2723 tests passed.
- UI: responsibility page and component tests (16 files, 158 tests,
including a new filed-skill test) pass. `pnpm --filter @paperclipai/ui
typecheck` passed.
- `@paperclipai/db` `check:migrations` passes and reports the override.
The snapshot drift test passes. `tsc` passes for `db`, `shared` and the
server layers.
- Runner: catalog and semantic-tool tests (317) pass. All generator
checks pass.

## Risks

- The migration changes a check constraint on `responsibility_links`.
The replacement accepts every row that the old check accepted. It is
swapped as NOT VALID and then validated. The previous release never sets
`skill_id`.
- Deleting a company skill removes its links (cascade).
- Filed skills follow the reader's skill visibility. A skill the reader
may not see is not listed.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-19 09:30:49 -06:00
b7319eb581 feat(routines): let agents list and change their person's routines in plain terms (#22274)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Routines are recurring work that an agent runs on a schedule,
webhook or request. People file them under their responsibilities.
> - #22253 and #22261 let agents read and change areas and
responsibilities. Agents still have no dedicated routine tools, and they
can change only routines assigned to themselves.
> - When a person says "move my weekly digest to Monday mornings" or
"map my routines to my responsibilities", the agent must use the generic
API, write cron, and it is refused for the person's routines.
> - This pull request adds `list_routines` and `update_routine`. It also
lets an agent change routines that the person its run works for is
responsible for.
> - The benefit is that an agent can find routines under no
responsibility, file them, and change their schedule in plain terms.
Each change is a routine revision the person can restore.

## Linked Issues or Issue Description

Refs #22261 (phase 2; this pull request is stacked on it), #22253 (phase
1).

**Subsystem affected**

Server (routine routes, native runner tool authority), the native runner
tool catalog, and the `paperclip` skill.

**Problem or motivation**

An agent can list and change only its own routines through the generic
API. It cannot change the schedule of a routine its person is
responsible for. It must write cron expressions, and it cannot find the
routines that are filed under no responsibility.

**Proposed solution**

Make the existing scenario operation `list_routines` live, with filters
for words, status, agent, project, responsible person, a responsibility
and "unfiled". Add `update_routine` for the title, instructions, agent,
project, status, priority, a plain `schedule`, and filing under a
responsibility. Let an agent manage a routine when the person its run
works for is responsible for it. Handing the routine to another agent
needs that person to have `tasks:assign`.

**Alternatives considered**

A new `routines` permission grant would add a setting with no current
need. Routine revisions already give an undo path, so this change adds
no new history.

## What Changed

- `list_routines` is now live (it was a scenario-only mock) and keeps
its scenario mock.
- Add `update_routine`. A `schedule` such as `{ frequency: "weekly",
dayOfWeek: "monday", time: "09:00", timezone }` maps to the routine's
schedule trigger through the existing routine-offer cadence helpers.
`schedule: null` turns the schedule off. `fileUnder` files the routine
under a responsibility.
- Add `server/src/services/routine-tools.ts`. It calls the routine
routes with the run token. Fields and schedule go in one `PATCH
/routines/:id` (one transaction and revision, guarded by
`baseRevisionId`); filing follows, and if it fails after the routine
changed the tool reports what was saved. Each part reports its own undo:
the routine revision, or the Dashboard history for filing.
- `PATCH /routines/:id` takes `schedule` (`{ cronExpression, timezone?
}` or `null`) and replaces the schedule under the routine row lock, so
racing changes leave one schedule. Without a zone it keeps the current
schedule's, else uses the person's saved time zone, else answers 422.
- `GET /companies/:companyId/routines` filters `q`, `status`,
`assigneeAgentId`, `responsibleUserId`, `responsibilityId` and
`unfiled=true` in SQL.
- In `routes/routines.ts`, an agent may manage routines assigned to
itself or routines its person is responsible for. On a routine the agent
does not run, taking it from another agent (or handing it on), turning
it on, changing its schedule (also through the trigger routes), starting
a run, or restoring a revision requires that person's `tasks:assign`, as
for a person. The person must have directed the run (see #22261).
`routine.updated` activity records `onBehalfOfUserId`.
- The routine and responsibility routes share one rule,
`agentPersonInCompany` in `routes/agent-person-authz.ts`.
- The catalog keeps each live action's `requiredClaims` (`list_routines`
is `routines:read`).
- The routine tools stay visible when Responsibilities is off. Only
`responsibilityId`, `unfiled` and `fileUnder` need it.
- Update `search_api` tool pointers, the routines reference, the skill
section, the chat directive, the operation-groups spec and the pinned
counts.

## Verification

- Server: `routine-tools.test.ts` (new), `routines-routes.test.ts` (new:
an agent changes its person's routine, hands it off only with
`tasks:assign`, and is refused for another person's routine), the
`responsibilit*` tests, `paperclip-runner-tool-authority` and
`runner-api`: all pass. The first test in `routines-routes.test.ts`
(Fireflies) times out at 15 s in my sandbox on a cold import, with or
without this change. It passes with `--testTimeout=60000`.
- Server `node scripts/typecheck-layers.mjs`: passed.
- Runner: catalog, semantic-tool, eval, scenario and permission-policy
tests (316) pass. All generator checks pass.

## Risks

- Medium-low. Agents get write access to their person's routines. The
route still checks the company, the routine revision guard and
`tasks:assign` for a hand-off. The person can restore any earlier
routine revision.
- `list_routines` changes from a scenario-only mock to a live tool, so
the reconciled catalog counts change.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-19 09:30:03 -06:00
DottaandClaude Opus 5.5 c0aa7751f8 feat(responsibilities): let agents change areas and responsibilities for their person, with undo (#22261)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People keep their areas and responsibilities in Paperclip
(experimental, `enableResponsibilities`). #22253 lets agents read them.
> - Agents can still only propose. They cannot rename or archive an
area, change an active responsibility, let one go, or move work between
responsibilities in one step.
> - A person who says "file my weekly routines under Sales" must then do
it by hand.
> - The maintainer chose this rule: an agent may make each change that
its person could make, the person can undo it from the Dashboard
history, and a permanent delete still needs the person's yes on a card.
> - This pull request adds eight change tools and the server rules
behind them.
> - The benefit is that an agent maps work to responsibilities for the
person, and the person keeps control through undo and the delete card.

## Linked Issues or Issue Description

Refs #22253 (phase 1; this pull request is stacked on it).

**Subsystem affected**

Server (responsibility and area routes, Dashboard versions, native
runner tool authority), shared validators and Dashboard types, the
native runner tool catalog, the `paperclip` skill, and the Dashboard
history UI.

**Problem or motivation**

An agent that works for a person can only propose areas and
responsibilities. It cannot change active ones, archive or delete them,
or move filed work in one step. Unfiling needs link IDs. There is no way
for the person to undo an agent's change.

**Proposed solution**

When an agent acts for the person its run works for (`onBehalfOfUserId`)
and that person directed the run (the run's own task is one they made,
such as their chat, or their own message woke it; parent tasks do not
count), apply that person's own rights: their own responsibilities, or
anyone's for a company owner or admin; areas for any active member. Save
each such change as a version of the person's Dashboard, so the existing
history and Undo line show it under the agent's name. Make a permanent
delete wait for an accepted confirmation card that the agent posted. Add
`file` and `unfile` routes that take items by kind and ID, with an
optional move.

**Alternatives considered**

One card for every change, or propose-only. The maintainer chose "act
for me, with undo". A separate undo journal would need a new table and a
new UI; Dashboard versions already have both. A new version cause needs
a check-constraint migration; this change uses `change_set` with no
`interactionId` instead.

## What Changed

- Add tools `create_area`, `update_area`, `delete_area`,
`create_responsibility`, `update_responsibility`,
`delete_responsibility`, `file_under_responsibility` and
`remove_from_responsibility`. They are standard-mode only and hidden
while Responsibilities is off.
- Add the writer rule for agents in `routes/agent-person-authz.ts`
(`agentPersonInCompany`, `agentActsForResponsibilityOwner`), used by
`routes/responsibilities.ts` and `routes/responsibility-areas.ts`. The
person must have directed the run: a person can name anyone as a task's
responsible person, and a run can also work for a routine's person or
the company default, so those runs do not borrow that person's rights.
An agent with no such person keeps the old propose-only rules.
- The response names only the version of the responsibility's owner (for
area changes, the acting person's), and `X-Paperclip-Dashboard-Owner`
says whose history holds it; tools return `dashboardOwnerUserId`. A
change Undo cannot reverse (the name, description, order or un-archiving
of an area other people's proposed or active responsibilities share)
saves no version and is not reported as undoable.
- `/file` without `moveFrom` follows the `/links` rule, so
`create_responsibility` with `fileItems` works for a proposal.
- A title-only create is allowed for an acting agent. It is active by
default and records the agent in `proposedByAgentId`.
- Add `responsibility-agent-changes.ts`. In one transaction that holds
each affected owner's Dashboard lock, it saves the dashboard before the
change (when a person changed it by hand), runs the change on services
bound to that transaction, and saves the result as a `change_set`
version by that agent. The response names the version in
`X-Paperclip-Dashboard-Version`. The tools say Undo only when a version
was saved.
- Dashboard snapshots now also hold each responsibility's filed items,
description, "run by" agent, evidence, health override and note, and
each area's description (optional fields). Restore puts them back. Older
versions still restore as before. A matching older version is upgraded
in place, so the first change after it compares and undoes in full.
- Create and update accept `areaTitle`. The server finds or makes that
area in the same transaction, so a refused change leaves no area. Agent
creates reuse a live responsibility with the same title under a lock, so
retries and parallel calls make no copies.
- Add `POST /api/responsibilities/:id/file` (`items`, optional
`moveFrom`: an ID or `all`) and `POST /api/responsibilities/:id/unfile`
(`items`), each in one transaction.
- Agent deletes need `confirmationId`: an accepted
`request_confirmation` card that this agent posted, with target key
`responsibility-delete:<id>` or `responsibility-area-delete:<id>`.
Someone who may manage the owner must accept it, on the card or by a
verified chat reply. The tools post the card (named after the item),
return `awaiting_confirmation`, and delete on the next call after
acceptance.
- `DELETE /api/responsibility-areas/:id` accepts
`moveResponsibilitiesTo`. Area changes and deletes save versions for
everyone whose Dashboard shows the area.
- Agent activity details now include `onBehalfOfUserId`. The Dashboard
history shows "<agent>, for you" for these versions.
`ResponsibilityDashboardVersionSummary` now has `interactionId`.
- Update the OpenAPI registry and metadata, the `search_api` tool
pointers, the skill, the API reference, the chat directive, the
operation-groups spec, and the pinned counts and SDK source pins.

## Verification

- Server: `vitest run` of the `responsibilit*` tests,
`responsibility-tools.test.ts`, `openapi-routes`, `runner-api*`,
`paperclip-runner-tool-authority` and `task-markdown` all pass. New
tests cover these cases: acting-agent create, update and undo; file,
move and undo; the rule for a person who may not manage the owner;
delete only after an accepted card, including a verified chat reply;
area delete with move and a version for the other owner; undo of a
health override, an area description and an un-archived empty area;
upgrade of an older version; an area by title that a refused change
rolls back; parallel creates.
- UI: `DashboardEditModule.test.tsx` and
`responsibility-dashboard.test.ts` passed. `pnpm --filter
@paperclipai/ui typecheck` passed.
- Server `node scripts/typecheck-layers.mjs` passed.
- Runner: catalog, semantic-tool, eval, protocol-action and
permission-policy tests (284) passed. All generator checks pass.
- I did not run the browser e2e suite locally.

## Risks

- Medium. Agents that work for a person can now change and archive that
person's responsibilities and areas without a card. Every change is a
Dashboard version that the person can restore, and a delete still needs
the card.
- Undo restores titles, sentences, areas, order, status, filed items,
description and the "run by" agent. It does not restore a bar or metric
change; those stay in the metric history.
- A restore of a version from before this change does not touch filed
items, which matches the earlier behavior.
- Undo of a rename or archive of an area that other people also use
keeps its name, as the existing restore rule does for shared areas.
- Manual edits do not take the Dashboard lock. A person's edit that
commits while an agent's change is open can appear in that agent's
version.
- Changes to proposed or let-go responsibilities are not on a Dashboard,
so they cannot be undone there. The tools say so.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 09:29:11 -06:00
DottaandClaude Opus 5.5 ffcd49f6aa feat(responsibilities): let agents read a person's areas and responsibilities (#22253)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People keep their areas and responsibilities in Paperclip
(experimental, `enableResponsibilities`). Agents chat with them about
this work.
> - Agents have no dedicated tool to read a person's responsibilities.
The only path is the generic `search_api` and `call_api` fallback.
> - That path needs the person's user ID, and nothing in a chat tells
the agent that responsibilities exist. When the feature is off, every
route returns a 404 that looks like "nothing there".
> - So a chief-of-staff agent asked the person "what are your
responsibilities, or where are they written down?" when Paperclip
already had them.
> - This pull request adds three read tools, makes the owner default to
the person the run works for, and tells agents in chats how to use the
tools or that the feature is off.
> - The benefit is that an agent reads and maps a person's
responsibilities itself. This is phase 1 of 4. Later pull requests add
the change tools for areas, responsibilities, routines and skills.

## Linked Issues or Issue Description

**Subsystem affected**

Server (responsibility routes, native runner tool authority, chat
prompt), native runner tool catalog, and the `paperclip` skill.

**Problem or motivation**

An agent in a chat cannot find a person's responsibilities. It has no
dedicated tool. The list route requires `ownerUserId` from an agent, and
the chat does not give the agent that ID. The chat prompt does not
mention responsibilities. When the experiment is off, the 404 does not
tell the agent that the feature is off. The agent then asks the person
to list their responsibilities again.

**Proposed solution**

Add three optional native tools: `list_responsibilities`,
`get_responsibility` and `find_responsibilities_for`. They call the
existing routes with the run's agent token, so all route checks and the
activity log stay the same. The owner defaults to the person the run
works for. The routes return a `responsibilities_disabled` code while
the feature is off. The tools are hidden while the feature is off, and
the chat prompt then says that the feature is off.

**Alternatives considered**

Better `search_api` guidance alone still needs the user ID and many
calls. A new read service in the tool authority would copy the route
permission rules. The tools use the routes instead.

## What Changed

- Add `list_responsibilities` (areas with their responsibilities,
responsibilities with no area, totals, and company work filed under no
responsibility), `get_responsibility` (one responsibility with its filed
items, link IDs, skills and latest status report), and
`find_responsibilities_for` (where a project, task or routine is filed;
a task can be a `PAP-12` identifier).
- Add `server/src/services/responsibility-tools.ts`. It calls the routes
over HTTP with the run token and returns compact results with page
links.
- When an agent sends no `ownerUserId`, the responsibility list, create,
summary, automation and dashboard routes use the person the run works
for (`onBehalfOfUserId`). A run that works for no person still gets a
422 with a clear message.
- The nine "not enabled" checks now throw one shared error. The 404 and
its message do not change. The body now also has `code:
"responsibilities_disabled"` and a remediation line. One
`RESPONSIBILITIES_OFF_GUIDANCE` text is the remediation, the tool error
and the chat prompt note.
- `list_responsibilities` with `areaId` counts only that area. Archived
areas are listed only on request. When only proposals are hidden, the
tool says so.
- `services/paperclip-api-loopback.ts` holds the tools' authenticated
API call; the project tools use it too.
- Show the tools only while the instance has Responsibilities on. A
hidden tool that is called by name returns "Areas and responsibilities
are turned off".
- In standing chats, add an "Areas and responsibilities" prompt block:
use the tools before you ask the person, or say that the feature is off.
- `search_api` results for these routes now name the dedicated tools.
- Add an "Areas and responsibilities" section to
`skills/paperclip/SKILL.md` and to `references/api-reference.md`. Add
one line to the chief-of-staff persona.
- Register the tools in the runner catalogs. Regenerate the semantic
contracts, the action catalog, protocol coverage, the operation-groups
spec and the capability inventory. Update the pinned counts.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/catalog src/semantic-tools src/tools src/eval/workflow-evals.test.ts
src/protocol-actions src/evals/native-execution.test.ts
src/drivers/acpx/permission-policy.test.ts`: 276 passed.
- All runner generator checks pass: protocol manifest, capability
contract, capability inventory, protocol coverage, semantic contracts,
semantic action catalog, operation groups, replay goldens, workflow
traceability.
- Server: `vitest run` of `responsibilit*` route tests,
`responsibility-tools.test.ts` (new), `task-markdown.test.ts`,
`task-context*`, `paperclip-runner-tool-authority.test.ts`,
`runner-api.test.ts`, onboarding first-task tests and
`user-facing-wording-guidance.test.ts`: 20 files, 2620 tests passed.
- `node scripts/typecheck-layers.mjs` for the server: passed.
- New tests cover these items: the owner default for an agent and the
422 without a person, the `responsibilities_disabled` code on every
route, the tool projections, an identifier lookup, the "turned off"
mapping, the tool visibility switch, and the chat prompt block.

## Risks

- Low risk. The routes keep their permission checks. Agents can still
only read and propose in this phase.
- An agent that sends no `ownerUserId` now gets the responsible person's
list instead of a 422. Agents that name an owner see no change.
- The chat prompt gets one more short block in standing chats.
- The flag is pinned when a turn opens. If an admin changes it during a
turn, the tool list changes on the next turn. The routes always check
the current setting.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 08:52:42 -06:00
Nicky Leach 1c67687998 feat(runner): start a released Runner again as the next run (#22288) 2026-11-18 23:17:30 -08:00
170e1b0c07 feat(runner): pin Codex 0.170.0 and Agent SDK 0.3.328 so Fable 5.2 and GPT-6.2 run (#22278)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude and Codex agents from one shared
provider pack. The pack pins an exact Claude Agent SDK (which bundles
Claude Code) and an exact Codex CLI.
> - #22222 added Claude Fable 5.2 (minimum Claude Code 2.1.321) and
GPT-6.2 Sol and Astra (minimum Codex 0.169.1). The pack still bundles
Claude Code 2.1.286 and Codex 0.160.0.
> - Because of this, Paperclip Runner stops these models with a named
version error, and native remote Codex accepts only `>=0.149.0
<0.161.0`.
> - This pull request moves the provider pack to Agent SDK 0.3.328
(Claude Code 2.1.328) and Codex 0.170.0. It moves every constant that
depends on these pins, and the remote Codex window to `<0.171.0`.
> - The benefit is that Fable 5.2 and GPT-6.2 run on Paperclip Runner
after a sandbox image with this pack is promoted.

## Linked Issues or Issue Description

Refs #22222 (adds the models and the version floors; this pull request
makes them runnable on Paperclip Runner).
Refs #18488 (moves only the Codex pin to 0.166.0 and the window to
`<0.167.0`; it is not updated since November 1 and has conflicts with
`master`).
Refs #18486 (GPT-6.1 Terra needs Codex 0.165.0; Codex 0.170.0 satisfies
it).

**Overlap with #18488.** #18488 changes the same Codex pin locations.
This pull request uses a newer pin (0.170.0) and also moves the Claude
runtime. If this pull request merges first, #18488 has no remaining
change. I did not change the #18488 branch.

## What Changed

- Claude runtime: `@anthropic-ai/claude-agent-sdk` 0.3.286 → 0.3.328
(Claude Code 2.1.286 → 2.1.328). This changes the root `pnpm.overrides`,
`pnpm-workspace.yaml`, the ACPX `claude` profile, and the qualified
Claude runtime in `installation-integrity.ts`, with new executable
digests for Linux x64, Linux ARM64, macOS ARM64, and macOS x64.
- Codex runtime: `@openai/codex` 0.160.0 → 0.170.0. This changes the
runner dependency, the `codex-acp@1.6.2>@openai/codex` override, the
`codex-acp` patch dependency line, the ACPX `codex` profile, and the
qualified Linux x64 runtime with its new executable digest.
- The ACP bridges do not change: `claude-agent-acp` 0.73.0 and the
patched `codex-acp` 1.6.2. Their `commandDigest` values and profile
versions do not change.
- Generated declarations: `generated-profiles.ts` and
`generated_acpx_profiles.rs` are regenerated with
`generate:acpx-profiles`.
- Remote native Codex window: `REMOTE_CODEX_MAXIMUM_VERSION_EXCLUSIVE`
0.161.0 → 0.171.0. The minimum stays at 0.149.0. Images with Codex
0.149.0 through 0.170.x are accepted.
- Codex company observability: the qualified-version check moves to
`codex-cli 0.170.0`. The `otel` configuration keys that Paperclip writes
(`exporter`, `trace_exporter`, `metrics_exporter`) are present in the
0.170.0 configuration schema. The `OtelExporterKind`,
`OtelHttpProtocol`, and `OtelTlsConfig` definitions are identical to
0.160.0.
- Daytona runner image probes (`Dockerfile`, `Dockerfile.release`)
expect `codex-cli 0.170.0` and `2.1.328 (Claude Code)`. The remote npm
install spec in `server/src/adapters/registry.ts` is
`@openai/codex@0.170.0`.
- Tests: fixtures and assertions that mirror the pins use the new
versions. The window tests accept 0.161.0 through 0.170.99 and reject
0.171.0. The ACPX gate tests now show that Fable 5.2 and GPT-6.2 are
admitted. A new test mocks an older bundled runtime and keeps the named
version-gap errors covered. The fallback test shows that GPT-6.2 stays
on 0.170.0 and still falls back on 0.160.0.
- Opt-in qualification scripts
(`scripts/tests/native-cleanup-paginated-codex.mjs`,
`native-provider-startup-codex.mjs`) require Codex 0.170.0.
- Documentation: `docker/daytona-runner/README.md`,
`doc/architecture/paperclip-runner-compatibility.md`,
`doc/agent-observability.md`, `docs/deploy/environment-variables.md`,
and the deferred item in `doc/adapter-model-audit-2026-11-18.md`.
- `pnpm-lock.yaml` is not part of this pull request. The refresh bot
regenerates it on `master`, as for the previous pin refreshes (#14942,
#14993).

## Verification

Sources, checked on 2026-11-19:

- npm registry: `@openai/codex@0.170.0` (published 2026-11-18T03:45Z)
and `@anthropic-ai/claude-agent-sdk@0.3.328` (published
2026-11-18T23:13Z, `claudeCodeVersion` 2.1.328). Both are the `latest`
tags.
- [Codex releases](https://github.com/openai/codex/releases) 0.161.0
through 0.170.0. The notes record no app-server protocol break. 0.169.0
keeps project configuration disabled for `codex exec` in untrusted
directories; Paperclip's CLI path already uses `--skip-git-repo-check`,
and the native path uses `app-server`.
- [Claude Code
changelog](https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md)
2.1.287 through 2.1.328. See "Risks" for the headless changes that I
reviewed.

Digests and package shape:

- I downloaded each platform archive with `npm pack`, which verifies
registry integrity, and computed the SHA-256 of the executable. As a
method check, the same command on the 0.3.286 Linux x64 archive gives
the digest that is on `master` (`fe503f65…`). The Codex 0.160.0 archive
also gives the `master` digest (`12eb3e81…`).
- The extracted executables report `codex-cli 0.170.0` and `2.1.328
(Claude Code)`.
- The Codex 0.170.0 Linux x64 archive no longer contains
`codex-resources/zsh/bin/zsh`. Paperclip does not reference that path.
- The Agent SDK 0.3.328 `optionalDependencies` have the same platform
set as 0.3.286. The Codex 0.170.0 `optionalDependencies` use the same
`npm:@openai/codex@<version>-<platform>` aliases.
- Bridge link check: I installed
`@agentclientprotocol/claude-agent-acp@0.73.0` with an override to Agent
SDK 0.3.328 and imported its entry module. Node linked all of the named
SDK imports, and the import completed.

Real Codex qualification (no model turn, no credentials):

- `PAPERCLIP_TEST_CODEX_BINARY=<installed 0.170.0 Linux x64 binary> node
--import ./server/node_modules/tsx/dist/loader.mjs
scripts/tests/native-cleanup-paginated-codex.mjs` passed: `green: true`,
with the expected red control and paginated `thread/resume` through the
real 0.170.0 `app-server`. The installed binary digest matches the
pinned digest.

Focused suites (Linux x64, Node 25.9.0, local `pnpm install
--no-frozen-lockfile`):

- `cd packages/paperclip-runner && pnpm run typecheck:typescript`
passed, including `check:acpx-profiles`.
- `npx tsc --noEmit` passed in `packages/adapters/codex-local` and
`packages/adapters/claude-local`.
- `packages/adapters/codex-local`: 625 passed.
`packages/adapters/claude-local`: 419 passed, 1 skipped.
- `node --test`: `acpx-codex-package-contract` 9 passed,
`controlled-evals/controller` 77 passed, `scripts/acpx-patch-packaging`
26 passed, `scripts/__tests__/grok-public-install-sandbox` 27 passed.
- `server`: `acpx-qualification`, `codex-runtime-compatibility`, and
`codex-model-fallback` 110 passed. `native-session-executor` 1541
passed, 1 failed. The failure (`compares export process compatibility …
db.transaction is not a function`) also fails on unchanged `master` on
this host.
- `packages/paperclip-runner` vitest (`installation-integrity`,
`qualified-profiles`, `codex-security-config`, `native-execution`,
`native-backend-factory`): 283 passed, 25 failed. All 25 failures are in
`installation-integrity.test.ts` with "ACPX provider lifetime guardian
failed to start". The same 25 tests fail on unchanged `master` on this
host (master fails 30 here).

Rust (toolchain 1.97.1 from `rust-toolchain.toml`, Linux x64):

- `cargo test -p paperclip-runner-core --lib`: 658 passed, 2 failed, 4
ignored. `warm_run_attachment_reuses_the_provider_ingress_listener` also
fails on unchanged `master` on this host.
`reaping_stops_a_dead_runners_provider_and_its_escaped_tool_job` is
flaky here: on this branch it passed 2 of 3 reruns.
- `cargo test -p paperclip-runner-core --test acpx_warm_attachment
--test native_provider_backend`: 7 and 45 passed.
- The new and changed saved-session tests pass:
`a_session_saved_by_the_previous_agent_runtime_is_refused_by_name`,
`unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class`,
and `a_settled_session_saved_by_the_previous_acpx_release_continues`.
`rustfmt --check` passes.
- `server`: `native-session-resume.test.ts` and
`acpx-qualification.test.ts` 110 passed, 2 skipped. Both new resume
cases run and pass.

Not run here:

- `native-provider-startup-codex.mjs`, because it needs a compiled
`paperclip-runnerd`. The trusted CI runner workflow covers startup and
session-resume probes.
- The Docker image build and any live model turn. A live Fable 5.2 and
GPT-6.2 test follows the staging image rollout.

## Saved sessions from the previous release

This change keeps the ACPX bridge command digests and profile versions.
It changes `agentRuntimeVersion` in the `claude` and `codex` ACPX
profiles. That field is part of the saved session identity. The previous
release saved sessions with Agent SDK 0.3.286 or Codex 0.160.0. This
build refuses those saved sessions. It does not migrate them.

- **Next run (the usual case):** the saved provider descriptor names the
old runtime. `rebindNativeSessionCheckpoint` does not match it, so the
next run starts a fresh provider session in the same workspace with the
complete handoff prompt. No run fails. The earlier provider conversation
does not carry over.
- **A run in flight at the deploy (settled or mid-turn):** the runner
restore refuses the saved descriptor with the permanent class
`provider_state_incompatible`. The server reports it as
`native_provider_state_incompatible` and stops automatic retries. The
runner opens no provider and keeps the saved state bytes and the
workspace. The next run on the task starts a fresh provider session.
- **Native Codex (`codex_app_server`) and OpenCode** sessions do not
record this field, so their resume decision does not change. After a new
image is promoted, native Codex resumes a saved thread with Codex
0.170.0 instead of 0.160.0. This pull request does not test that
cross-version thread resume.

New tests:
- `acpx_provider_backend.rs`:
`a_session_saved_by_the_previous_agent_runtime_is_refused_by_name`
covers settled Claude and Codex sessions and a Claude turn interrupted
mid-run by the deploy.
`unrestorable_saved_sessions_keep_strict_bindings_and_name_their_class`
has a new `agent_runtime_release` case. Each case checks the named
class, that no provider session opens, and that the saved bytes do not
change.
- `native-session-resume.test.ts`: "a session saved by the previous
Claude or Codex ACPX runtime" checks that a same-release session
resumes, and that a session saved with Agent SDK 0.3.286 or Codex
0.160.0 starts fresh in the same workspace.

## Risks

- Medium. Every Paperclip Runner Claude and Codex run uses the new
executables. The digests, generated profiles, image probes, and window
must agree. They agree locally, and CI runs the Rust and runner suites
again.
- Claude Code 2.1.287 through 2.1.328 change some headless defaults. I
checked each one against Paperclip:
- 2.1.302: `-p` and Agent SDK sessions with no permission mode start in
`default` mode on third-party providers. Paperclip always sets a mode
(`bypassPermissions` or `default`), so this does not apply.
- 2.1.299: MCP allow rules now match only the whole server name.
Paperclip already writes `mcp__<server>__*` and exact
`mcp__paperclip__<tool>` rules.
- 2.1.302: fork subagents are on by default in `-p` and the Agent SDK,
as in the terminal. This pull request keeps the upstream default and
does not set `CLAUDE_CODE_FORK_SUBAGENT=0`. Reviewers can ask to keep
them off.
- 2.1.299 and later: stdio MCP connections negotiate MCP protocol
2026-07-28 first and fall back to the older protocol for servers that do
not answer.
- Agent SDK 0.3.328 is less than one day old. If a regression appears,
revert this pull request to return to 0.3.286 and Codex 0.160.0.
- Sandbox images built before this change still carry Codex 0.160.0 and
Claude Code 2.1.286. They stay accepted. On those images, Fable 5.2 and
GPT-6.2 keep the existing named errors and fallbacks until a new image
is promoted.
- Until the refresh bot updates `pnpm-lock.yaml` on `master`, a local
`--frozen-lockfile` install fails, as with the previous pin refreshes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude, model ID `claude-opus-5-5`, through Claude Code with
tool use (shell, file edit, `gh`) and code execution. The context window
and reasoning mode are not reported in this session.
- The registry archives, digests, release notes, configuration schemas,
and changelog were read directly. No model output was used as a source
for those facts.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 20:45:18 -08:00
DottaandClaude Opus 5.5 07d9316218 fix: restore master's typecheck and runner SDK input pin (#22242)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - #22235 added a runner regression test for a turn lost to a mid-run
`runnerd` restore
> - That test reads `startTurn.mock` inside the initializer of
`startTurn` itself
> - The CI test-program typecheck rejects this with TS7022 and TS7024,
so master fails "Typecheck + Release Registry"
> - This pull request counts calls in a local variable instead
> - The benefit is that master typechecks again, and the test behaves
exactly as before

## Linked Issues or Issue Description

Refs #22235, Refs #22234

**What happened?**
After #22235 merged, `ci / Typecheck + Release Registry` failed on
`packages/paperclip-runner/src/native-session-runtime.test.ts(724)` with
TS7022 ("'startTurn' implicitly has type 'any'...") and TS7024.

**Steps to reproduce**
1. Run the CI typecheck lane (`typecheck test programs`) on master after
#22235.
2. See TS7022 and TS7024 at line 724 of the test.

**Expected behavior**
The test program typechecks.

## What Changed

- The `startTurn` mock in the new test uses a local call counter instead
of reading its own `mock.calls`.
- `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the pin for
`packages/shared/src/app-definitions/github.json`, which #22234 changed
without updating it. The Runner support suite fails on master without
this.

## Verification

- `npx tsc -p packages/paperclip-runner/tsconfig.test.json --noEmit`
reports no errors in the test file.
- `npx vitest run src/native-session-runtime.test.ts` in
`packages/paperclip-runner`: 301/301 pass.

## Risks

- Low risk. This changes a test and regenerates a pin with the
repository script (`node tests/runner-e2e/source-sdk-public-inputs.mjs
<sha> --write`). `source-sdk-public-inputs.test.mjs`: 13/13 pass.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 16:49:12 -06:00
DottaandClaude Opus 5.5 d668eeb660 fix(runner): keep Claude MCP bearer tokens out of process arguments (#22206)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for work
> - The runner starts Claude through ACPX and the Claude ACP agent.
> - The runner gives Claude HTTP MCP servers: the task tool bridge and hosted connectors. Each server has a bearer token.
> - The Claude Agent SDK writes these servers, with their headers, into the inline `--mcp-config` argument of the Claude CLI.
> - Thus a `ps aux` or `ps -ef` command in the sandbox shows the bearer tokens. Tool logs then keep the tokens.
> - This pull request sends a `${VAR}` reference in the header. The token goes into the launch environment of the agent.
> - The benefit is that process lists do not show MCP bearer tokens.

## Linked Issues or Issue Description

**What happened?**
An agent ran `ps aux` in its sandbox. The output showed the Claude CLI process with an `--mcp-config` argument. That argument had an `Authorization: Bearer <token>` header for each HTTP MCP server. The tool log kept the full output.

**What did you expect to happen?**
Process arguments must not contain credentials. A process list must not show MCP bearer tokens.

**Steps to reproduce**
1. Start a Claude agent on the runner with the task tool bridge or a hosted connector.
2. In the agent, run `ps -ef`.
3. Look at the Claude CLI line. The `--mcp-config` JSON contains `"Authorization":"Bearer ..."`.

**Root cause**
`openQualifiedAcpxRuntime` sent the literal `Bearer <token>` header in the ACP `mcpServers` list. The Claude ACP agent gives this list to the Claude Agent SDK. The SDK always serializes it into `--mcp-config <json>` on the CLI command line.

## What Changed

- For the Claude ACPX profile, each HTTP MCP header now has the value `Bearer ${PAPERCLIP_ACPX_MCP_BEARER_<n>}`.
- The runner puts each token in the Claude launch environment under the same name. Claude Code expands `${VAR}` in `--mcp-config` when it connects to the server.
- Other ACPX agents keep the literal header. Their CLIs do not have a known `${VAR}` expansion.
- Added a regression test. Updated one exact launch-environment assertion for the new variable.

## Verification

- Failing before the fix: `npx vitest run src/drivers/acpx/acpx-runtime-adapter.test.ts -t "MCP bearer tokens out of the CLI argv"` (in `packages/paperclip-runner`). It failed with `expected '[{"type":"http","name":"paperclip",…' not to contain 'bridge-secret'`. The projected header had the literal synthetic token.
- Passing after the fix: `npx vitest run src/drivers/acpx/acpx-runtime-adapter` — 7 files, 136 tests passed.
- Manual check of the expansion: I ran the pinned Claude CLI (agent SDK 0.3.286) with `--mcp-config` and a header `Bearer ${SYNTH_MCP_BEARER}` against a local HTTP server. The server got the expanded synthetic value.
- `tsc --noEmit -p packages/paperclip-runner`: no new errors. The only errors are in `src/eval/workflow-harness.ts` and occur on `master` too, because my local install did not include the eval kernel package.
- The full `src/drivers/acpx/` folder has other failures in this sandbox (`EACCES` on `/proc/self/fd` spawn, directory fsync). The same failures occur on `master` without this change.

## Risks

- Medium-low risk. If a future Claude CLI stops the `${VAR}` expansion in `--mcp-config`, MCP calls fail authentication. They do not leak the token. The regression test pins the header shape.
- The token is now in the Claude process environment. Only the same user can read `/proc/<pid>/environ`. Other credentials are already in this environment. Shell tools that Claude starts also inherit the environment, so `env` output can still show the token. Log redaction must continue to cover bearer values.
- Tokens that already appeared in logs are not revoked by this change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context to this change
- [x] I have specified the model used (with version and capability details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before requesting merge

Related: #13572 removes bearer tokens from curl arguments in the agent helper scripts. This pull request covers the Claude CLI MCP configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)



Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 15:58:46 -06:00
DottaandClaude Opus 5.5 06750cb18e fix(runner): continue a turn lost to a mid-run runnerd restore (#22235)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents in a managed sandbox run under the Paperclip runner
(`runnerd`). When `runnerd` exits mid-run, the transport relaunches it
in the same execution, and the runner restores the same conversation
> - An agent can fill the sandbox disk with its own work. `runnerd` then
cannot write its state and exits. #22068 added a disk reserve so the
relaunch succeeds. Its stated goal was that "the agent's turn can
continue and see the full disk"
> - The relaunch now works, but the turn still dies. The runner reports
the in-flight turn as `provider_turn_lost_on_restore`. The runtime sends
its single same-conversation continuation only for a session recovered
at startup under a control-plane restart claim. A relaunch inside the
live execution has neither, so the run fails as
`native_provider_terminal_failed`
> - The generic transient retry then starts on a fresh sandbox, redoes
the same install, fills the disk again, and ends in board-owned
"Recovery needed"
> - This pull request lets the execution that started a turn send the
existing one-time continuation when the runner reports that same turn
lost on restore
> - The benefit is that the agent keeps its conversation and workspace
and sees the full disk in its own tools, instead of the run dying and
retrying into the same wall

## Linked Issues or Issue Description

Refs #22068

**What happened?**
On staging, two agent tasks on the Paperclip monorepo each failed on
every attempt (5 failed runs in total). Each run used a fresh managed
Daytona sandbox with 10 GiB of disk. The agent's own `df` showed `/` at
10G of 10G with 236K free. `node_modules` used 3.2G and
`.paperclip-runtime` used 6.1G: about 3.0G of provider pack and a 2.8G
pnpm store under the session data directory. `runnerd` exited with
`failed to create private durable state temporary file: No space left on
device (os error 28)`. The transport relaunched it in about 20 s, and
runnerd restored its durable session. Then the runner reported
`turn.failed {code: provider_turn_lost_on_restore, recoverable: true}`,
and the run failed as `native_provider_terminal_failed`. Retries on
fresh sandboxes hit the same failure. The tasks ended in "Recovery
needed" with `provider_failure_meaning_unverified`.

**Steps to reproduce**
1. Run an ACPX or Codex agent through `paperclip_runner` in a managed
sandbox.
2. Have the agent fill the disk during a turn, or end `runnerd` in any
other way mid-turn while the server keeps running.
3. The transport relaunches `runnerd`, and the runner reports the turn
as lost on restore.
4. The run fails instead of continuing the turn.

**Expected behavior**
The execution that started the turn sends one continuation turn in the
restored conversation, exactly as it does after a server restart. The
agent continues from its recorded progress and sees the full disk in its
tools.

**Environment**
Managed Daytona sandbox (fleet snapshot, 4 CPU, 8 GiB memory, 10 GiB
disk), `paperclip_runner` native runtime with Claude through ACPX.

## What Changed

- `packages/paperclip-runner/src/native-session-runtime.ts`: the
execution records the turn id of the turn it starts.
`resumeInterruptedTurn` now accepts two owners of a turn lost on
restore:
- a session recovered at startup with the control-plane restart claim,
for the checkpointed active turn (unchanged), or
- a session this execution opened itself, for the turn this execution
started (new).
- All other guards stay: one continuation per run, the provider
allowlist, no compaction or goal runs, no pending runtime requests, and
the snapshot must show that same turn as restart-interrupted.
- `native-session-runtime.test.ts`: a new regression for an ACPX session
that loses its own turn to a mid-execution `runnerd` restore, plus a
negative case for a lost turn this execution never started.

## Verification

- Red before the fix: `npx vitest run
src/native-session-runtime.test.ts` failed `continues its own turn once
when runnerd is restored mid-execution` with
`NativeProviderTerminalFailure: Provider turn failed`, the same failure
seen on staging.
- Green after the fix: `native-session-runtime.test.ts` and
`native-session-runtime.provider-turn.test.ts` pass (337/337). The
existing `requires the control plane restart claim` case still refuses a
recovered session without the claim.
- `pnpm verify` (affected selection): see the CI results on this PR.
- Live acceptance: after deploy, retry the affected staging tasks and
confirm that a mid-turn `runnerd` restore leads to one continuation turn
in the same sandbox instead of `native_provider_terminal_failed`.

## Risks

- **Second executor.** The continuation can start only from the
execution that opened the session and started the turn. It is not used
for a recovered or adopted session without a claim. It runs at most once
per run (`restartContinuationStarted`). The runner emits this code only
after it proves the old provider process ended.
- **Run starts.** This starts no new runs. It adds at most one turn to a
run that would otherwise fail and schedule a fresh-sandbox retry. During
a crash loop, `MAX_UNSTABLE_RUNNER_RESTORES` in the transport still
fails the transport.
- **Disk still full.** If the continuation fills the disk again, the
second lost turn finds `restartContinuationStarted` set, and the run
fails as it did before. The worst case is the current behavior.
- **Not in this PR.** Paperclip's own footprint (about 3G of provider
pack per sandbox, and a per-session pnpm store) and classifying a
disk-full failure so it does not retry into the same wall. Both are
follow-ups.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, browser, and a code-tracing subagent).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 15:55:16 -06:00
DottaandPaperclip ee68098584 fix(runner): keep current task requests readable in OpenCode traces (#21584)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Company traces let operators compare each request with its reply.
> - OpenCode limits each text field to 8 KiB.
> - A fresh session can put old history before the current request.
> - The limit can remove the current request from the trace.
> - This change selects the current task text first and links the
retained submission.

## Linked Issues or Issue Description

Refs #18098 and #18699. Neither merged change selects the current task
request.

**What existing behavior does this improve?**

OpenCode company traces can contain only part of the old session
history.

**Current behavior**

The exporter clips the full model envelope from its start. A sampled
trace has a useful HOLD reply but a request that ends inside old
history. A truncation flag is not proof that the task failed.

**Proposed behavior**

Select the task prompt before the text limit. Remove a valid leading
generated handoff packet from the excerpt. Link the full retained run
event through the existing API.

[Source
pattern](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/e6d8da04-ca3e-4bd9-bd90-737f7717a9e1).

### Hypothesis and thesis for a fix

The baseline exporter spends its text budget on the envelope and old
history. The new regression confirms this cause: the received OTLP input
loses a short current request after a long handoff. Select the current
request in the OpenCode projection before redaction and clipping. Keep
storage and access checks with their existing owners.

## What Changed

- Select task identity and prompt for known native envelopes. Put
pending user messages first in continuation excerpts.
- Omit only a valid leading generated handoff packet. Keep plain text
and unknown formats on the existing path.
- Add a relative run-events reference under the prompt capture switch.
Readers must sign in to the source instance and page to the matching
turn.
- Keep raw tool argument values excluded. Keep redaction, opt-in,
capture switches, company access and size limits.
- Add exporter, proxy, installed-runtime and private-run retrieval
tests. Fix the proxy test's session-directory lookup so it skips the
output-custody directory.
- Document the excerpt and full-event retrieval path.

## Verification

- Baseline `6b6c94a4ea8f3d1b58767d67a7a2fc4ceaecda9c`: the new
long-handoff regression failed because OTLP input did not contain the
current request.
- Same regression passes after the fix. Exporter, redaction and bundled
proxy suites: 39 tests passed.
- `PAPERCLIP_OPENCODE_QUALIFY=1 pnpm --dir packages/paperclip-runner
exec vitest run
src/drivers/opencode/company-observability-runtime.test.ts`: 2 passed.
Real installed OpenCode 1.18.34 and real HTTP OTLP receiver; local SSE
model fixture. Both capture settings pass. The model receives the full
original submission.
- Private-run API suite: 31 tests passed on base
`340157d1d6ec01ad0fa55970e148c6cd332bbcb1` with these changes.
Authorized readers retrieve a complete request over 8 KiB through
pagination. Other readers receive 404.
- Runner TypeScript source and test checks and TypeScript build passed.
`git diff --check` passed.
- `pnpm verify --explain` returned exit 2 and required broader CI. No
full local-suite pass is claimed.
- Initial harness attempts used the wrong Vitest working directory,
lacked built plugin-sdk, selected the wrong proxy fixture directory, or
expected 403 instead of the API's 404. These are not red regression
results. The corrected checks above pass.
- Review follow-up `dfe995a77f3eb3280a6b081b4df9428fac85f540` removes
added code comments required by CONTRIBUTING.md. Exporter + installed
OpenCode runtime suites reran: 30 tests passed, with the local SSE model
and real HTTP OTLP receiver. `git diff --check` passed. The review
thread is resolved. Greptile passed on this exact revision with
confidence 5/5 and no actionable findings. Final-head typecheck, build,
Runner and E2E checks passed; server test shards and the amd64 Runner
image build remain in progress. No hosted model call or deployed
recovery is claimed.

## Risks

Known envelope excerpts change shape from serialized envelope text to
readable task text. The handoff selector depends on the current
generated packet format; unknown framing falls back to the full prompt
excerpt. Long current requests still clip. Full-event access depends on
source-instance permission and retention. Export remains best effort.
The new reference grants no access and includes no credential. This PR
is unmerged and not deployed.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use and code execution.
The exact serving variant and context size are not exposed to this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Final verification (2026-11-14): exact head
`dfe995a77f3eb3280a6b081b4df9428fac85f540` has 60 successful checks, one
neutral security scan, and five conditional skips (cache write,
publication/attestation, incremental selection, Storybook paths).
Applicable typecheck, build, general and serialized tests, Runner tests,
E2E and both Runner image architectures passed.
[CI](https://github.com/paperclipai/paperclip/actions/runs/43572758306),
[Runner
images](https://github.com/paperclipai/paperclip/actions/runs/43572758233),
[Greptile](https://github.com/paperclipai/paperclip/runs/132380487679).
Greptile 5/5, no unresolved actionable findings. Unmerged and not
deployed.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-18 15:08:04 -06:00
DottaandClaude Opus 5.5 1c556d5340 fix(native): never rest a task blocked on the agent's own blocker (#22140)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run ends with one semantic report. A `paperclip_block`
report names a blocker, its owner and the unblock action
> - The status arbiter binds a task-wide blocker. When the owner is not
a person, it binds the blocker to the same agent and wakes that agent
> - The dispatcher refuses those self-wakes
(`chat_control_recovery_proof_unresolved`), because only continuation
and retry effects are proof for a native wake. The task then rests
blocked with no owner, no wake and no Retry
> - #21959 continues a system blocker in a fresh session, but only for
the reason codes `environment_unavailable` and `tool_unavailable`. A
restart-chaos rerun showed the same dead end with the reason code
`tool_output_failure`
> - This pull request makes sure that a blocker that the agent reports
for itself always has a next owner: the automatic environment lane, one
bounded self-continuation, or a person with a Retry that works
> - The benefit is that no task rests blocked with nobody to act, and no
path wakes the agent in a loop

## Linked Issues or Issue Description

Refs #21959

**What happened?**

A native OpenCode run (DeepSeek) said that every tool call returned the
text "cryptic". It reported `paperclip_block` with owner `{kind:
system}`, scope `task_wide`, reason code `tool_output_failure`, and an
unblock action about the tool-output failure. The run succeeded with
disposition `blocked`. The arbiter bound the blocker to the same agent
(`task_wide_blocker_bound`). The `notify-owner` wake coalesced, and the
`blocker-owner` wake was skipped as
`chat_control_recovery_proof_unresolved`. The task rested blocked
(`needs_attention`, 0 blockers), with no Retry and no wake. An earlier
rerun showed the same shape with reason code `environment_unavailable`,
which #21959 fixed for those two codes only.

**Expected behavior**

A blocker that the agent reports for its own tools or environment is a
harness problem. Paperclip continues it automatically, with a bound. A
blocker that the agent says it owns gets one fresh session. A blocker
that needs a person goes to a person, with a clear prompt and a Retry
that starts the task again. Nothing is left with nobody.

**Steps to reproduce**

1. Run a native task. Make the agent call `paperclip_block` with owner
kind `system`, scope `task_wide` and a reason code other than
`environment_unavailable` or `tool_unavailable` (for example
`tool_output_failure`).
2. See the decision `task_wide_blocker_bound` with the agent as the
unblock owner.
3. See both owner wakes refused, and the task blocked with no Retry.

**Paperclip version or commit**

`master` at 9e1a85fb46.

## What Changed

- `native-environment-blocker.ts`: a system-owned blocker now uses the
bounded fresh-session environment lane for any reason code, except the
person-only codes `permission_denied`, `credential_missing`,
`policy_restricted` and `budget_exhausted`. An agent-owned blocker with
`environment_unavailable` or `tool_unavailable` also uses it. The lane
is unchanged: at most 2 retries on the shared failure counter, then a
board blocker.
- New `native-agent-blocker.ts` and a new arbiter branch for a task-wide
blocker that is not person-owned, from the current owner's own succeeded
run (not a chat, not a review):
- Owner kind `agent`: one fresh-session continuation
(`native_agent_blocker_continuation`). It uses the shared failure
counter with a maximum of 1, so it never runs after another automatic
retry in the same chain. A paused task does not get it.
- All other cases, and a repeated blocker: the blocker goes to the board
with the agent's unblock action (`native_agent_blocker_needs_person`).
There is no self-wake.
- `native-provider-capacity-retry.ts`, `status-decision-committer.ts`,
`native-run-finalizer.ts`: materialize the continuation through the
existing fresh-session retry lane (`forceFreshSession`, `retryOfRunId`,
30 s delay, refuses a stopped run). Both new reason codes need the
provider-failure owner recheck under the issue lock. The finalizer reads
the pause hold for every blocked result.
- `heartbeat.ts`, `run-dispatch/adapters/postgres.ts`: the continuation
gets the same dispatch ownership checks as the environment retry.
- `task-interruption-presentation.ts`, shared
`TaskInterruptionPresentation.userAction`: a task that is blocked by a
`native_agent_blocker_needs_person` decision on its latest run shows
`userAction: "unblock"` with `canRetry: true`. `ExecutionBlockerNotice`
shows "The agent stopped and needs your help to continue. Read its last
comment, do what it asks, then Retry." with a Retry button. The inbox
label is "The agent needs your help". The assigned-work invariant reads
it as `assignee_declared_block`.
- `blocked-question-answer.ts`: an answer to the agent's open question
still wakes it after the new hand-off.
- Runner `completion-result.ts`: the "Paperclip continues this blocker
automatically" message now matches the server's environment lane.
- I did not change `readChatControlNativeParent`. The fix removes the
self-bound blocker wakes instead of accepting them as proof, so a
self-wake loop cannot form.

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-agent-blocker.integration.test.ts`
(new, embedded Postgres, real executor, finalizer and committer). Red
before the fix (3 of 3 fail with `task_wide_blocker_bound`), green
after:
- The t205 shape (system owner, `tool_output_failure`) gets one
fresh-session retry, no owner wakes, and the retry run that reports done
moves the task to `done`.
- An agent-owned blocker continues once. The same blocker in the
continuation goes to the board, with no further retry and no owner
wakes.
- A system blocker with `credential_missing` goes to the board with no
automatic retry.
- An external-owned blocker goes to the board. The task shows
`userAction: "unblock"` and `canRetry: true`. `requestTaskRetry` accepts
the Retry and `resumeTaskRetry` starts exactly one new run of the task.
- `npx vitest run src/services/native-runtime/status-arbiter.test.ts`:
new case for the once-only continuation, the hand-off, the pause hold,
the chat exclusion and the non-owner exclusion.
- `cd ui && npx vitest run
src/components/ExecutionBlockerNotice.test.tsx`: new case for the help
prompt and its Retry.
- `cd packages/paperclip-runner && npx vitest run
src/contracts/completion-result.test.ts`: new cases for the conflict
message.
- Related suites pass (11 files, 523 tests): arbiter corpus,
task-interruption presentation and query, assigned-work invariant,
blocked-question answer, liveness escalation, run-dispatch adapter,
failed-turn continuation, tool-receipt limit and provider capacity.
- `tsc --noEmit` passes for `server`, `packages/shared`,
`packages/paperclip-runner` and `ui` (only the existing generated
Storybook catalog errors). `scripts/check-token-gates.mjs` is clean.
- `pnpm verify` exits 2: the source SDK pin refresh is a global CI
input, so required CI supplies the broad coverage.

## Risks

- More automatic runs: an agent-owned blocker can add 1 run per chain,
and a system blocker with a new reason code can add up to 2 (the
existing lane). Both share the failure-retry counter, so a chain never
gets more than the existing maximum. The company-wide automatic-run
brake admits these runs like any other recovery run. Judge the change by
company-wide automatic run starts per hour and their success rate.
- A blocker that only a person can resolve, but that the agent labels as
its own, now takes one extra run (about 30 s later) before it reaches a
person. This is intended.
- Board-owned blockers now show an attention card and a Retry, where
before they named the agent and showed nothing. A Retry starts a new run
of the task; the person decides when.
- Previous release: no persisted shape changes. The new reason codes are
new values in existing text columns. Decisions saved by the previous
release keep their saved effects.
- Chats keep their current behavior, because the person in the chat
answers directly.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (code search, editing, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 02:43:28 -06:00
43b2d8be22 feat(dashboard): add an experimental areas and responsibilities dashboard (#22069)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The dashboard is the first page a person sees in a company
> - The classic dashboard shows agents, runs, costs and recent tasks
> - It does not show what the person is responsible for, or if that work
goes well
> - People think in areas, such as Fundraising or Customers, and in the
things they own in each area
> - This pull request adds a dashboard built on areas and
responsibilities, behind an experimental setting that is off by default
> - The benefit is that a person sees what they own, how it is going,
and what agents do for it, on one page

## Linked Issues or Issue Description

No public issue exists for this work. The description follows the
feature request template.

**Subsystem affected**

Dashboard, navigation, the task conversation, routines, and the instance
experimental settings.

**Problem or motivation**

The classic dashboard counts agents and tasks. It does not say what a
person owns or if it goes well. People who run a company with agents
want to see their areas, the responsibilities in each area, the bar each
one must meet, and the agents and routines that serve it.

**Proposed solution**

A person owns responsibilities. Each responsibility belongs to an area
and has a bar, a number to reach. Projects, tasks and routines link to a
responsibility, and the page shows its status from those links. An agent
can interview the person and draft the dashboard. The person accepts or
declines each change, and any change can be undone from a saved version.

**Alternatives considered**

Add the same data to the classic dashboard. That page is a fixed set of
cards for company activity, so a second, person-centred page is clearer.
The setting lets a person pick one of the two.

**Roadmap alignment**

This is an experimental feature behind `enableResponsibilities`. It is
off by default and changes nothing for a person who leaves it off.

## What Changed

- Add the setting "Areas & Responsibilities dashboard"
(`enableResponsibilities`, off by default). On, Dashboard in the sidebar
opens the new page, and the classic dashboard link is not shown in the
sidebars or the command palette. Off, nothing changes.
- Add tables and migrations 0492 to 0497 for areas, responsibilities,
links to work, metrics, status reports, saved dashboard versions, and
model-call records for summaries.
- Add the API: areas, responsibilities, links, bars, metrics, status
reports, summaries, runs, automation, and the dashboard conversation
with versions and undo. Each route checks company access and writes to
the activity log.
- Add four task conversations with an agent: about a bar, about an
automation, about a routine, and about the whole dashboard. The agent
proposes a change. Nothing applies until a person accepts it.
- Add standing checks: routines that an agent carries out on its
heartbeat and reports.
- Add the Dashboard page: search, counts, areas that open and close, a
primary New area button, and a message box at the bottom of the page.
Add the responsibility page with its bar, metrics, work and routines.
- Add the Paperclip skill sections and runner API entries that let an
agent use these routes. The runner skill inventory is regenerated (166
to 174 rows).
- Keep the new runner API tests in their own file so
`runner-api.test.ts` stays at 985 lines.
- Onboarding is not changed. The onboarding changes that build on this
work are a separate pull request.

## Verification

- Type checks pass for `packages/db`, `packages/shared`, `server` (all
layers) and `ui`.
- `pnpm --filter @paperclipai/db check:migrations` passes. A new
`drizzle-kit generate` run produces no further change.
- Tests for every changed file pass: 26 server test files (3,009 tests),
116 UI test files (1,991 tests) and 28 shared test files (432 tests).
- The runner capability inventory checks pass.
- I started the app on a local database that already held earlier data.
Master's new migrations and these migrations applied without error. With
the setting on, the Dashboard and a responsibility page loaded with no
console or server errors. The More menu and the command palette show one
Dashboard.
- I did not run `pnpm verify`, the full local suite, or the browser
suites. Required CI covers them.
- The agent conversations are tested with simulated agents. No real
model has run them.

## Risks

- Medium. The change is large, and the setting is off by default. With
it off, the new routes refuse requests and the UI is unchanged.
- Migrations only add tables and columns. Two replace a CHECK
constraint, and one builds an index `CONCURRENTLY`. These two have the
`online_compatible` override with a reason in the file, and the second
one is listed in `MIGRATIONS_WITH_TOP_LEVEL_STATEMENTS`.
- Master can add migrations before this merges. The migrations then need
new numbers.
- `responsibilities.ts` (1,663 lines), `responsibility-summaries.ts`
(1,239 lines) and one Storybook fixture file (1,305 lines) are new files
over 1,000 lines. They each keep one cohesive unit together: the service
for one table family, the model-call path for summaries, and the
fixtures that the stories share. No existing file crosses 1,000 lines.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude: Claude Fable 5.2 (`claude-fable-5-2`) wrote most of
the change, with Claude Sonnet 5.5 (`claude-sonnet-5-5`) for the final
steps. Both used tool use, code execution and browser checks.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Scott Tong <scott@scottsmbpm5max.lan>
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-17 20:45:44 -08:00
Nicky LeachandPaperclip f05b90508a refactor(core): remove retired compatibility paths (#22055)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core manages chat authorization, AI connection defaults, navigation,
and the plugin host.
> - Core still writes old default rows and checks an Agent Chat flag
that always reads true.
> - It also carries a desktop rail that cannot open and private aliases
with canonical equivalents.
> - This pull request removes those paths and moves their tests onto the
paths Core uses.
> - The result is fewer Core database operations and less code to
maintain.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Core AI default writes, Agent Chat execution and delivery, private
plugin host and Runner setup, and board navigation.

**Subsystem affected**

Cross-cutting. Core server, board UI, private Runner helpers, and a
repository guard script.

**Current behavior**

The server writes provider-scoped and per-method AI defaults, then reads
only the provider-scoped rows. It reads settings to decide whether to
block Agent Chat, although every settings view enables it. The desktop
sidebar renders expanded while still tracking requests to open and close
a retired icon rail. The agent client retries conflicts that the server
already resolves when there is exactly one active match.

**Proposed behavior**

Write provider-scoped defaults directly. Use the existing Agent Chat
authorization and retry checks without the redundant settings reads.
Render the current sidebar directly and let the API report true agent
reference conflicts.

**Reason and benefit**

A personal non-custom connection save or explicit default selection
sends one fewer SQL statement. It also avoids the provider-default write
that the legacy trigger performed when the old statement inserted or
updated a row. A successful task-to-chat send or receipt replay avoids
two transaction settings SELECTs when the singleton row already exists.
Conversation comment, wake release, and retry paths also avoid a
transaction settings read when they previously entered the gate. Savings
in other settings paths depend on cache hits. Core-to-plugin RPC and
plugin-to-Launcher HTTP traffic stay the same.

**Breaking changes**

The public plugin SDK, host navigation, and organization-switcher fields
retain their existing interfaces and behavior. The settings API still
accepts and reports the Agent Chat compatibility key. This change needs
no schema migration or rollout ordering.

**Additional context**

Refs #10610 for prior reporting on the retired sidebar rail. Refs #21756
for adjacent Coordinator plugin slots, which remain a separate change.

## What Changed

- Stop writing per-method AI defaults. Preserve the provider-scope
upsert and first-account behavior. Keep the existing tables and
migration trigger.
- Remove Agent Chat checks that read a constant true value. Keep
conversation ownership, task scope, session, and request-retry checks.
- Remove unused lock-owner fields and private plugin lifecycle, secret
extraction, Gmail connector, PRP setup, heartbeat resolver, and ACPX
aliases.
- Use a single API request for agent references. Remove retired sidebar
rail state, hooks, and branches across the active and production
snapshot layouts. Preserve desktop resizing, mobile drawer controls,
contextual navigation, and public plugin slot inputs.
- Remove unused FeedCard and AgentDetail inputs and the test-only Git
push matcher alias.
- Keep the cleanup that erases historical plaintext chat retries from
browser storage.

Production source in the touched files is 106,800 → 105,633 nonblank
lines, down 1,167. The production Git diff adds 251 physical lines and
removes 1,480, down 1,229. Those figures exclude tests, fixtures, docs,
and Storybook. The entire Git diff adds 482 physical lines and removes
2,325, down 1,843.

## Verification

- Merged master at `b56f240f45` into the PR branch as `f73fea4009`. The
five sidebar files retain the expanded navigation cleanup and master’s
explicit session-replay masking contract. The merge also preserves
master’s account-repair recovery guard and GitHub star-prompt behavior.
- Focused UI Vitest run passed 361 tests across 11 files. This covers
analytics privacy and replay masking, sidebar and layout, GitHub star
prompt, and agent detail.
- Focused server Vitest run passed 45 tests across three files. This
covers restored configuration handling, conversation recovery, and Agent
Chat authority and receipt replay.
- `pnpm --filter @paperclipai/ui typecheck`, the Git push guard and its
14 tests, and `git diff --check origin/master...HEAD` passed.
- `pnpm verify --base origin/master` classified the Git-push guard as a
global CI input and returned exit 2. GitHub CI supplied the broader
coverage.
- GitHub CI and Runner image checks passed on `f73fea4009`. Both
required checks, `ci / verify` and `ci / e2e`, passed. [CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261752)
· [Runner image
workflow](https://github.com/paperclipai/paperclip/actions/runs/44181261413)
- Standard Greptile reported 5/5 with no actionable findings or open
review threads. [Greptile
review](https://github.com/paperclipai/paperclip/pull/22055#issuecomment-6959078925)
- The optional Superagent Security Scan finished neutral with an
inconclusive workflow result and zero annotations. The final rollup had
61 successes, five skips, and one neutral, with no failed or pending
checks. [PR
checks](https://github.com/paperclipai/paperclip/pull/22055/checks)

## Risks

Agent Chat authorization and retry are sensitive to small query changes.
The final diff keeps the conversation agent field and all existing
authorization and idempotency checks. The tests cover revoked defaults,
legacy migration reads, bound-listener authority, and mobile and desktop
navigation. Installations that still run software old enough to read the
per-method AI table will no longer see new defaults in that table.
Current Core reads the provider-scoped table.

## Model Used

OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with source inspection and
shell-based tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 14:00:14 -08:00
DottaandClaude Opus 5.5 73dc33c620 fix(runner): keep a disk reserve so a full sandbox disk does not strand the runner (#22068)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents in a Paperclip-managed sandbox run under the Paperclip runner
(`runnerd`), which keeps durable state on the sandbox disk
> - An agent can fill that disk with its own work. In one case the agent
installed a Rust toolchain and a second checkout with dependencies on a
10 GiB sandbox
> - `runnerd` then could not write its state and exited. Each relaunch
tried to upload its launch files to the full disk and failed, 83 times
in 7 minutes
> - The run ended as a generic provider failure that asked a person to
inspect it, and Retry repeated the same failure
> - This pull request keeps a small disk reserve that belongs to
Paperclip. Recovery releases it to relaunch the runner, and stops at
once with a named code when nothing is left to free
> - The benefit is that the agent's turn can continue and see the full
disk in its own tool output, instead of the run dying with it

## Linked Issues or Issue Description

**What happened?**
An agent in a managed Daytona sandbox filled the 10 GiB disk. `runnerd`
exited with `failed to persist bounded diagnostic: No space left on
device (os error 28)`. Runner process recovery then retried for the full
420000 ms reconnect grace. Every attempt failed with
`.paperclip-upload-<id>.tar.gz: create: ... no space left on device`.
The run failed as `native_provider_terminal_failed`, and the recovery
asked a board user to verify that the provider stopped.

**Steps to reproduce**
1. Run an agent in a managed sandbox and have it fill the disk (for
example a Rust toolchain plus a full dependency install).
2. `runnerd` exits when it next writes its state.
3. Watch recovery retry the relaunch upload until the grace expires.

**Expected behavior**
Paperclip relaunches its runner on a disk the agent filled, so the turn
continues and the agent sees the full disk in its tools. When the runner
truly cannot start, recovery says so at once with a named cause instead
of retrying for 7 minutes.

**Environment**
Managed Daytona sandbox (fleet snapshot, 4 CPU, 8 GiB memory, 10 GiB
disk), `paperclip_runner` native runtime.

## What Changed

- New `server/src/services/native-runtime/remote-runner-disk-reserve.ts`
owns the reserve: hold/release scripts, the full-disk classifier, and
`createRemoteRunnerDiskReserve`. A release waits for an in-flight hold,
and no hold starts after a release.
- Both remote runner launchers (agent runs and Dot) get a reserve path
in managed sandboxes only:
`<workspace>/.paperclip-runtime/paperclip-runner/disk-reserve`.
Workspace saves and Git already exclude `.paperclip-runtime`. SSH hosts
get no reserve.
- After the first launch's identity is confirmed, the launcher holds 256
MiB in the background, only while 2 GiB more is free. It is off the
startup path.
- A relaunch releases the reserve before it stages. A launch that hits a
full disk while staging releases the reserve and stages once more.
- Release tries twice and reports `released`, `not_held` or `failed`.
Only `not_held` ends the launch with `sandbox_disk_full`. A release that
cannot run rethrows the original error, so recovery still retries.
- Command-based staging (no `syncIn`) keeps a full disk recognizable
with an `ENOSPC` code instead of a bare
`runner_remote_*_staging_failed`.
- `SANDBOX_DISK_FULL_FAILURE_CODE` and `isSandboxDiskFullFailure` live
in the runner package. Runner process recovery checks the error code and
stops at once instead of retrying for the whole grace. Other staging
failures still retry.

## Verification

- Red before the fix, green after:
-
`server/src/services/native-runtime/remote-runner-disk-reserve.test.ts`:
release-and-restage, release before relaunch, `sandbox_disk_full` when
release frees too little, no retry for a failure that is not a full
disk, hold/release scripts in a real shell, no hold below the free-space
floor. Before the fix, the launch rejected with the exact upload error
from the incident.
- `packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts`
"fails at once when a replacement cannot be staged because the sandbox
disk is full": before the fix it was still retrying after 4 s; after, it
fails with `sandbox_disk_full` after one replacement.
- Kept: "retries a replacement whose staging failed before any process
launched" still passes, so a raw transient staging failure still
retries.
- Updated: "confirms no replacement process exists when staging fails
before the launch RPC" now expects `sandbox_disk_full` with the upload
error as cause. Exit is still confirmed and no launch RPC runs.
- Real sandbox from the same fleet snapshot (Daytona, 10 GiB): hold
created a 256 MiB reserve. The disk was filled to 100%. A 5 MB write
then failed. Release returned `true`, and a 50 MB write then succeeded.
A second release returned `false`. Hold on the now 98% full disk did not
create a reserve.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` passes. `pnpm
verify` result is in a comment below.
- Review round: tests for a lost release RPC, hold/release ordering,
command-based staging on a full disk, the hold after a dispatched first
launch (not on relaunch), and no reserve on SSH. Each launcher-level
test failed with its fix removed.
- Focused suites on the review revision: transport 335/335, launcher
130/130 (filtered), launch assets 35/35, reserve 14/14. Server and
runner typecheck pass.
- Not run: a full product run in which an agent fills the disk and the
relaunched `runnerd` resumes the turn end to end.

## Risks

- Each managed sandbox holds 256 MiB more disk while it has more than 2
GiB free. That is 2.5% of a 10 GiB sandbox.
- A relaunch for any reason releases the reserve for the rest of that
run. The next run's launch takes it again.
- If the agent keeps writing after the relaunch, the disk can fill
again. The second time, nothing is left to release, and the run fails at
once with `sandbox_disk_full` instead of after 7 minutes.
- The run's error code is still `native_provider_terminal_failed`. The
message now names `sandbox_disk_full`. A dedicated failure code and
recovery text are a follow-up.
- `runnerd` loses its own error message when it cannot persist its
diagnostic. This PR does not change the Rust binary.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:45:20 -06:00
DottaandPaperclip 9911b7052f Expose continuation retry keys and preserve account-repair resumes (#22046)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Runner gives agents a tool to report work and wait for a later
check.
> - That tool requires a stable retry key for each explicit wait.
> - Codex can hide the wait fields when it renders our nested schema.
> - Hidden fields make a correct report harder to construct.
> - This pull request changes the provider schema structure and
preserves its rules.
> - Agents can see the required fields before they submit a report.

## Linked Issues or Issue Description

**What happened?**

The completion tool appears in Codex code mode with `continuation?:
unknown`. An agent can omit the required retry key and receive a
rejected report. The top-level schema was fixed in #21398. The nested
continuation still uses the same problematic structure.

**Expected behavior**

Codex must show the required `kind`, `summary`, and `idempotencyKey`
fields. Invalid reports must still fail validation.

**Steps to reproduce**

Render `PRP_COMPLETION_RESULT_PROVIDER_INPUT_SCHEMA` with the public
Codex code-mode renderer at `b271854568d6c9282e999d6f36e5e85f447d027d`.
Its `render_map` function processes `allOf` before sibling properties.
The conditional-only branch renders as `unknown` and hides those
properties.

Source:
https://github.com/openai/codex/blob/b271854568d6c9282e999d6f36e5e85f447d027d/codex-rs/code-mode-protocol/src/json_schema_types.rs#L127

**Paperclip version or commit**

Baseline: `01451fae5b25f784956e9932f12031dc194ac247`.

The first complete CI browser attempt also exposed an intermittent
account-repair race. Repair restores the task and resolves its recovery
action, then a sweep can reclassify the same old failure and block it
again before the queued reply starts. The repair in this PR honors that
exact persisted resolution while holding the existing task lock. A later
failed run still escalates.

## What Changed

- Express the single provider continuation condition as direct `if` and
`then` fields. This is equivalent to its previous `allOf` wrapper.
- Keep the canonical output schema unchanged. Keep required retry keys
and case handoff IDs.
- Add a discovery guard, a 60-case provider/canonical validation
comparison, and authenticated HTTP bridge checks.

### Hypothesis and thesis for a fix

The nested `allOf` hides fields in the tested Codex renderer. The
equivalent direct condition lets that renderer reach the object fields.
This belongs in the existing provider compatibility schema. It changes
no controller policy, company access, saved state, or retry identity.
The experiment proves field visibility. It does not prove which renderer
served a historical request or measure model omission rates.

- Honor a resolved/restored configuration recovery for the same company,
task and failed run before re-escalating. Dispatch checks and recovery
for new failures remain intact.
- Add a PostgreSQL regression covering repeat sweeps, a new failure
identity, an unresolved recovery, and another task's repair. This
belongs in the existing locked escalation function; no new state or
subsystem is introduced. `service.ts` was already over 1,000 lines.

- Define the handoff condition once and explicitly compose both
canonical and provider schemas from it, removing the implicit
one-element `allOf` assumption raised in review.

## Verification

- Built and ran the exact pinned public Codex renderer in a disposable
Rust crate. The complete baseline schema renders `continuation?:
unknown`. The changed schema renders a nullable object with required
`kind`, `summary`, and `idempotencyKey` fields.
- The new discovery regression fails on the baseline. The same test
passes with the fix.
- Runner: `pnpm exec vitest run src/contracts/completion-result.test.ts
src/contracts/tool-input-variants.test.ts
src/contracts/provider-tool-schema-shape.test.ts
src/drivers/runner-tool-bridge.schemas.test.ts
src/drivers/runner-tool-bridge.continuation.test.ts` — 48 tests pass.
- Runner: `pnpm exec tsc -p tsconfig.json` and `node
scripts/generate-semantic-contracts.mjs --check` pass.
- Server: the selected `task-monitor.integration.test.ts` test for
unfinished monitor waits passes with embedded PostgreSQL. It rejects an
unscheduled wait and accepts the saved monitor.
- Server: the selected `issue-monitor-scheduler.test.ts` recovery test
passes with embedded PostgreSQL. It checks persistence, same-key replay,
and one scheduled wake. The fixture's successor run fails its separate
responsible-user setup. This is not proof of successful successor
execution.
- HTTP bridge handler is simulated. It verifies rejection before
dispatch, valid monitor/response-wake/handoff reports, retry identity,
and authentication.
- Recovery regression: baseline 1 failed / 3 passed (the restored task
was blocked again); unchanged test after the fix 4 passed. Together with
recovery state-read tests: 7 passed. Existing configuration-recovery
selection: 4 passed, 108 deselected.
- Final completion schema and authenticated HTTP bridge selection: 33
passed.
- Local real browser journey with disposable embedded PostgreSQL and
ACP/provider fixtures: baseline repeat5 produced 2 failures / 3 passes,
matching CI. Fixed repeat5: 5 passed in 1.8 minutes. The test
independently checks exactly one successful successor, stored connection
binding and persisted reply after reload.
- No full local suite, deployed recovery, or probabilistic model result
is claimed. Fresh CI and review are required for the updated head.

- Review follow-up: 48 focused schema/discovery/HTTP tests pass; `node
scripts/generate-semantic-contracts.mjs --check` passes. Fresh CI/review
required after the refactor.

## Risks

Low runtime risk. The accepted input rules are unchanged. A provider
that handles direct conditionals differently could expose a
compatibility issue. The canonical validator still enforces the output
contract. No migration or saved-session reset is needed. Previously
saved state keeps the same format. Recovery now honors existing
resolved/restored actions only for the exact failed source run, so a new
configuration failure remains eligible for escalation. Company, task,
ownership and dispatch checks remain in force.

## Model Used

OpenAI GPT-6 Codex agent, with reasoning, tool use, and code execution.
The runtime did not expose a more specific model ID.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-17 14:23:19 -06:00
Nicky Leach bc0a236b95 feat(runner): accept a resume-compatible build for a relaunched released Runner (#21976) 2026-11-17 06:15:01 -08:00
DottaandClaude Opus 5.5 41b0f76d75 fix(runner): tell a provider its accepted result is final when it reports again (#21984)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run reports its result with `paperclip_finish` or
`paperclip_block`, and a turn's first accepted report is final
> - When the model reports again with a different result, the Runner
refuses the second report
> - After a committed blocker, the refusal tells the model to stop and
end the turn (#21959)
> - After a committed done, needs_review or yielded result, the refusal
was only "A different semantic result was already committed", so the
model did not know what to do and tried again
> - Each extra call is one more model request, and the turn holds the
agent's slot until it ends; the task's status commits only when the turn
ends
> - This pull request gives every refused later report a clear, final
message: the result is already accepted, stop calling `paperclip_finish`
and `paperclip_block`, put extra notes in the final response, and end
the turn
> - The benefit is that the model ends the turn after one refusal, so
the slot is free sooner and the task completes sooner

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos wave, an OpenCode agent (DeepSeek through OpenRouter)
finished a task, and the Runner accepted its `done` report. The model
then called `paperclip_finish` again with a different report. The Runner
replied only "A different semantic result was already committed". The
model repeated the call about 79 times over 277 seconds. The agent's
slot stayed busy, and the task's completion was late, because the task
status commits when the turn ends.

**Expected behavior**
After a turn commits any result (done, needs_review, yielded or
blocked), each later report with a different result gets a clear and
final instruction: the result is already accepted, so do not call
`paperclip_finish` or `paperclip_block` again, put any extra notes in
the final response, and end the turn. An identical repeat of the
accepted report stays an idempotent success.

**Steps to reproduce**
1. Start a native OpenCode turn that calls `paperclip_finish` with a
done result.
2. In the same turn, call `paperclip_finish` again with a different
summary, or call `paperclip_block`.
3. On master, the tool error is only "A different semantic result was
already committed", with no instruction to stop.

**Paperclip version or commit**
master 54f8068df4

## What Changed

- `packages/paperclip-runner/src/contracts/completion-result.ts`:
replace the bare `PRP_SEMANTIC_RESULT_CONFLICT_MESSAGE` with
`PRP_RESULT_ALREADY_ACCEPTED_MESSAGE`.
`prpSemanticResultConflictMessage` returns it for every committed result
that is not a blocker. The blocker messages share the same stop
instruction, which now names both `paperclip_finish` and
`paperclip_block`, and ask for extra notes in the final response.
- The OpenCode, Codex and Codex ACPX drivers already return
`prpSemanticResultConflictMessage(...)`, so all three get the new text
with no driver change.
- `packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs`:
add a `finish-after-finish` scenario. After the accepted
`paperclip_finish`, the fake provider sends an identical repeat, a
changed `paperclip_finish` and a late `paperclip_block`.
-
`packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts`:
a new test checks that the identical repeat succeeds, that both
conflicting reports get the final message, that only one result is
proposed, and that the turn completes.
- `packages/paperclip-runner/src/contracts/completion-result.test.ts`:
unit cases for done, needs_review, yielded and a missing committed
result, and a check that both blocker messages name both tools and say
to end the turn.
- Not in this change: a hard bound that ends the turn after N refused
reports. In the OpenCode driver the only way to stop a provider turn is
`/abort`, which the driver reports as `turn.cancelled`, not
`turn.completed`. A safe bound needs a new terminal path in each driver,
so it is not "small". This change fixes the cause (the message) first.

## Verification

- Red: with the old message, the 4 new `completion-result.test.ts` cases
and the new OpenCode driver test fail (the refusal text does not contain
the stop instruction).
- Green: `npx vitest run src/contracts/completion-result.test.ts
src/drivers/opencode/opencode-server-driver.test.ts` in
`packages/paperclip-runner`: 2 files, 123 tests pass.
- `npx vitest run src/drivers/acpx/codex-acpx-driver.test.ts
src/drivers/codex`: 28 files, 563 tests pass.
- `pnpm --filter @paperclipai/paperclip-runner typecheck` passes.
- `pnpm verify` selected broad coverage (exit 2) because the shared
OpenCode fixture changed. Required CI supplies that coverage.

## Risks

- Low risk. Only the text of a tool error changes. Result admission,
fingerprints and idempotent repeats do not change.
- Mixed versions during a deploy: the Runner creates this text and sends
it straight to the model. The server does not read or match it (no
server, Rust or test code matches the old string). An old server with a
new Runner gets the new text. A new server with an old Runner keeps the
old text. Neither case changes a stored record or a protocol shape.
- ACPX: no ACPX profile, pin, schema or `protocol/` file changes, so no
provider `commandDigest` or pack contract changes.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 03:53:12 -06:00
DottaandClaude Opus 5.5 22d3f9a03c fix(native): finish a failed warm session's cleanup by itself and continue the task (#21977)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runner runs keep a provider session per task. A follow-up run
can reuse the warm session of the run before it, in a remote sandbox.
> - When a warm follow-up fails, the previous session is closed. The
close saves a backup of the provider session first.
> - The close looked up the backup owner through the task's current tool
authority. The failed follow-up had already released that authority.
Every close attempt failed with `native_harness_storage_owner_missing`,
so the close could never finish.
> - After nine failed attempts, the task's admission gate became
`operator_required`. The gate retires only for a stopped local process
or for the exact run that owns the session. A remote sandbox session
owned by an earlier run never matched. A queued continuation and a Retry
stayed queued forever, and the task showed a Retry button that could not
work.
> - Separately, a remote transport failure could never produce the local
stop proof that the safe-replacement lane waits for. Nothing continued
the task until an unrelated wake arrived 15 minutes later.
> - This pull request makes the close save its backup under its own
owner, retires a cleanup gate when the task's sandbox is confirmed
terminated, and refuses an unprovable remote transport failure at once
so the task continues in a fresh isolated run.
> - The benefit is that recovery finishes by itself, and a Retry that is
shown starts a run.

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos wave, a warm follow-up run on a remote sandbox failed
with `provider_transport_failed` (`PRP command run.attach timed out`).
The previous run's session close then failed on every attempt with
`native_harness_storage_owner_missing`. The cleanup gate became
`operator_required`. The continuation run was created 15 minutes later
and then stayed queued until the wave ended ("queued run kept queued
while the task's previous session finishes cleanup"). The task showed a
Retry banner, but a Retry would wait behind the same gate.

**Expected behavior**
A failed session close settles or is retired by itself with evidence.
The next run of the task starts in seconds. A Retry that is shown starts
a run.

**Steps to reproduce**
1. Run a native task on a remote sandbox with a warm session lifecycle.
2. Make the warm follow-up run fail during startup, after it has reused
the previous session.
3. Watch the previous session's close: every attempt fails with
`native_harness_storage_owner_missing`, and the queued next run of the
task never starts.

**Paperclip version or commit**
master at b2b2c1e60b (wave) and 7685b7a4fc (this branch base).

**Deployment mode**
Hosted server with remote Daytona sandboxes (restart-chaos harness).

## What Changed

- `native-session-executor.ts`: the remote checkpoint resolves its
backup owner from the current scope authority, and falls back to the
backend's own authority epoch when the scope has none. The
controller-persistence `authorityRunId` closure uses the same fallback.
A newer run that took over the session still wins.
- `paperclip-runner` `native-session-runtime.ts`: new
`nativeSessionCleanupRemoteScopes` lists the remote resources whose
quarantined cleanup holds a task. New
`completeTerminatedRemoteNativeSessionCleanupForTask` retires settled
quarantines of the task in one terminated remote resource, whichever
run's session they hold. An in-flight close keeps its entry.
- `remote-execution-termination.ts`: new `terminatedRemoteCleanupScope`
confirms that the company's newest lease on the remote resource is
released with its termination receipt and that no lease on it is still
held.
- `native-cleanup-admission-hold.ts`: queue admission now retires a held
cleanup when its sandbox is confirmed terminated. It logs the retirement
and records a `native.session_cleanup_gate_retired` run event with the
termination receipt. The local stopped-process path is unchanged.
- `native-safe-replacement.ts`: a transport failure on a remote or
non-Codex session can never produce the local stop proof. It is now
refused at once with `provider_failure_meaning_unverified`. The existing
disposition pass then continues the task in a fresh isolated run from
saved work. A local Codex run keeps the existing wait for stop proof.

## Verification

- Red/green, each verified red without its fix:
- `server/src/services/native-runtime/native-session-executor.test.ts` —
"saves a closing session's backup under its own owner after a failed
follow-up released the scope authority". Red:
`native_harness_storage_owner_missing` (the incident error).
- `server/src/__tests__/heartbeat-native-cleanup-admission-hold.test.ts`
(embedded Postgres) — "starts a queued run once the previous session's
sandbox is confirmed terminated, whichever run released it", plus "keeps
the run queued while a different sandbox of the task is terminated".
Red: the queued run stays queued.
- `server/src/services/native-runtime/native-safe-replacement.test.ts`
(embedded Postgres) — "continues a transport failure whose stop can
never be proven locally without waiting for a person" for a remote
sandbox (refused and continued fresh) and a local Codex run (unchanged).
Red: no denial and no hold.
- `pnpm verify` and server and runner typechecks pass locally.

## Risks

- The admission gate now trusts a provider termination receipt on the
newest lease of the same remote resource, not only one bound to the
session's own run. A reused sandbox with a newer held lease is not
retired.
- A remote transport failure no longer waits for a stop proof that
cannot arrive. The fresh continuation is isolated (write authority
revoked, new root) and stays under the existing per-task and per-minute
continuation brakes. Nothing is added before `turn.start`.
- The run.attach timeout itself is not fixed here. The controller held
the attach frame behind a checkpoint flush to the follow-up's writer.
The run's own diagnostics were lost because they were written to the
predecessor's run log. This PR makes the failure recover by itself; the
attach stall needs a separate fix with better diagnostics.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, agentic coding with
tool use and code execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 02:25:02 -06:00
DottaandClaude Opus 5.5 7685b7a4fc fix(native): continue a run that reports a system environment blocker (#21959)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run ends with one semantic report: `paperclip_finish` or
`paperclip_block`
> - An agent can report a system environment blocker, for example when
it thinks its tool results are corrupted
> - Today that blocker binds to the same agent, its wake is dropped, and
the task stays blocked with no Retry
> - In a restart-chaos rerun, an OpenCode agent reported such a blocker,
then recovered in the same turn and did all the work, but its later
`paperclip_finish` was refused with an opaque conflict error
> - This pull request makes Paperclip continue a system environment
blocker automatically in a fresh session, and tells the agent clearly
that a turn's accepted blocker is final
> - The benefit is that a temporary environment problem no longer needs
a person to unstick the task

## Linked Issues or Issue Description

**What happened?**

A native OpenCode run reported `paperclip_block` with owner `{kind:
system, name: execution-environment}`, `reasonCode:
environment_unavailable` and the unblock action "Resume/retry the run".
In the same turn the agent recovered: it saved the task document and
posted the requested comment. Its later `paperclip_finish` failed with
"A different semantic result was already committed". The run succeeded
with disposition `blocked`. The status decision bound a task-wide
blocker to the same agent. Both owner wakes were skipped
(`chat_control_recovery_proof_unresolved`). The task stayed blocked with
no Retry.

**Expected behavior**

A system environment or tool blocker is a harness problem. Paperclip
must continue the task automatically, with a bound, in a fresh session.
A provider that tries to replace an accepted blocker must get clear
guidance, not an opaque conflict.

**Steps to reproduce**

1. Run a native task with an agent that calls `paperclip_block` with
owner kind `system` and reason code `environment_unavailable`.
2. In the same turn, call `paperclip_finish` with a done report.
3. See that the finish is refused with "A different semantic result was
already committed" and that the task stays blocked with no Retry.

**Paperclip version or commit**

`master` at b2b2c1e60b.

## What Changed

- `server/src/services/native-runtime/native-environment-blocker.ts`:
new lane constants and `isNativeEnvironmentBlocker`. A blocker qualifies
when its owner kind is `system` and its reason code is
`environment_unavailable` or `tool_unavailable`.
- `status-arbiter.ts`: for the current owner's environment blocker (not
a chat, not paused), schedule a fresh-session retry
(`native_environment_blocker`). After 2 retries on the shared failure
counter, bind a board-owned blocker
(`native_environment_blocker_exhausted`) with a clear next step. Other
blockers keep their current behavior.
- `native-run-finalizer.ts`: compute the environment-blocker fact from
the accepted result, only for a succeeded run that the task's owner
still holds and that nobody stopped. Both new reason codes require the
provider-failure owner recheck under the issue lock.
- `status-decision-committer.ts`, `native-provider-capacity-retry.ts`:
materialize the retry through the existing fresh-session retry lane
(`forceFreshSession`, `retryOfRunId`, 30 s then 60 s delay, refuses a
stopped run). The retry run carries a liveness instruction that tells
the agent to check saved files, documents and comments before it repeats
a step.
- Runner `completion-result.ts` and the OpenCode, Codex and ACPX
drivers: when a turn already committed a blocker, a later different
report gets `PRP_BLOCKER_ALREADY_REPORTED_MESSAGE`. It says the first
accepted report of a turn is final, tells the agent to list the
completed steps in its final response, and says that Paperclip continues
a system environment blocker automatically. Other conflicts keep the old
text.
- I kept the one-result-per-turn rule. runnerd
(`admit_terminal_tool_authority`, `paperclip/runResult`), the native
runtime and checkpoint replay all enforce it, so a superseding second
result would need a protocol change across the Rust runner, three
drivers and replay. The automatic fresh-session retry closes the gap
without that change.
- OpenCode permission denial after reattach: not a product bug. The
`external_directory` rules come only from the run's working directory
and its instruction root, which stay the same across a hand-off. The
event log shows the same workspace and session paths after both resumes.
Only one call was denied, during the model's burst of garbled tool
calls, which matches the intended workspace confinement.

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-environment-blocker.integration.test.ts`
(new). It runs the real executor, finalizer and committer on embedded
Postgres. Red before the arbiter change (2 of 3 fail), green after.
- An environment blocker schedules one fresh-session retry. The task
stays `in_progress` and shows `retry_scheduled` with no Retry button.
- With the retries spent, the task is blocked with a board-owned unblock
action.
  - A user-owned `credential_missing` blocker still binds as before.
- `cd server && npx vitest run
src/services/native-runtime/status-arbiter.test.ts`: new case for the
bound, pause hold, chat and non-environment exclusions.
- `cd packages/paperclip-runner && npx vitest run
src/drivers/opencode/opencode-server-driver.test.ts -t "accepted blocker
is final"` (new). The fake OpenCode server blocks, then tries to finish
in the same turn. Red before the driver change, green after.
- Focused suites pass: OpenCode, Codex semantic-result and ACPX driver
tests plus runner `src/cli` (26 files, 554 tests); arbiter corpus,
tool-receipt-limit and provider-capacity integration tests (5 files, 198
tests).
- `tsc --noEmit` passes for `packages/paperclip-runner` and `server`.
- `pnpm verify` asks for broad CI coverage because the change touches a
shared runner fixture. Required CI supplies that coverage.

## Risks

- More automatic runs: each environment blocker can add at most 2 runs
in a chain. The retries share the failure counter with the other failure
lanes. The company-wide automatic-run brake admits them like any other
recovery run. Worst case during an outage is 2 extra starts for each
blocked task, and the brake holds them if they keep failing.
- A real, persistent environment problem now takes up to about 90
seconds of retries before it reaches the board. This is intended.
- Previous release: no persisted shape changes. The work assessment JSON
is unchanged, because the fact is computed from the saved result at
finalization. A run finalized by the previous release keeps its saved
decision.
- Agents see a new message when they try to replace an accepted blocker.
The tool descriptions are unchanged, so the prompt size is the same.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (code search, editing, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 22:40:37 -06:00
Nicky LeachandClaude Opus 5.5 fb288f5666 fix(acpx): only treat an error result as a Claude sign-in failure (#21921)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Claude agents run through the Claude ACP bridge (`claude-agent-acp`
0.73.0), which Paperclip patches in `patches/`
> - The bridge reads Claude Code's final result for each turn. If the
result text contains "Please run /login", the bridge fails the turn as a
sign-in failure
> - That check does not look at `is_error`. A successful reply that
quotes the CLI's sign-in hint also fails the turn
> - Agents quote that hint when they report the output of a `claude`
command that they ran. The work is done, but the run fails with "ACP
agent reported a terminal access failure."
> - Upstream fixed this in
[agentclientprotocol/claude-agent-acp#1103](https://github.com/agentclientprotocol/claude-agent-acp/pull/1103)
(commit
[`6e97e07`](https://github.com/agentclientprotocol/claude-agent-acp/commit/6e97e071c581002b43d9556dbd1d1376ebf9516f),
released in 0.81.1), "fix: gate login result text on error state"
> - This pull request ports that upstream change into the 0.73.0 bridge
patch
> - The benefit is that a correct, finished run is not reported as a
failed sign-in

## Linked Issues or Issue Description

Upstream fix:
[agentclientprotocol/claude-agent-acp#1103](https://github.com/agentclientprotocol/claude-agent-acp/pull/1103)
(commit
[`6e97e07`](https://github.com/agentclientprotocol/claude-agent-acp/commit/6e97e071c581002b43d9556dbd1d1376ebf9516f),
released in 0.81.1). This pull request ports it.

Refs #14089. That pull request fixes the same class of problem in the
Claude CLI lane (`execute.ts`). This pull request fixes the ACP lane.

Related: #14093 shows the same error text, but it has a different cause.
This pull request does not fix #14093.

**What happened?**

An agent ran a script that calls the `claude` CLI without credentials.
The CLI printed "Not logged in · Please run /login". The agent finished
its task and quoted that text in its final reply. Claude Code ended the
turn with a successful result (`is_error: false`). The bridge saw
"Please run /login" in the result text and failed the turn with
`auth_required`. Paperclip then failed the run with "ACP agent reported
a terminal access failure."

**Expected behavior**

A successful result completes the turn. Only an error result with the
sign-in hint is a sign-in failure.

**Steps to reproduce**

1. Run a Claude agent on the ACP lane.
2. Make the model's final reply contain the text "Please run /login"
(for example, quote the output of `claude -p hi` without credentials).
3. See that the run fails with "ACP agent reported a terminal access
failure." The new test in this pull request does these steps against a
local API stand-in.

**Agent adapter(s) involved**

`claude_local` on the ACP engine, and the Runner's ACPX Claude profile.
Both use the same patched bridge.

## What Changed

- `patches/@agentclientprotocol__claude-agent-acp@0.73.0.patch`: port
[agentclientprotocol/claude-agent-acp#1103](https://github.com/agentclientprotocol/claude-agent-acp/pull/1103)
(commit
[`6e97e07`](https://github.com/agentclientprotocol/claude-agent-acp/commit/6e97e071c581002b43d9556dbd1d1376ebf9516f),
released in 0.81.1). It has two parts, the same as upstream:
- The `success` result check is now `message.is_error &&
message.result.includes("Please run /login")`. A successful result that
quotes the sign-in hint no longer fails the turn.
- The goal-command check also requires `message.is_error`. A `/goal`
command whose successful result quotes the sign-in hint no longer rolls
the session goal back.
- Both patch comments name the upstream change. All other patch hunks
are unchanged.
-
`packages/paperclip-runner/src/drivers/acpx/acpx-runtime-adapter.claude-login-text.test.ts`:
a new test. It runs the real bridge and the bundled Claude Code against
a local API stand-in. The model's reply quotes the sign-in hint, and the
test expects the turn to complete.

## Verification

- Red: on `master`'s patch, the new test fails with
`{"status":"failed","error":{"message":"ACP agent reported a terminal
access failure."}}`.
- Green: with this patch, the new test passes (`status: "completed"`,
`stopReason: "end_turn"`).
- I ran Claude Code 2.1.286 (the bundled version) with no credentials.
Its real sign-in failure is a result with `is_error: true` and the text
"Not logged in · Please run /login". The guard still catches it. The
separate synthetic sign-in message check is unchanged.
- The goal-command part has no new test here. The upstream change adds a
test for it. The runner's `session-goals.test.ts` passes with the new
patch.
- `node --test scripts/acpx-patch-packaging.test.mjs`: 26 passed.
- `node --test
packages/paperclip-runner/test/acpx-codex-package-contract.test.mjs`: 9
passed.
- Runner real-bridge Claude tests
(`acpx-runtime-adapter.claude-*.test.ts`,
`acpx-runtime-adapter.test.ts`,
`claude-scheduling-tools.claude-acp.test.ts`,
`runner-tool-bridge.claude-*.test.ts`): 139 passed.
- `claude-local` ACP tests (`acp.test.ts`, `acp.auth.test.ts`,
`acp.quota.test.ts`): 95 passed.
- `pnpm --filter @paperclipai/paperclip-runner run
typecheck:typescript`: passed.
- I did not commit `pnpm-lock.yaml`. The patch hash changes, and CI
resolves the lockfile for this run.

## Risks

- Low risk. The change adds one condition to each of two checks in the
bridge patch. Both match the upstream change.
- The bridge's `commandDigest` does not change, because the patch does
not touch `dist/index.js`. Saved sessions are not reset.
- A sign-in failure that Claude Code reports as a successful, non-error
result is no longer caught by this check. Claude Code 2.1.286 reports
sign-in failures as error results, and upstream uses the same rule.

## Model Used

Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, subagents) and extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:14:57 -08:00
Devin FoleyandPaperclip 0308108531 fix(native): pipeline tool progress events with text deltas (#21533)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs store every relayed runner event in
`heartbeat_run_events`. Each stored event also locks and updates the
run's `heartbeat_runs` row.
> - Text deltas already commit in batches. Every other event, including
each `tool.execution.progressed` event, commits in its own transaction.
> - Tool receipts and usage accounting lock the same run row. Under
parallel tool calls, accounting failed with "other updates to the same
run kept conflicting", and a provider-pack receipt save hit a lock
timeout.
> - #21520 made events that arrive while a batch commits join the next
batch. This pull request adds tool progress events to the pipelined
types, so they use those batches too.
> - The benefit is fewer run-row transactions in tool-heavy turns,
without changing event order.

## Linked Issues or Issue Description

**What happened?**
A routine run started several subagents that made about 190 tool calls
in about 10 minutes. Three tool calls failed with "could not save this
run's budget or usage records because other updates to the same run kept
conflicting", and a provider-pack admission receipt failed with a lock
timeout (55P03). Every tool progress event of the run was a separate
transaction that updated the same run row.

**Expected behavior**
Observational events such as tool progress commit in batches, so they
hold the run row lock fewer times.

**Steps to reproduce**
1. Run a turn that makes many tool calls in parallel.
2. Each `tool.execution.progressed` event commits in its own transaction
and updates `heartbeat_runs`.

**Relevant logs or output**
```
could not save this run's budget or usage records because other updates to the same run kept conflicting
native provider-pack admission receipt could not be saved (cause=55P03)
```

Related: #21531 tried letting the runner relay events faster, which
would make this contention matter more. It is closed.

## What Changed

- `paperclip-control-plane-port.ts`:
- `pipelinedEventTypes` names `item.delta` and
`tool.execution.progressed`. Both go through the batched append path and
its group commit from #21520.
- The text-delta boundary is now tracked when an event is appended: a
pipelined event that is not a delta ends it, and a delta with text
starts it. The first text delta after any other event still commits at
once.
- `contracts/control-plane-port.ts`: optional `pipelinedEventTypes`, and
the `streaming` option documents that it applies to those types.
- `native-session-runtime.ts`: the runtime pipelines the port's
`pipelinedEventTypes`, and only `item.delta` when the port does not name
them.
- Tests:
- `paperclip-control-plane-port.test.ts`: three progress events appended
while a batch commits share one transaction, and a following event
commits after them.
- `native-session-runtime.test.ts`: progress appends start without
waiting only when the port pipelines them.

## Verification

- After rebasing onto current master: `pnpm exec vitest run
src/services/native-runtime/paperclip-control-plane-port.test.ts` (in
`server/`): 108 passed.
- `npx vitest run src/native-session-runtime.test.ts` (in
`packages/paperclip-runner`): 298 passed, 1 failed. `settles a governed
wait through real Runner shutdown after a missing Codex terminal` needs
the built Rust test binaries (`fake-codex-app-server`), which my
checkout does not have. CI builds them.
- `pnpm exec tsc --noEmit -p .` in `server/` (after building its
workspace dependencies) and in `packages/paperclip-runner` pass.
- `npx vitest run src/__tests__/chat-channels.integration.test.ts
src/__tests__/heartbeat-process-recovery.test.ts
src/services/native-runtime/native-plan-wait.test.ts
src/services/native-runtime/paperclip-control-plane-port.test.ts` (in
`server/`): 2,586 passed, 2 failed. `binds Slack tools to the admitted
linked request...` reads a skills checkout that is not next to my
worktree. `materializes direct external-chat finals once...` failed in
the full file run but passes alone, with and without this change.

## Risks

- Progress events now commit up to one transaction later than before.
Their receipts and live UI updates follow their batch.
- A batch that contains progress events still writes rows one by one
inside its transaction, because progress events register chat-progress
wakes. The gain is one transaction and one lock wait per batch, not one
row update per batch.
- If a batch fails, every event in it fails, and the runner resends
them. That is the same as for text deltas today.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-16 11:25:59 -08:00
DottaandClaude Opus 5.5 46e5e4450f fix(runner): leave an undispatched tool call to the next controller on restart detach (#21786)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner keeps an agent session alive while its
controller restarts, for example during a deploy, and the next
controller adopts it
> - A Paperclip tool call can be between its committed input and its
handler when the controller detaches
> - In that window the old controller journaled "The turn stopped before
this operation was dispatched" as the final, non-retryable result, but
the turn did not stop
> - The agent then got a false failure for a call that never ran, and
the next controller could not recover the call. A restart must never
fail work
> - This pull request leaves that call unsettled for the next
controller, the same as a call that was already in its handler
> - The benefit is that a controller restart no longer turns an agent's
pending tool call into a permanent error, and a CI test that failed on
four pull requests now passes deterministically

## Linked Issues or Issue Description

**What happened?**
`src/live/runnerd-codex-transport-orphaned-tool-at-attach.test.ts` ›
`settles an earlier generation's in-flight tool call at attach even when
that attach never finishes startup` failed in "ci / Verify Paperclip
Runner (vitest package (3/3))" on 2026-11-15. In each job the first try
and the CI retry both failed:
- PR #21757, head `d04825d812`, job 132824209377
- PR #21713, head `7758d56719`, job 132681892394
- PR #21729, head `c63a05a013`, job 132732839971

Every failure is the same, at line 74, right after
`detachControllerForRestart()`:

```
AssertionError: expected [ { …(8) } ] to deeply equal []
+   "type": "semantic_tool.result", "status": "pending",
+   "result": { "error": { "code": "semantic_tool_not_dispatched",
+     "message": "The turn stopped before this operation was dispatched.", "retryable": false } }
```

**Expected behavior**
When a controller detaches for restart, a tool call that it has not
dispatched stays unsettled. The next controller recovers it at attach,
the same as a call that is in its handler.

**Steps to reproduce**
1. Hold the semantic tool dispatch at a gate after the Runner commits
the tool input.
2. Call `detachControllerForRestart()`, then release the gate.
3. On `master`, the control-plane state has the
`semantic_tool_not_dispatched` result above. It is byte-identical to CI
(same command id and payload). The new test case `(still before dispatch
at detach)` does this.

**Root cause**
This is a Runner defect, not a test timing problem.
`#detachControllerOnce` sets `#closed`, resolves the closed signal and
resolves the turn-start admission with `false`. A call in
`#handleSemanticToolInput` that has not reached the handler then throws
`SemanticToolNotDispatchedError`. The durable control plane turns that
error into a journaled `semantic_tool_not_dispatched` result. That
result is correct when the turn stops. It is false when the controller
only detaches for restart, because the Runner and the turn continue. The
old test waited only for the committed input, so on a loaded CI runner
the detach sometimes came before the handler call.

**Production impact**
Yes. During a controller restart or deploy, an agent tool call in this
window got a non-retryable "turn stopped" error after the Runner was
adopted, and the next controller did not recover the call.

## What Changed

- `durable-prp-control-plane.ts`: new `SemanticToolHandedOffError`. When
the dispatcher rejects with it, the control plane journals no result.
The input stays unsettled for the next controller.
- `runnerd-codex-transport.ts`: the semantic tool callback goes through
`#dispatchSemanticToolInput`. If the call ends as not dispatched after
the transport detached for restart, it throws
`SemanticToolHandedOffError`. A normal close still journals
`semantic_tool_not_dispatched`.
- `runnerd-codex-transport-orphaned-tool-at-attach.test.ts`: the test
waits for the exact stage instead of the committed input. It runs two
cases: the call is in its handler at detach, and the call is held before
dispatch across the detach. Both then check the existing attach recovery
and the final `completed` result.

## Verification

- Red on `master`: the `(still before dispatch at detach)` case fails
every run with the CI assertion and payload.
- Green on this branch: `npx vitest run
src/live/runnerd-codex-transport-orphaned-tool-at-attach.test.ts` passes
2/2.
- Loop on macOS with 4 extra busy-loop processes (load average about 13
before them), 50 runs each: this branch 50/50 (100 test cases). The old
`master` test was 50/50 locally too, so the local loop does not show the
CI interleaving. The gate does.
- All 36 runner test files that use `detachControllerForRestart` or
`SemanticToolNotDispatchedError`: 1180/1182 pass with 3 workers under
load. The 2 failures (`runnerd-codex-transport-settlement.test.ts` and
`runnerd-acpx-source-reconciliation.test.ts`, `kill EPERM`) pass 36/36
when run alone.
- `npx tsc -p tsconfig.json --noEmit` and `npx tsc -p tsconfig.test.json
--noEmit` pass in `packages/paperclip-runner`.
- The other two static-check flakes from the same day are test-only and
are fixed in a separate pull request.

## Risks

- Medium-low risk. This changes Runner restart behavior for one window.
A call that was not dispatched now goes to the next controller's
recovery (`recoverSemanticToolInput`) instead of getting a final error.
That is the same path that already handles a call in its handler, so no
new recovery code is involved.
- If no successor ever attaches, the input stays unsettled. That is the
same state as for a call that was in its handler at detach.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, GitHub CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 10:27:04 -06:00
DottaandClaude Opus 5.5 348ed0ae10 fix(runner): keep closing tool-call tags and model control tokens out of OpenCode chat (#21761)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner OpenCode driver streams model text into the
chat as progress and selects the final reply
> - DeepSeek v4.1 flash (through OpenRouter) wrote tool-call scaffolding
and tokenizer control tokens as ordinary text
> - #21361 hides tool-call markup, but its detector matched only opening
DSML tags, and nothing recognised control tokens
> - A person saw raw `</|DSML| invoke>` lines in a progress message, and
in another conversation got a degenerate reply instead of an answer
> - This pull request extends the same detector to closing DSML tags and
to model control tokens, on the streaming path and at completion
> - The benefit is that scaffolding never reaches the chat, and a
degenerate reply goes to the existing correction instead of being posted

## Linked Issues or Issue Description

Refs #21361

**What happened?**
Chat-eval cohort 16 (OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017`, master 819f61f1fd)
showed two defects.
1. In `task-coordination.block-on-design`, after a real
`paperclip_finish` call, the model streamed a text part that held only
closing tags (`\r\n</|DSML| invoke>\n</|DSML|
calls>…</invoke>\r\n</parameter>`). The driver emitted it as an
`agentMessage` progress delta, so the chat showed raw markup. The cause:
`SENTINEL_FORMS` matched `<|DSML|` but not `</|DSML|`.
2. In `cases.case-history`, the reply to the second follow-up was
`<|begin▁of▁sentence|><?xml version="1.0" …?>\n<output>\nSearch output
truncated. Let me search more specifically…\n</output>`. The driver took
it as the final text part and posted it. The question was never
answered.

**Expected behavior**
Tool-call scaffolding and model control tokens never appear in progress
or in the reply. A reply made only of them goes to the empty-response
correction, which asks the model for the real answer. Real answers,
including answers about XML, chat templates or tool tags, stay as
written.

**Steps to reproduce**
Feed the recorded text parts above to the OpenCode driver (the new
`closing-markup` and `control-token-reply` cases in
`empty-response.test.ts` do this).

## What Changed

- `tool-call-markup.ts`: closing DSML tags (`</|DSML|`, `</|DSML|`)
count as tool-call markup.
- `tool-call-markup.ts`: new `controlTokenStart` finds DeepSeek, ChatML
and Llama control tokens (for example `<|begin▁of▁sentence|>`,
`<|im_start|>`, `<|eot_id|>`) outside code. A new
`visibleBeforeModelScaffolding` holds back both kinds while text
streams.
- Sentinels inside inline code spans are ignored, as fenced code already
was.
- A per-line fast path skips the sentinel scan when a line has no `<|`,
`<|`, `</|` or `</|`.
- `opencode-server-driver.ts`: prose before a control token stays the
answer, with the token removed (for example a trailing end-of-sentence
token). A part with no prose before the token is discarded with the
`opencode_control_token_reply_discarded` diagnostic. The existing
empty-response correction then asks for the answer, with an opening that
names the control tokens. Unlike tool-call markup, this does not make
earlier prose ineligible.

## Verification

- `pnpm vitest run packages/paperclip-runner/src/drivers/opencode/` (20
files, 296 tests pass).
- Red/green: with master's detector and driver, 9 new detector tests and
the `closing-markup` and `control-token-reply` driver cases fail. With
this change they pass.
- The tests use the recorded cohort 16 text. Negative controls: an XML
answer, an answer that names an `<output>` tag, control tokens in inline
code and in a fence, HTML, and comparison operators are left alone. A
driver case checks that `answer.<|end▁of▁sentence|>` keeps the answer
with no correction.
- Per-message cost (local tsx micro-benchmark, 20k calls, one prose line
with `<`): 500 chars 0.3 → 0.8 µs; 2 KB 2.2 → 2.6 µs; 8 KB about 8–16 µs
before and 7–9 µs after (noise range). The detector runs per text-part
update, as before. Nothing runs before the turn starts.
- Paid narrow eval run: results are added in a PR comment.

## Risks

- A real answer that writes a literal control token outside code is cut
at the token. Prose before it stays. That text is never meaningful
prose.
- Closing DSML tags in plain prose (outside code) now hide the rest of
the part. DSML tags are model-internal, so prose does not contain them.

## Model Used

- Claude, `claude-opus-5-5` (Claude Code agent with tool use and code
execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 08:44:00 -06:00
DottaandClaude Opus 5.5 01c1e708e9 fix(runner): stop a turn that keeps resending a refused tool call and continue fresh (#21734)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native agent runs call Paperclip tools, and
`RepeatedToolFailureGuard` refuses an exact repeat of a call that
already failed validation twice
> - The refusals are not counted and never end the turn, so a model
whose host drops an argument can resend the same refused call until the
run's wall clock
> - In a recorded restart-chaos wave, an opencode/DeepSeek agent sent
the same `PATCH /api/issues/{id}` through `call_api` 431 times in 40
minutes without `body`; 427 were refused. The run held an agent slot and
delayed other tasks
> - The system knows the next step (stop the turn and continue on a
fresh session), so the harness must own it, with no Retry banner
> - This pull request counts refusals, stops the provider turn after a
bounded number with the classified cause `agent_tool_call_loop`, and
queues an automatic fresh-session continuation that tells the model to
use a dedicated tool
> - It also records the declared argument names that each opencode tool
call delivered, so we can tell whether the bridge or the model drops
`body`
> - The benefit is that a looping agent frees its slot in seconds
instead of 40 minutes, and the task continues by itself

## Linked Issues or Issue Description

**What happened?**
An opencode agent on DeepSeek created two subtasks and then tried to set
their descriptions with `call_api` `PATCH /api/issues/{id}`. Every call
arrived with `operationId`, `pathParams`, `query` and `responseText`,
but without `body`. The guard refused each repeat. The model said "I
need to include body" more than 20 times and kept sending the same call:
431 calls, 427 refused, over 40 minutes. The run stayed active and its
agent slot delayed other queued work.

**Expected behavior**
After a small, bounded number of refusals, Paperclip stops the provider
turn and continues the task automatically on a fresh session, with a
hint to use a dedicated Paperclip tool. The board shows no Retry banner
because the system owns this recovery.

**Steps to reproduce**
1. Run a native agent whose model resends one tool call with a missing
required argument.
2. The guard refuses the third and later identical calls.
3. On master, the refusals never end the turn.

**Paperclip version or commit**
`master` at 6400698a8e.

## What Changed

- `repeated-tool-failure-guard.ts`: counts refusals per exact call and
per run. At 6 refusals of one call, or 20 refusals in the run, it throws
`ToolCallLoopError` (`code: "agent_tool_call_loop"`, tool, counts and
argument names, never values). The message tells the model the turn
stops and gives a short hint.
- `paperclip-runner-tool-authority.ts`:
`scheduleToolCallLoopContinuation` writes one idempotent native
status-wake outbox row (key `agent_tool_call_loop:<runId>`) with
`forceFreshSession: true` and the hint as the wake instruction. The
outbox dispatcher admits it with `allowRunCoalescing: false`, so it
never merges into a deferred comment wake or the active run. An assignee
run uses `issue_continuation_needed`. A completion-review run uses
`native_completion_review` with its review assignment, which the
dispatcher revalidates, so the fresh reviewer session keeps the review.
- `native-session-executor.ts`: the session authority epoch queues that
continuation when the guard raises the loop error. The governed-wait
lookup finds the wake on the next tool event and ends the provider turn
as `yielded` with a `same_agent` continuation, the same way the
connection-tools refresh ends a turn. If the wake cannot be queued, the
refusal still reaches the model and the error is logged.
- `status-decision-committer.ts`: a yielded continuation with that key
binds to the queued wake instead of creating a second one.
- `status-arbiter.ts` and `native-run-finalizer.ts`: a pending review
run that ended through this continuation skips the
`native_review_unresolved` resume recovery, because the queued review
wake owns the next reviewer run.
- `mcp-bridge.ts` and `opencode-proxy-finish-diagnostics.ts`: the
opencode tool-delivery diagnostic records `argumentNames` (only names
declared in the tool's input schema, sorted, at most 1,024 characters of
JSON), `argumentNamesTruncated` and `undeclaredArgumentCount`. Values
and undeclared names are never recorded. The forwarder's closed key
allowlist accepts the new fields and validates them, so a record stays
under its 2,048-character limit.

### Threshold choice

- **6 refusals of one exact call.** The guard already runs a call twice
and then refuses with an explicit correction. Six more refusals (eight
identical attempts in total) give a model that reads errors many chances
to change its arguments. In the recorded run the model never changed
them in 427 refusals, so six refusals end that loop within seconds.
- **20 refusals in the run.** A model can alternate between a few
refused variants and keep each one under 6. Refusals happen only for
exact repeats of a certain validation failure, so a healthy run almost
never reaches 20.

### Previous-release story

The guard counters live in memory for one run's tool authority, as
before. A deploy or restart during a run starts the counters again at
zero on the new process; that only delays the stop. The continuation is
an ordinary wake row. If a server on the previous release finalizes a
run that a new server parked, it does not know the key and queues its
own `same_agent` continuation too; the issue execution lock lets one run
and the other coalesces.

## Verification

- Red then green, `repeated-tool-failure-guard.test.ts`: "after N
refusals the guard raises a turn-stopping loop error" and "stops the
turn once refusals across different calls reach the run limit". Both
fail on master.
- Red then green, `native-session-executor.test.ts`: "a turn whose model
repeats a refused call N times ends with agent_tool_call_loop and
schedules a continuation". With the epoch handling disabled it fails (13
calls instead of 8, nothing parked). Also "keeps the refused call's
error when the loop continuation cannot be queued".
- `paperclip-runner-tool-authority.test.ts`: "queues one fresh-session
continuation outbox for the assignee / a completion reviewer stopped by
a tool-call loop" (real Postgres; idempotent).
- `native-tool-call-loop-continuation.test.ts`: with a comment wake
already deferred behind the looping run, the continuation keeps its own
keyed deferred wake, the comment wake is unchanged, and after release a
run starts with `forceFreshSession` and the loop cause.
- `status-arbiter.test.ts`: a pending reviewer stopped by a loop
releases its checkout without a resume recovery.
- `native-status-arbiter-corpus.test.ts`: the existing wake-binding test
now also covers the `agent_tool_call_loop` key.
- `mcp-bridge-delivery.test.ts` and
`opencode-proxy-finish-diagnostics.test.ts`: argument names are
recorded, values and undeclared names are not, invalid records are
dropped, and a boundary test with 32 names of 64 characters stays
forwardable with `argumentNamesTruncated: true`.
- Server: the six touched test files pass (1755 tests). Runner:
`src/drivers/opencode` and `src/cli` pass (569 tests). `tsc --noEmit`
passes for `server` and `packages/paperclip-runner`.
- `pnpm verify` exits 2 (runner package changes need broad coverage), so
required CI supplies the broad run.

## Risks

- A run that hits the limit ends its turn early. The limits are far
above what a model that reads its errors needs, and the task continues
automatically on a fresh session.
- The fresh session loses the in-turn conversation. The task, its
comments, documents and workspace stay, and the continuation names the
cause and the hint.
- The diagnostic now carries declared argument names. These are schema
names, not values, and the forwarder validates the shape before it
forwards a record.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Refs #21360

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 01:52:50 -06:00
Nicky LeachandClaude Opus 5.5 a78be8b601 feat(native): send launch preparation and run digests to plugin task prepare (#21717)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environment plugins can run a native Runner as a plugin task, and a
warm session keeps that Runner between turns
> - On each warm turn Core sends the plugin a `prepare` operation with
the full launch requirements for the new run
> - The requirements' `requirementsDigest` covers the run identity, and
the run identity changes on every turn
> - So a plugin cannot tell that only the run moved, and it must redo
its whole preparation every warm turn
> - This pull request gives the plugin two digests: one that stays the
same while its preparation stays valid, and one that names the run
> - The benefit is that a plugin can transfer only the run identity when
nothing it reconciles changed

## Linked Issues or Issue Description

**What existing behavior does this improve?**
The `prepare` operation that Core sends to an environment plugin for a
plugin Runner task, on cold starts and on warm attaches.

**Subsystem affected**
Native runtime, plugin Runner tasks, the Runner launch contract, and the
plugin SDK environment task contract.

**Current behavior**
A warm attach sends the full requirements. The only digest available,
`requirementsDigest`, includes `runId`, `turnId` and `itemId`, so it
changes on every turn even when the launch is otherwise the same.

**Proposed behavior**
Core computes two digests for every `prepare`:
- `preparationDigest`: the launch requirements an environment reconciles
when it prepares a Runner:
  - the runner, lease and session identity;
  - provider, model and settings;
  - the prompt and instruction text;
- the runtime context's instruction and skill bundles, working copy and
connection instructions.

It leaves out the `runId`, `turnId` and `itemId`, and the runtime
context's MCP assignment and binding (with `aggregateDigest`, which
hashes the assignment). MCP reaches the Runner through Core's tool
authority on every run attachment, never through files the environment
publishes. Core already continues a warm session across an MCP-only
change (`nativeRuntimeContextsMatch` with tool refresh). The binding is
also named per run (`native-mcp:<runId>`). So the digest stays the same
across warm turns of one Runner, lease and session, including when the
agent's MCP access changes.
- `runDigest`: `preparationDigest` plus everything it leaves out:
`runId`, `turnId`, `itemId`, and the MCP assignment and binding. It is
unique to one run's launch.

A plugin that advertises `launchDigestVersion: 1` in its task
description gets both fields on `prepare`.

**Reason and benefit**
A plugin can compare `preparationDigest` with its saved preparation and
transfer only the run identity when they match, instead of a full
preparation.

**Breaking changes**
None. Plugins without `launchDigestVersion` get no new fields.

## What Changed

- `packages/paperclip-runner/src/live/runner-provider-launch.ts`: add
`runnerLaunchPreparationDigest` and `runnerLaunchRunDigest`. The
preparation digest drops the MCP assignment and binding; the run digest
takes the full requirements and includes them. Both use the same
canonical JSON as `runnerLaunchRequirementsDigest`, with their own
schema tags so the three digests never collide. Export them from the
package and the server's vendored runner shim.
- `packages/plugins/sdk/src/environment-tasks.ts`: add optional
`preparationDigest` and `runDigest` to the `prepare` operation, and
optional `launchDigestVersion: 1` to the task description. The
capability gates both digests.
- `server/src/services/native-runtime/plugin-runner-task.ts` and
`native-session-executor.ts`: send both digests on `prepare` when the
description advertises `launchDigestVersion: 1`.
- `server/src/services/environment-task-runtime.ts`: recompute both
digests from the requirements, and refuse a `prepare` whose digests are
missing in part or do not match.
- `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the SDK
public source pin.
- Tests:
- the preparation digest is stable across runs and changes when the
runner, lease, session, instructions, model, approval policy or settings
change;
- with a runtime context, an MCP-only change (the run's binding, no
binding, a changed access set) keeps the preparation digest and changes
the run digest;
- a change to the instruction bundle, instruction entry, a skill bundle,
the skill set, the working copy or the connection instructions changes
the preparation digest;
  - the run digest changes per run;
  - Core sends the digests only when advertised;
- the runtime refuses forged or partial digests before it calls the
plugin.

## Verification

- `npx vitest run src/live/runner-launch-preparation.test.ts` in
`packages/paperclip-runner`: 26 passed.
- `npx vitest run src/services/native-runtime/plugin-runner-task.test.ts
src/__tests__/environment-task-runtime.test.ts` in `server`: 160 passed.
- `npx vitest run source-sdk-public-inputs.test.mjs
source-sdk-evidence.test.ts source-sdk-public-type.test.ts` in
`tests/runner-e2e`: 59 passed.
- `pnpm typecheck` in `server` and in `packages/paperclip-runner`:
clean.
- Not run locally: the full suite. CI covers it.

## Risks

Low risk. The digests are new optional fields behind a plugin capability
version. Core computes them and checks them again before it calls the
plugin, so a plugin can rely on them. They prove only that the launch
requirements are unchanged. A plugin that skips work on a match must
still check its own state, such as files it published into the sandbox.
If the runtime context carries values that change per run, the
preparation digest changes too, so a plugin does its full preparation.
That is safe, but the plugin saves nothing. The MCP binding is the one
such value Core sets per run today, and MCP is excluded entirely. A
plugin that does publish MCP configuration into its sandbox must not
rely on this digest for it. Values outside the requirements are not
covered at all: the account identity and credentials, the GitHub broker,
project IDs, and the plugin's own runtime revision. A plugin must keep
checking those itself. A match lets a plugin skip publishing its launch
configuration, not computing the descriptor: `prepare` must still return
a launch whose `requirementsDigest` matches the current requirements,
which carry the current MCP assignment and binding.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, code search).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:07:54 -08:00
Nicky LeachandClaude Opus 5.5 c603f591dc fix(runner): bound ACPX instructions only where the sandbox delivers them (#21646)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner prepares a private sandbox for each ACPX agent before it
opens the provider session
> - `validateProviderPolicy` in that sandbox stage refuses system
instructions over 32 KiB for every ACPX agent
> - Only Pi gets the text from the sandbox stage. Claude, Codex, Grok
and Antigravity do not use it there. The host accepts 256 KiB
> - An agent with a large instruction bundle fails every run at
`session.open`. The error says only `classification=admission_sandbox`,
so the operator cannot see the cause
> - This pull request applies the 32 KiB sandbox bound only to Pi, and
gives every instructions rejection its own closed admission code
> - The benefit is that Claude and Codex agents with large instructions
can start, and an over-bound rejection names its real cause

## Linked Issues or Issue Description

Fixes: #20814

Refs #14430 (this pull request added the 32 KiB check, with no stated
reason for the other providers)

Related: #21644 (also fixes #20814, but only for Claude, and does not
change the classification)

**What happened?**
Every run of a Claude ACPX agent failed with
`provider_session_open_rejected`. The message was `ACPX sidecar command
session.open was rejected (retryable=false,
classification=admission_sandbox)`. A reproduction in a sandbox from the
same image showed that all the filesystem steps of
`prepareAcpxRuntimeSandbox` succeed. The real cause: the agent's
instructions entry file was 54,734 bytes. That is more than the 32 KiB
check in `validateProviderPolicy`. The host bound (`boundedInstructions`
in `runtime-host.ts`) is 256 KiB. Agents with smaller instructions (12.7
KB) work.

**Expected behavior**
The sandbox stage must refuse only the text that it delivers. #14430
added the check when only Pi consumed `systemInstructions` in the
sandbox (as the `PAPERCLIP_PI_SYSTEM_INSTRUCTIONS` env var). The check
runs before any per-provider branch, so it also refuses Claude, Codex,
Grok and Antigravity. Those providers get the text in the session
request. When a bound refuses the text, the error must name the
instructions, not the sandbox.

**Steps to reproduce**
1. Create a Claude ACPX agent whose composed instructions are more than
32 KiB (for example, an `AGENTS.md` of 54 KB).
2. Start a run of the agent on the Paperclip Runner.
3. See that `session.open` fails with
`classification=admission_sandbox`.

**Paperclip version/commit**
`master` at 163d9f323c.

**Deployment mode**
Paperclip Runner with ACPX providers (any deployment).

Where each provider consumes `systemInstructions` today:

| Provider | Delivery | Bound |
| --- | --- | --- |
| Pi | `PAPERCLIP_PI_SYSTEM_INSTRUCTIONS` env var (sandbox) | 32 KiB in
`validateProviderPolicy` (kept) |
| Cursor | `PAPERCLIP_CURSOR_INSTRUCTIONS` env var (sandbox) | 32 KiB in
`cursorInstructionBinding` |
| Kimi | `PAPERCLIP_KIMI_INSTRUCTIONS` env var (sandbox) | 32 KiB in
`kimiInstructionBinding` |
| Copilot | `copilot-instructions.md` (host, after the sandbox stage) |
32 KiB in `composeCopilotSystemInstructions` and
`refreshCopilotSystemInstructions` |
| Claude, Codex, Grok, Antigravity | session request
(`systemPrompt.append` or the provider's own session admission) | host
bound, 256 KiB |

## What Changed

- `runtime-sandbox.ts`: `validateProviderPolicy` gets the agent. It
applies the 32 KiB bound only to Pi. The type and NUL checks stay for
all agents. Cursor, Kimi and Copilot keep their own 32 KiB checks.
- `admission-diagnostics.ts`: new closed admission code
`ACPX_ADMISSION_INSTRUCTIONS`, and `acpxInstructionsAdmissionError`,
which tags the error with that code. The tag wins over the stage code
that the host records later.
- The instructions bound and NUL rejections in `runtime-sandbox.ts`,
`runtime-host.ts` (`boundedInstructions`), `cursor-instructions.ts`,
`kimi-instructions.ts` and `copilot-profile.ts` use the new tagged
error. The messages contain no instruction text and no provider output.
- `acpx_sidecar_transport.rs`: `ACPX_ADMISSION_INSTRUCTIONS` maps to
`classification=admission_instructions`. The classification set stays
closed.
- `antigravity-connection-probe.ts`: the closed classification pattern
accepts `admission_instructions`.
- `copilot-installation.test.ts`: the v70 identity test compared two
historical source hashes with the live files. Since #18325, Runner
source changes do not advance profile identities. The test now pins the
hashes that the v70 record holds.

## Verification

- Red-green: the new tests failed on `master` (11 failed) and pass with
the fix.
- `cd packages/paperclip-runner && npx vitest run
src/drivers/acpx/runtime-sandbox.test.ts
src/drivers/acpx/runtime-host.test.ts
src/drivers/acpx/kimi-instructions.test.ts
src/drivers/acpx/cursor-instructions.test.ts
src/drivers/acpx/copilot-profile.test.ts
src/drivers/acpx/copilot-runtime-host.test.ts
src/drivers/acpx/pi-acp-runtime.test.ts src/cli`: 29 files, 489 tests
pass.
- New tests:
- The sandbox admits 54,734-byte instructions for Claude, Codex, Grok
and Antigravity, and does not put them in the launch environment.
- The host opens a Claude runtime with 54,734-byte instructions and
passes the full text to the provider.
- Pi and Cursor sandboxes refuse more than 32 KiB with
`ACPX_ADMISSION_INSTRUCTIONS`. NUL also gives that code.
- Through the host and the sidecar error mapping
(`acpxSidecarErrorCode`), oversized Pi (32 KiB + 1), Copilot (32 KiB +
1) and Claude (256 KiB + 1) instructions give
`ACPX_ADMISSION_INSTRUCTIONS` before the provider launches.
- The Rust classification test maps `ACPX_ADMISSION_INSTRUCTIONS` to
`admission_instructions`.
- The Antigravity probe logs `admission_instructions` as a closed
classification.
- `cargo test -p paperclip-runner-core --lib
classifies_only_allowlisted_internal_sidecar_failures` passes. `cargo
fmt --check` passes.
- `tsc -p packages/paperclip-runner/tsconfig.json --noEmit` passes.
- `cd server && npx vitest run
src/__tests__/antigravity-connection-probe.test.ts`: 11 tests pass.
- I did not do a live provider run. I did not run the full local suite.
CI supplies that coverage.

## Risks

- Low risk. The bound gets less strict only for providers that do not
get the text from the sandbox stage. Those providers stay under the host
bound (256 KiB). Pi, Cursor, Kimi and Copilot keep 32 KiB.
- An over-bound rejection now reports `admission_instructions` and not
`admission_sandbox` (or `admission_configuration` for Copilot). A
dashboard or alert that matches the old classification for this case
must also match the new one. Server code that checks a specific
classification does not check these values, except the Antigravity
probe, which this pull request updates.
- The NUL rejection message in `validateProviderPolicy` changes from
"exceed their bounded size" to "are invalid".

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 13:40:20 -08:00
Nicky LeachandClaude Fable 5.2 ddfbbfff20 fix(runner): recover provider-started turns after a controller restart (#21612)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - The Runner can keep Claude while a background shell is running and
give the cycle of Claude to the controller as a turn that the provider
started. Core can wait for that turn. #21607 lets a caller ask the
transport for it.
> - A controller can restart at any time, for example at a deploy. A
wait can last 10 minutes and a provider-started turn can last longer, so
a restart during one of them is a normal event.
> - Tests with the real `paperclip-runnerd` showed that such a restart
failed the session. The checkpoint that Core had was the checkpoint of
the turn before, and the recovery of the driver refused or mislabelled
the turns that it did not know.
> - This pull request makes that restart safe. It changes no production
run, because no caller asks for provider-started turns before part 3.
> - The benefit is that part 3 can turn the behavior on without a failed
run at each deploy.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

#21456 turned the behavior on and had grown too large to review. It is
now three stacked pull requests:

1. #21607 (merged): the transport can ask the Runner for
provider-started turns. No caller asks.
2. **This pull request:** the recovery after a controller restart during
a wait or during a provider-started turn.
3. #21456: Core sets the option for native ACPX Claude runs. Only that
part changes what a production run does.

This is part 2. Its base branch is `master`. Refs #19912 (design),
#21456.

**What existing behavior does this improve?**

None in production. It makes a controller restart safe for a run that
asked for provider-started turns.

**Subsystem affected**

Paperclip Runner package: the consumer
(`src/native-session-runtime.ts`), the session backend
(`src/backends/harness-driver-backend.ts`), the driver
(`src/drivers/codex/`) and the transport (`src/live/`). Only for a run
that asked for provider-started turns.

**Current behavior**

- The consumer saves no checkpoint when a wait starts or when a
provider-started turn starts. During a wait and during a
provider-started turn, the checkpoint that Core has is the checkpoint
from the start of the turn before. It names that turn as active.
- A restart from that checkpoint fails: the driver has the old turn as
active, gets the events of a provider-started turn, and stops the
session with `turn_binding_mismatch`.
- The recovery of the driver adopts one turn that it does not know as a
disposition recovery turn, and refuses two or more. With two or more, it
closes the transport, and that stops the Runner and Claude.
- The Runner does not send again an event that the controller
acknowledged. If the start of a provider-started turn was acknowledged
before the restart, the new driver gets the other events of that turn
with no start.
- A turn that ends waiting for a provider-started turn sets the terminal
of the session, as a turn that ends the run does. After a restart, if
the provider-started turn already has its result when recovery
completes, the consumer finds a result and a terminal, settles the run,
and does not read the events of that turn. The start, the items, the
usage event and the terminal of that turn do not reach Core.
- The recovered driver reads notifications from its construction, and
`reconcile()` maps the terminal of the saved turn only after its reads.
If the saved checkpoint is still the checkpoint of the controller turn
and the start of the provider-started turn is already queued, the driver
judges that start too early and stops the session with
`turn_binding_mismatch`.
- A checkpoint that waits can also have a consumed disposition marker (a
recovery turn after an earlier restart ended waiting). The driver
restores that session as closed and reads no notifications, and the
consumer settles the run on the earlier terminal with no result.

**Proposed behavior**

- **Checkpoints.** The consumer saves a checkpoint when a wait starts
and when a provider-started turn starts, with the function that it
already uses for a governed wait. A checkpoint that fails stops the
turn, as for a governed wait.
- **A checkpoint as of the event that Core stored.** The driver gives
each event its sequence when it reads it from the provider, before the
consumer stores it. A snapshot that is taken later can be ahead of Core.
For these two checkpoints, the driver captures its snapshot when it has
handled the notification that made the event (the `turn.completed` that
waits, or the `turn.started` of the provider), with the sequence of that
event. The consumer asks for the snapshot as of the event that it has
just appended. The saved state and cursor are then those of an event
that Core has.
- **Session terminal.** A turn that ends waiting for a provider-started
turn does not set the terminal of the session (`HarnessNativeSession`).
The terminal of the session is the end of the run, and that turn does
not end the run. A checkpoint that is saved during a wait or during a
provider-started turn has no terminal, so a recovered session settles
only from the events that it reads, in order, to the terminal of the
turn that has the result.
- **Driver, a checkpoint that waits.** Recovery keeps the wait. It does
not adopt and does not refuse the turns after the last terminal that it
knows. The Runner sends again, in order, each event that was not
acknowledged, and the driver admits each provider-started turn from that
stream, as it does with no restart.
- **Driver, a checkpoint with an active turn.** `reconcile()` accepts
one turn in progress that comes after the saved active turn, when the
saved turn ended with the wait marker. It refuses each other unexpected
active turn as before, and it refuses this one too when the reconciled
turn settled a result.
- **Driver, a start that is queued before recovery reconciled the saved
turn.** A `turn/started` of the provider that arrives while the session
does not wait for one first waits until `reconcile()` has settled, and
is then judged as before. `reconcile()` judges the live state after its
reads; if the stream already settled the saved turn into a wait, it
accepts exactly one turn in progress after the last terminal turn and
does not adopt it. The driver admits that turn at its start.
- **A wait with a consumed disposition marker.** The driver keeps
reading notifications for a restored session that waits, as it does for
a wait with no restart. It still refuses a second controller-started
turn. The consumer does not take the consumed-marker shortcut for a
recovered snapshot that waits, so the wait is resumed.
- **Transport, the start comes first.** For each driver, the transport
gives the `turn/started` of a provider-started turn before each other
notification and before a tool call of that turn. If it did not give
that start to this driver, it builds it from the `turn.started` event in
its stored events. It does not do this for the turn that the recovered
driver already has as active (`resumeActiveTurnId`). It binds the events
and the tool call of a provider-started turn to that turn, not to the
turn that the Runner reports as active at the time of the recovery read.
- **Transport, which start owns an event.** The stored events can have
gaps: the window removes old events but keeps the latest of each anchor
type, and the controller checkpoint drops streamed deltas. The owner of
an event is the last stored `turn.started` before it, if it is a
provider start, and if the first turn terminal that is stored after that
start is not before the event and is the terminal of the same provider
turn (`providerTurnId`). If the first stored terminal after the start is
the terminal of a different turn, the end of the provider turn is not
known, and the transport builds no start.

**Reason and benefit**

- The checkpoint is the cause. With a true checkpoint, the two states of
a restart are states that recovery can handle: a wait, or an active
turn.
- The Runner already sends again each event that was not acknowledged.
Recovery uses that stream and does not build the turns again from a
read.
- The window removes a terminal only while a later terminal of the same
type is stored. Thus the first stored terminal after a start shows if a
terminal of that turn was removed. A rule that only ignores the gaps is
not safe: the Runner can make a `turn.failed` for a turn that did not
emit its start (`provider_turn_lost_on_restore`).

## What Changed

Production code: 113 lines added, 19 removed.

- `src/native-session-runtime.ts` (15 added, 4 removed): the two
checkpoints in `consumeTurn`, with the sequence of the appended event,
and the resumed wait with a consumed disposition marker.
- `src/drivers/codex/codex-session-state.ts` (24 added, 1 removed),
`codex-session-terminal.ts` (2 added), `codex-session-notifications.ts`
(2 added), `codex-harness-session.ts` (24 added, 7 removed),
`codex-app-server-driver-impl.ts` (4 added, 1 removed): the snapshot at
the event, and the recovery of the driver.
- `src/contracts/harness-driver.ts` and
`src/contracts/native-session-backend.ts` (1 added each): an optional
`boundarySnapshot` on a driver session, and an optional sequence in the
snapshot options. Other drivers do not implement it and are not changed.
- `src/backends/harness-driver-backend.ts` (7 added, 5 removed): the
session terminal, and the snapshot as of a sequence.
- `src/live/runnerd-codex-transport.ts` (15 added, 1 removed) and
`src/live/provider-turn-tool-order.ts` (18 added): the start-first rule
and the owner of an event.
- Tests: `src/live/runnerd-acpx-provider-turn.test.ts` (572 added, 5
removed),
`src/drivers/codex/codex-app-server-driver.provider-turn.test.ts` (348
added), `src/live/runnerd-codex-transport.test.ts` (137 added),
`src/control-plane/durable-prp-control-plane.test.ts` (84 added),
`src/live/provider-turn-tool-order.test.ts` (new, 61),
`src/native-session-runtime.provider-turn.test.ts` (92 added, 6
removed), `src/backends/harness-driver-backend.test.ts` (56 added).
- `runner/crates/runner-core/src/bin/fake-acpx-sidecar.rs` (test binary
only, 123 added, 14 removed): a delay before a provider-started turn, a
goal state at its terminal, a list of provider-started turns for one
session, a gap between them, a turn that stays open until the new
controller reads the snapshot, and a turn that starts after that read.
The modes that existed are not changed, and the fake reads its input on
one thread as before.

## State saved by the previous release

- No new persisted field. The driver session already has
`awaitingProviderTurn` (#21453).
- **More checkpoints:** one when a wait starts and one when a
provider-started turn starts. An ordinary turn saves the same number of
checkpoints as before (4 in the unit test, measured before and after).
- **No terminal in a checkpoint of a wait.** A checkpoint that is saved
during a wait or during a provider-started turn has `terminal: null`.
When a wait ends with no provider-started turn (the limit, a comment of
a person, or the Runner ends the run), the last checkpoint of the run
also has `terminal: null`; before, it had the terminal of the turn that
waited. One reader uses that field with no result: the text of a cleanup
failure (`nativeCheckpointRecordsCompletedTurn`), which then does not
say that the turn completed. In the tests with the real daemon (the
limit, the provider process lost, the sidecar exits), the run still ends
on the path of today (`native_finalization_missing`).
- **A checkpoint of the previous release** has no waiting state.
Recovery takes the path of today.
- **Rollback of the Runner package during a wait:** the previous code
reads the checkpoint, ignores `awaitingProviderTurn`, and takes its
recovery path of today.

## Verification

With nothing from part 3. The Rust checks and the build checks in the
first item were run before the last seven commits; these commits change
no Rust. The other items were run at the head of this branch
(`36d5e5e587`), or at an earlier commit of this branch for a suite whose
files the later commits do not change.

- Before the last seven commits: `pnpm run typecheck:typescript`, `pnpm
run build:typescript` and `node --test
test/acpx-sidecar-contract.test.mjs` (18 passed) in
`packages/paperclip-runner`; `node
scripts/verify-runner-vendor-dependencies.mjs` in `server`; `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`; `cargo fmt
--check`, `cargo build`, the 17 `acpx_*` suites (241 passed) and the
`acpx` unit tests (102 passed): all passed.
- `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p
tsconfig.test.json` in `packages/paperclip-runner`; `pnpm typecheck` in
`server`: all passed.
- `src/live/runnerd-acpx-provider-turn.test.ts`, 4 runs with the
production code of this head, 26 of 26 passed in each. The Core of these
tests follows the production append (`appendNativeSourceEvent`): it
stores an event that is out of order, keeps its contiguous cursor below
a gap, and reports a duplicate or a conflict for an event that it has.
It does not reject a gap in the sequence; production does not. The
restart cases, all with the real `paperclip-runnerd`:
- **From the checkpoint that the real consumer saved.** The first
`executeNativeSession` is fenced and detached as the server does for a
restart, and a second one recovers from the last checkpoint that the
first one saved. (a) Restart during the wait; the provider-started turn
starts after the new controller is attached. (b) Restart while a
provider-started turn is in progress. In each: the test asserts the
shape of that checkpoint, the run settles with the result of the
provider-started turn, each event is appended one time and in order, the
turn has one `turn.started` with `origin: "provider"`, and the session
closes with no error.
- **Without the two checkpoints, each of these cases fails:** the saved
checkpoint is the checkpoint of the turn before, and the second
controller fails with `turn_binding_mismatch` or with a replay conflict.
- **A slow `appendEvent`, with events that arrive behind the stored
event** (3 cases, each run 3 times with the fix and 3 times without it,
with the same result in each run). R7: the append of the
`turn.completed` that waits is slow, and the provider-started turn
starts behind it. With the snapshot at the event, the checkpoint is the
waiting state at the sequence that Core has, and Core gets the
`turn.started` of the turn one time. Without it, the checkpoint names
the provider-started turn as active, the transport builds no start, and
Core never gets that `turn.started`. R8: the first controller answers
the `paperclip_finish` call while the append of the provider
`turn.started` is slow. With the snapshot at the event, the checkpoint
has no result and the run ends with `native_finalization_missing` (see
Risks). Without it, the checkpoint has a result that Core does not have,
and recovery stops with "session cannot start another turn". R6: the
append of the provider `turn.started` is slow and a delta is queued
behind it. With the snapshot at the event, the new controller continues
the sequence that Core has. Without it, the only difference is one
sequence number that Core never gets.
- **The start was acknowledged before the restart** (3 cases: one turn
open across the restart; two turns; a turn that waits for the answer to
its `paperclip_finish` call). The new driver gets the start of each turn
one time before its other events, the finish call is answered exactly
one time, and the session closes with no error.
- **The turn ran with no controller attached** (2 cases, with and with
no result) and **two turns in one outage** (2 cases): the turns arrive
in order after the recovery.
- **A provider-started turn that is active in the checkpoint** gets no
second start, and the next provider-started turn arrives after it.
- **The turn already has its result when recovery completes** (3 cases:
the turn made its `paperclip_finish` call with no controller attached;
the turn waited for the answer to that call at the restart; two turns
across the restart). The test asserts the exact list of events that Core
gets: each provider-started turn has one start, its usage item, its
message item and its terminal, and then `run.result.accepted` and
`run.terminal`. `completeRun` is called one time with the turn that has
the result. Before the change to the session terminal, the second
controller appended only `run.result.accepted` and `run.terminal` in
each of these cases.
- **The wait checkpoint is not saved, then a restart** (R10). The last
checkpoint is the checkpoint of the controller turn, Core has the
terminal of that turn, and the start of the provider-started turn is
queued. The run settles with the result of the provider-started turn,
Core has one `turn.started` for it, and the finish call is answered one
time. Without the wait in the driver, recovery stops with
`turn_binding_mismatch`.
- **An ordinary turn**, with no option. One that waits for the answer to
its finish call at a restart: Core gets its result, its terminal and the
run terminal one time. One whose checkpoint after `startTurn` is ahead
of Core at a restart (R9): the run settles, and Core does not have only
the one event that the first controller did not store. These cases pass
with and without this pull request.
- **No checkpoint with a terminal and no result** is saved by the two
controllers in the cases above.
- `src/live/provider-turn-tool-order.test.ts` and
`src/live/runnerd-codex-transport.test.ts`: 345 passed. The helper tests
cover a stored start with removed events after it, a dropped delta, the
terminal of the turn before and after the event, a terminal of a
different turn after a removed terminal, and a controller start. With no
option, with a different agent or provider, and for a turn that the
controller started, the notifications of the transport are the same as
before this pull request.
- `src/control-plane/durable-prp-control-plane.test.ts`: 230 passed.
Three new tests use the real commit path: a full window of 4,096 events
removes the events after a provider start and the start is found; the
window removes the `turn.failed` of a provider turn after a later
`turn.failed` with no start, and no start is found; a restored
checkpoint that dropped the deltas of the turn, and the start is found.
- `src/drivers/codex` and `src/backends/harness-driver-backend.test.ts`:
491 passed. The driver tests cover a provider start that is queued
before `recoverSession()` is called, the wait state with one turn in
progress at `reconcile()` (and the two cases that it must still refuse),
a restored wait with a consumed disposition marker (the provider turn is
admitted and `startTurn` is refused), the snapshot at the two events
(also after the driver moved on, and for a restart continuation that
ends waiting), one later turn in progress, one completed and a second in
progress, two completed with a result, and the cases that `reconcile()`
must still refuse. The backend tests cover the session terminal and the
snapshot as of a sequence.
- `src/native-session-runtime.provider-turn.test.ts` and
`src/native-session-runtime.test.ts`: 333 passed. Two new cases resume a
restored wait with a consumed disposition marker: the provider-started
turn arrives and settles the run, or the wait ends at its limit and the
run takes the path of today with no new turn.
- `server`: `native-session-executor.test.ts`,
`native-provider-turn-wait.test.ts` and
`native-runner-restart-recovery.integration.test.ts`: 1,743 passed, 21
skipped.
- Mutation checks, each made named tests fail: no checkpoint at the
start of a wait; no checkpoint at the start of a provider-started turn;
the consumer does not pass the sequence (R6, R7 and R8 fail: a hole in
the sequence, no `turn.started` in Core, a different failure); the
backend ignores the sequence; the driver has no snapshot at the event;
no start-first rule; no exclusion of the active turn of the recovered
driver; the turn binding only at the first delivery; the contiguous
sequence required again; no check of the first terminal after the start;
the adoption of later turns put back; no exception in `reconcile()`; no
check after the settlement in `reconcile()`; the terminal of the session
set again by a turn that ends waiting; no wait of the provider start for
`reconcile()`; no exception in `reconcile()` for a session that waits;
the pump deferred for a restored wait; the consumed-marker shortcut
taken for a snapshot that waits.

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** Each change is behind the waiting state, a
provider `turn.started`, or the option of the transport. No caller sets
the option before part 3.
- **An event that the first controller received but Core did not store
is not sent again after a restart.** This is the same on `master` and is
not changed here. Only the terminal of a turn is built again. If the
lost event is the result of a provider-started turn (the first
controller answered the `paperclip_finish` call and stopped before Core
stored the proposal), the run ends on the path of today for a turn with
no result. R8 shows this. R9 shows the same loss for one event of an
ordinary turn.
- **A second `turn.started` for one provider-started turn can reach
Core.** This occurs if the restart comes after Core stored the start and
before the checkpoint of that start is saved (the time of one checkpoint
write). The last checkpoint then waits, and the transport builds the
start again for the new driver. Seen with the real daemon when the
checkpoint write is held; not in a committed test. The usage accounting
of the server uses the turn id as its key, so there is no second count,
but the second start resets the "restored" mark of that turn. The
transcript does not read `turn.started`.
- **The older checkpoints can also save a cursor that is ahead of Core**
(measured for the checkpoint after `startTurn` of an ordinary turn).
Production accepts the gap in the sequence that follows, so this is not
a failure there. It is on `master` and is not changed here.
- **When the terminal of a provider-started turn was removed from the
stored events,** the transport builds no start for an event of that
turn, and the session stops on that event, as before. This needs a later
terminal of the same type with no start of its own (a turn that was lost
while it started) and a full window.
- **The last checkpoint of a wait that ends with no provider-started
turn has no terminal,** as the section above says. The change is only in
a run that waits, and no run waits before part 3.
- **A wait with a consumed disposition marker has no test with the real
daemon.** The state needs two restarts in one run, and the test helper
does one. Unit tests of the driver and of the consumer cover it.
- **A provider start that waits for `reconcile()`** waits as long as the
reads of `reconcile()` take. Each read ends at the command deadline of
the Runner (30 seconds by default) or when the transport closes.
- **`item.delta` events across a restart are not asserted.** A streamed
delta that the first controller received is not sent again. The
completed items are asserted.
- **The driver keeps a maximum of 8 snapshots at events** and drops each
one when the consumer asks for it or for a later one. If the consumer is
more than 8 such events behind, the checkpoint uses the current
snapshot, as before this change.
- **A second provider-started turn that also ends during the same
outage** is not tested with the real daemon: the fake gives each
terminal the same goal revision, and the Runner then holds the second
terminal until a controller is attached. The end of the reconnect time
of the Runner is not tested here.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, decided the designs, reviewed the diffs, and ran a
part of the verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as Claude Code subagents with
tool use (shell, file edits, local test runs), wrote the change and the
tests, reproduced the review findings with the real daemon, and verified
this slice alone.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 12:51:52 -08:00
Nicky LeachandClaude Opus 5.5 68d5c9c120 feat(runner): let a caller ask the transport for provider-started turns (#21607)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - Today the run ends when the turn ends. The command stops, and the
run fails with `native_finalization_missing`.
> - The Runner can already keep Claude while a background shell is
running and give the cycle of Claude to the controller as a turn that
the provider started (#21423, #21316). Core can already wait for that
turn (#21453, #21454, #21455). No run does this yet, because no
controller asks the Runner for it.
> - The request goes through the transport of the Runner package. This
pull request gives the transport an option for it and the behavior that
the option needs.
> - No caller sets the option in this pull request. It changes no
production run.
> - The benefit is a small base for the two pull requests after it: the
recovery after a controller restart, and the switch that turns the
behavior on.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

#21456 turned the behavior on and had grown too large to review. It is
now three stacked pull requests:

1. **This pull request:** the transport can ask the Runner for
provider-started turns. No caller asks.
2. #21612: the recovery after a controller restart during a wait or
during a provider-started turn.
3. #21456: Core sets the option for native ACPX Claude runs. Only that
part changes what a production run does.

Refs #19912 (design), #21456.

**What existing behavior does this improve?**

None in production. It adds the support that parts 2 and 3 use.

**Subsystem affected**

Paperclip Runner package, the controller side:
`src/live/runnerd-codex-transport.ts`. ACPX Claude only, and only for a
caller that sets the new option.

**Current behavior**

- The transport has no way to ask the Runner for provider-started turns.

**Proposed behavior**

- The transport options have a new field, `providerStartedTurns`. The
transport uses it only for ACPX with the agent `claude`. In each other
case the field has no effect.
- With the option, the transport sends `providerStartedTurns: true` in
`turn.start`. It does not send it for a compaction turn.
- With the option, a tool call in a provider-started turn waits until
the transport gave each earlier event to the driver. Without this, a
`paperclip_finish` of that turn could reach the driver before the
`turn.started` of that turn, and the driver would refuse it.
- With the option, the transport gives a `run.terminal` of the Runner to
the driver as a marked internal notification (`paperclip/runTerminal`).
The driver uses it only during a wait, so a wait ends at once when the
Runner says that the run is over.
- With no option, the transport does the same as before this pull
request. Live sessions, evals, the CLI runner and the probes do not set
the option.

**Reason and benefit**

- The opt-in is a choice of the caller, not a property of the agent.
Only a consumer that can wait for a provider-started turn must ask for
one. `CapabilityLiveSession` completes a turn at the first
`turn/completed`, so it must not get the behavior because its agent is
Claude.

## What Changed

- `src/live/runnerd-codex-transport.ts` (31 lines added, 1 removed): the
option, the field in `turn.start`, the order of the tool call, and the
`paperclip/runTerminal` notification.
- `src/live/provider-turn-tool-order.ts` (new, 16 lines): finds the
source sequence of a tool call that belongs to a provider-started turn.
- `src/live/runnerd-acpx-provider-turn.test.ts` (new, 239 lines): 7
cases with the real `paperclip-runnerd`, the fake ACPX sidecar, and the
real consumer of Core.
- `src/live/live-session.test.ts` (12 lines added): a Claude live
session gives the transport no such option.
- `runner/crates/runner-core/src/bin/fake-acpx-sidecar.rs` (test binary
only): the mode `provider-result-resolved`. In this mode the
provider-started turn calls `paperclip_finish` and waits for the answer
of Core through the Runner (`tool.resolve`), as the real sidecar does.
The mode `provider-result` is not changed.

## State saved by the previous release

- No persisted state changes in this part.
- **A Runner that does not know the field** ignores it.
- **Rollback of the Runner package:** the previous transport does not
send the field, so the Runner does not wait.

## Verification

At the head of this branch (`2f2cb14b9c`), with nothing from parts 2 and
3:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript` and `pnpm run
build:typescript` in `packages/paperclip-runner`; `pnpm typecheck` and
`node scripts/verify-runner-vendor-dependencies.mjs` in `server`; `node
--test scripts/__tests__/runner-release-crate-cache.test.mjs`; `node
--test test/acpx-sidecar-contract.test.mjs` (18 passed): all passed.
- `src/live/runnerd-acpx-provider-turn.test.ts`, 3 runs, 7 of 7 passed
in each. The cases:
- background work, then the turn of Claude calls `paperclip_finish`;
Core answers; the Runner gives the answer to the sidecar with exactly
one `tool.resolve`; the turn completes; the run finalizes with one
`run.result.accepted`;
- a Claude run with no option from the caller takes the path of today:
no `providerStartedTurns` in `turn.start`, no `paperclip/runTerminal`,
no wait, and no `session.provider_turn` to the sidecar;
- the limit ends the wait, the run takes the path of today, and the
provider processes had exited before the cleanup of the test;
- the provider process is lost during the wait, and the run settles in
seconds;
  - the sidecar exits during the wait, and the run settles in seconds;
  - a turn with no background work does not wait;
- an ACPX agent that is not Claude is never opted in, also when the
caller sets the option.
- Each case asserts the result of the close of the session: clean in 6
cases, and a named failure in the case where the sidecar is killed. Each
clean case also asserts that there is no
`native_session_settlement_incomplete` diagnostic and no "conflicting
duplicate tool result" in the durable state.
- `src/live/live-session.test.ts`,
`src/live/runnerd-codex-transport.test.ts`,
`src/live/runnerd-acpx-provider-turn-wait.test.ts`,
`src/native-session-runtime.provider-turn.test.ts`: 434 passed.
- Rust, because the fake sidecar changed: `cargo fmt --check`, `cargo
build`, the `acpx_*` suites (244 passed) and the `acpx` unit tests (102
passed).
- Mutation checks, run on the full change before the split: with the
option ignored, 6 of the 7 assertions of the case with no option fail;
with the old mode of the fake, the case with the result fails at the
close of the session.

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** No caller sets the option. Core sets it in
part 3.
- **The transport is shared by each provider.** The three new behaviors
are behind one condition: the option, ACPX, and the agent `claude`. With
the condition false, the code path is the path of today.
- **The tool-call ordering uses a 10 ms poll** while earlier events are
not delivered, only in a provider-started turn.
- **A controller restart during a wait or during a provider-started turn
is not handled in this part.** Part 2 handles it. Part 3 must not merge
before part 2.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 08:16:24 -08:00
Nicky LeachandClaude Opus 5.5 b1f608296e feat(runner): keep a run open for a limited time while the Runner waits for the provider's own turn (#21454)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - Today the run ends when the turn ends. The command stops, and the
run fails with `native_finalization_missing`.
> - The Runner part of this work is complete in #21316 and the pull
requests below it: when a controller sends `providerStartedTurns` on
`turn.start`, the Runner keeps Claude while a background shell is
running, and gives the cycle of Claude to the controller as a turn that
the provider started.
> - `consumeTurn` reads the events of one turn and returns at the
terminal. It already continues past a terminal for a Codex goal.
> - This pull request makes it continue past a terminal that waits for a
turn of the provider, and it limits that wait.
> - The benefit is that a run continues in the turn that Claude starts,
and a wait always ends.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one of four parts on the Core side. They are
stacked on #21316:

1. The driver admits a turn that the provider started.
2. The consumer waits for that turn, for a limited time.
3. The server ends the wait when a person comments, and keeps the clock
across a restart.
4. The transport sends the opt-in. Only this part changes what a
production run does.

This is part 2. Its base branch is `feat/core-provider-turn-driver`.
Refs #19912 (design).

**What existing behavior does this improve?**

Nothing in production yet. What the consumer of a native session does
after a turn that ended with no result while the Runner waits.

**Subsystem affected**

Paperclip Runner package, the controller side:
`src/native-session-runtime.ts` and a new module
`src/native-provider-turn-wait.ts`.

**Current behavior**

- `consumeTurn` returns at the first terminal. With no result, the run
is finalized as `native_finalization_missing`.

**Proposed behavior**

- At a terminal with `awaitingProviderTurn` and no result, `consumeTurn`
continues to read. It consumes the provider-started turn as the next
turn of the same run. A turn that reports a result finalizes as a normal
run.
- The wait has a limit of 10 minutes from the first terminal that waits.
A later terminal that waits does not start the clock again. The limit is
never later than 5 seconds before the timeout of the run.
- The limit and an early end stop the wait only. They never interrupt a
provider-started turn that is running; that turn stays under the timeout
of the run. If that turn ends and the Runner waits again, the run takes
the path of today at once.
- When the wait ends with no turn active, the consumer returns the last
terminal, and the existing code for a run with no result decides
(`resolveMissingResult`, then `native_finalization_missing`). No new
failure code.
- The wait ends at once when the driver gives the acknowledgement of
part 1. This covers Stop, the limit, and a Runner that says the wait is
over.
- `executeNativeSession` gets one new option, `providerTurnWait`: the
start time of a wait that an earlier controller process saw, and a hook
that ends the wait early.
- When the wait ends a turn, the consumer first waits for the receipts
of the text deltas that are committed in batches (#21460), as each other
return path of `consumeTurn` does. Without this, a delta that arrived
during the wait was not in the event count of the turn, and a delta that
failed to commit was not reported.
- When the interrupt that ends the wait is not available or fails, the
limit forces the end of the wait. If that forced end starts a new turn
(the continuation that a restart owes), the consumer continues to read
that turn. The forced-end signal is used one time only, and the read
that is open at that moment is kept until it gets its event. Before this
fix, the consumer returned the terminal of the old turn and lost the
first event of the new turn, and the run could end as
`native_finalization_missing`.
- If the controller restarts during the wait, the run continues to wait.
When the saved checkpoint still shows the first turn as active, the
recovered session gives the waiting terminal again, and the wait starts
again with the start time that was recorded. When the checkpoint was
saved after the waiting terminal, the consumer finds that terminal in
the journal, starts the wait again, and does not send the
disposition-only recovery prompt. The consumer does this only if two
conditions are true: the recovered session has the waiting state
(`awaitingProviderTurn`, saved by the driver since #21453), and the
terminal in the journal has the marker. If one condition is false, the
run uses the recovery path of today. When a wait that started again ends
with no turn, the existing code for a run with no result decides, as for
each other wait.

**Reason and benefit**

- The 10 minutes are the maximum time of a foreground command in the
pinned Claude Code.
- A limit on the wait, not on the work, means that a long test run in
the turn of Claude is not cut.

## What Changed

- `src/native-provider-turn-wait.ts` (new, 129 lines) and
`src/native-session-runtime.ts` (58 lines added, 4 removed),
`src/index.ts` (1 export).
- `src/native-session-runtime.provider-turn.test.ts`: written again for
the new behavior (557 lines added, 61 removed).

## State saved by the previous release

- This part adds no persisted state. The wait is in memory. Part 3 reads
its start time from an event row that is already saved.

## Verification

At the head of this branch:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json` and `pnpm run build:typescript` in
`packages/paperclip-runner`; `node
scripts/verify-runner-vendor-dependencies.mjs` and `pnpm typecheck` in
`server`; `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`: all passed.
- `src/native-session-runtime.provider-turn.test.ts`: 30 tests. They
cover: a controller restart with the first turn still active in the
checkpoint (the wait continues; a provider-started turn settles its
result; with no turn the limit ends the wait on the recorded clock); a
controller restart with the checkpoint saved after the waiting terminal
(the same two results, no recovery prompt, no event of the journal
appended again); a limit that passed before the restart ends the wait at
once, with and without an interrupt; the recovery prompt of today when
the terminal in the journal has no marker, when the journal has no
terminal, and when the recovered session has no waiting state; a second
restart during the wait; a forced end that starts a new turn settles the
result of the new turn, with each event of that turn appended one time
and in order; an event that reaches the open read after the forced end
is appended exactly one time; a delta that arrives during the wait is
counted when the limit ends the turn (the test fails when the new wait
for receipts is removed); continue and settle the result of the
provider-started turn; a second wait does not start the clock again; the
limit with no turn active; the limit and a comment during a running turn
(not interrupted); a running turn that ends and waits again after the
limit; Stop; a `run.terminal` that the Runner held; the Runner stream
ends during the wait; the limit under the timeout of the run; a start
time from an earlier controller; a plan that a provider-started turn
proposes settles as the plan review wait.
- `src/native-session-runtime.test.ts`, not changed by this PR: 297
passed.
- Mutation checks, each made tests fail: no continue at a waiting
terminal; no limit; no early end; no clock from an earlier controller;
the clock starts again at each wait; a running turn is interrupted at
the limit.
- Mutation checks for the forced end, each made the two new tests fail:
the forced-end signal used on each loop (the run ends as
`native_finalization_missing`); a new read started after the forced end
(the first event of the new turn is lost).
- Mutation checks for the restart, each made named tests fail: the
recovery prompt not held back; the wait not started again in
`consumeTurn`; the wait started again on the session state alone (the
test with no marker in the journal fails).

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** No terminal has the marker before part 4.
- **`consumeTurn` runs for each native turn of each agent.** It makes
one small object for each turn and reads one more payload field at a
terminal.
- **If the interrupt that ends the wait is not available or fails,** the
limit forces the end: the consumer returns the last terminal, or
continues to read when the forced end starts a new turn. This is a
fallback path. On a forced return, one read of the event stream stays
open until the session closes, as before this PR.
- **A wait that starts again after a restart holds back the recovery
prompt.** This is the one change to the recovery flow. It needs the two
conditions above, and both exist only for a run that asked for
provider-started turns.
- **Known limit.** The driver does not clear its waiting state when the
consumer ends a wait with an interrupt. A restart in the short time
after that can start the wait again; the recorded clock and the comment
check then end it.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 05:59:42 +00:00
Nicky LeachandClaude Fable 5.2 82ce8e64ba feat(runner): let the session driver admit a turn that the provider started (#21453)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - Today the run ends when the turn ends. The command stops, and the
run fails with `native_finalization_missing`.
> - The Runner part of this work is complete in #21316 and the pull
requests below it: when a controller sends `providerStartedTurns` on
`turn.start`, the Runner keeps Claude while a background shell is
running, and gives the cycle of Claude to the controller as a turn that
the provider started.
> - Core reads Runner events through the Codex app-server driver, also
for ACPX agents. The driver refuses a turn that the controller did not
start, except a goal turn of Codex.
> - This pull request lets the driver accept a turn that the provider
started, only directly after a turn that the Runner marked as waiting.
> - The benefit is that the next parts can use that turn. Nothing
reaches this code until the last part sends the opt-in.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one of four parts on the Core side. They are
stacked on #21316:

1. The driver admits a turn that the provider started.
2. The consumer waits for that turn, for a limited time.
3. The server ends the wait when a person comments, and keeps the clock
across a restart.
4. The transport sends the opt-in. Only this part changes what a
production run does.

This is part 1. Its base branch is
`feat/adopt-claude-provider-started-turn`. Refs #19912 (design).

**What existing behavior does this improve?**

Nothing in production yet. What the driver does with a turn that the
provider started.

**Subsystem affected**

Paperclip Runner package, the controller side: `src/drivers/codex` (the
driver that Core uses to read a native session).

**Current behavior**

- A `turn/started` that the controller did not ask for fails the session
with `turn_binding_mismatch`, except for an active Codex goal.
- An interrupt with no active turn fails with "already terminal".

**Proposed behavior**

- A turn terminal that the Runner marked with `awaitingProviderTurn` and
that has no result puts the driver in a waiting state. The driver event
`turn.completed` carries the marker.
- In that state, a `turn/started` with `origin: "provider"` is admitted
the same way as a goal turn, and the driver event carries `origin:
"provider"`. In each other state it still fails closed.
- In that state with no active turn, an interrupt gives an
acknowledgement item with the status `provider_turn_wait_ended`. It
sends nothing to the Runner. The driver also gives that acknowledgement
when it gets the internal `paperclip/runTerminal` notification with a
private marker; a notification without the marker does nothing.
- An interrupt for a provider-started turn that is running is sent to
the Runner only when it names that turn.
- The driver saves the waiting state in the persisted session
(`awaitingProviderTurn`, written only during a wait). After a controller
restart, the restored session still admits the provider-started turn and
still acknowledges an interrupt during the wait. The driver restores the
state only when the session has no active turn, no stored result and at
least one finished turn, so a new run cannot take the wait of the run
before it.
- When a restart finds that the saved active turn is finished, the turn
read of the transport and `reconcile()` of the driver carry the same
marker as the live `turn/completed` notification.
`runnerd-codex-transport.ts` copies `awaitingProviderTurn: true` from
the saved `turn.completed` event to the turn object of a read.
- The driver ends the waiting state when the Runner reports the run
terminal.

**Reason and benefit**

The consumer in the next part needs these three signals: a marked
terminal, an admitted turn, and a clear end of the wait.

## What Changed

- `codex-session-terminal.ts`, `codex-session-notifications.ts`,
`codex-session-state.ts`, `codex-harness-session.ts`,
`codex-driver-values.ts`: 37 lines added, 4 removed.
- `codex-app-server-driver.provider-turn.test.ts` (new, 232 lines).
- `contracts/harness-driver.ts`, `contracts/native-session-backend.ts`,
`backends/harness-driver-backend.ts`, `codex-app-server-driver-impl.ts`:
the optional field in the two persisted shapes and its copy in each
direction.
- `live/runnerd-codex-transport.ts`: the marker on a recovered turn
terminal. The loop that builds the earlier terminals of a `thread/read`
is now the function `runnerdPriorTurnTerminals`, with the same logic, so
that a test can call it.

## State saved by the previous release

- One new optional field in the persisted driver session:
`awaitingProviderTurn`. A snapshot outside a wait has the shape of the
previous release.
- **State from the previous release, read by this release:** the field
is absent, so the driver does not wait. This is the behavior of today.
- **State from this release, read by the previous release:** the
previous reader copies the fields that it knows and ignores this one. No
reader rejects it, and the integrity digest covers the stored object as
written.

## Verification

At the head of this branch:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json` and `pnpm run build:typescript` in
`packages/paperclip-runner`; `node
scripts/verify-runner-vendor-dependencies.mjs` and `pnpm typecheck` in
`server`; `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`: all passed.
- 6 driver tests: a provider-started turn after a marked terminal is
admitted and can report a result; a provider-started turn with no marked
terminal fails closed; an interrupt during the wait is acknowledged; a
running provider-started turn is not interrupted by the end of the wait;
the marked notification ends the wait and a notification with no marker
does not; a normal terminal is not changed.
- Restart tests: a session restored from a snapshot with the waiting
state admits the provider-started turn and settles its result; a
snapshot outside a wait has no new field; a restored session with no
waiting state refuses a provider-started turn as today; an interrupt
during a restored wait is acknowledged. A saved active turn that the
read reports finished with the marker gives a `turn.completed` with
`awaitingProviderTurn: true`; with no marker it is an ordinary terminal.
- `harness-driver-backend.test.ts`: the field goes through a serialized
checkpoint only during a wait. `runnerd-codex-transport.test.ts` (324
passed): the marker is on a recovered terminal only for a
`turn.completed` with a boolean `true`.
- `src/native-session-runtime.test.ts`: 297 passed, with the real
`paperclip-runnerd`.
- Mutation checks for the restart work, each made a named test fail: no
restore of the state; no copy in the backend; no copy in the transport;
no marker argument in `reconcile()`; no end of the wait at the run
terminal.
- Mutation checks: the acknowledgement branch removed, the admission
removed, and the marker check removed each made a test fail.

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** The Runner sends the marker only after the
opt-in of part 4.
- **The admission rule is shared with Codex goal turns.** The existing
goal tests pass with no change.
- **Known limit after a controller restart.** If the Runner already runs
a provider-started turn and the saved active turn is the turn before it,
recovery fails closed, as today for each unexpected active turn.
- **A controller turn that starts during a wait** clears the waiting
state when its `turn/started` arrives, not when it is sent. A snapshot
in that short time can hold an active turn and the field together; the
restore rule drops the field.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 20:45:49 -08:00
Nicky LeachandClaude Fable 5.2 f7b1651799 feat(acpx-sidecar): adopt Claude's own cycle as a provider-started turn when the Runner arms it (#21316)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - The parts before this one give the adapter an ordered stream of
those cycles (#21421), declare the contract (#21422), and make
`paperclip-runnerd` wait for such a turn (#21423).
> - This pull request is the last part in the Runner. The sidecar reads
the stream, and gives a cycle to `paperclip-runnerd` as a
provider-started turn only when `paperclip-runnerd` armed it.
> - The benefit is that a later change in Core can let a run continue
when background work finishes. Until Core sends the opt-in, a production
run does the same as today.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request was first opened with the full change (49 files). It
is now split. The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. This pull request: the sidecar adopts a cycle.

Its base branch is `feat/runnerd-await-provider-turn` (#21423). The code
at the head of this branch is the same as before the split; only the
base changed. Review comments made before the split are about code that
is now in the parts above. Refs #19912 (design).

**What existing behavior does this improve?**

What the ACPX sidecar does with a cycle that Claude starts by itself.

**Subsystem affected**

Paperclip Runner: `src/cli/acpx-runtime-sidecar.ts`. Claude only.

**Current behavior**

- The sidecar does not read the cycle stream of the adapter (#21421), so
each cycle is declined and the background state becomes unknown.
- The sidecar does not know the command `session.provider_turn`, so
`paperclip-runnerd` (#21423) never waits.

**Proposed behavior**

- The command `session.provider_turn` with `armed: true` arms the
sidecar for one turn. The sidecar accepts it only for the terminal that
reported a running background shell, with no active turn and a provider
that is alive. One flag holds the arming. It is cleared at each turn
start and turn end, on suspend and close, and when the provider is lost.
- One loop, `consumeProviderCycles`, reads the stream. For each cycle:
- if the sidecar is armed and no turn is active, the sidecar adopts the
cycle: it emits `runtime.turn_started` with `{ "origin": "provider" }`,
then the events, the usage and the terminal of the cycle, as for a
normal turn;
- if the cycle cannot be decided yet (the prompt turn is still settling,
or `paperclip-runnerd` did not arm yet), the sidecar holds it for a
maximum of 5 seconds, and then adopts or drops it;
- in each other case the sidecar drops the cycle. The stream ends, and
the background state becomes unknown, so `paperclip-runnerd` restarts
the provider at the end of the turn as today.
- A Paperclip tool call or a permission request of a held cycle waits
for the decision of its own cycle. It is never bound to the prompt turn
or to a different cycle.
- If an adopted cycle proposes a plan (`ExitPlanMode`), the sidecar
publishes the same `plan.proposed` event as for a controller turn, and
the turn ends with the same terminal: completed, with no
`backgroundTasks`, so `paperclip-runnerd` does not wait again. A
controller turn is not changed.
- If the provider exits while the sidecar is armed with no turn active,
the sidecar sends a `runtime.diagnostic` with the code
`ACPX_PROVIDER_PROCESS_LOST`, and `paperclip-runnerd` ends the wait.
- `turn.cancel` for a provider-started turn cancels the cycle and closes
the provider.
- With no arming, nothing is shown. No controller sends the opt-in yet,
so `paperclip-runnerd` never arms.

**Reason and benefit**

- Only `paperclip-runnerd` decides that a cycle becomes a turn. The
sidecar cannot start a turn by itself.
- One reader of one ordered stream means that two cycles cannot overlap.

## What Changed

- `src/cli/acpx-runtime-sidecar.ts` (127 lines added): the command, the
arming flag, `consumeProviderCycles`, `adoptProviderCycle`,
`pumpProviderTurn`, and the wait of `waitForTool` for a cycle that is
not decided.
- `src/cli/acpx-runtime-sidecar.provider-turn.test.ts` (new, 320 lines)
and two fake hosts in `src/test-support` (227 lines). One host follows a
script. The second runs the real cycle stream and the real background
task tracker.
- `src/cli/acpx-runtime-sidecar.test.ts` (2 lines): one more parameter
for a test harness.

No schema, no Rust and no persisted state changes in this part.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `pnpm run typecheck:typescript`, `npx tsc --noEmit -p
tsconfig.test.json` and `pnpm run build:typescript`: no errors. The
server check `node scripts/verify-runner-vendor-dependencies.mjs`, the
cache guard and the contract tests: passed.
- `npx vitest run src/cli src/drivers/acpx
src/live/runnerd-acpx-provider-turn-wait.test.ts
src/drivers/runner-tool-bridge.test.ts
src/native-session-runtime.provider-turn.test.ts`: `128 passed (128)`
files, `2147 passed`, 9 skipped.
- `cargo test -p paperclip-runner-core --lib` and the `acpx_*` suites:
all passed. No Rust changed in this part.
- The sidecar tests of this part: a cycle that starts before the arming
command, or while the prompt turn settles, is held and then adopted with
its tool call; a cycle is never shown when `paperclip-runnerd` did not
arm, disarmed, armed for a different terminal, armed for a quiet
terminal, or armed after the provider exited; two cycles in sequence
with `{ live: 2 }`, then `{ live: 1 }`, then an unknown state; cancel; a
lost provider; Codex gets no change.
- Mutation checks, each made the named tests fail: the sidecar arms with
no matching terminal; no hold; the old hold condition that drops a cycle
while the prompt turn settles; no wait of a tool call for the decision.
- Live run with a real Claude account, not in CI, with the real ACP
runtime and the real Anthropic API (model `claude-sonnet-5-5`), at the
adapter level: one background shell gave `{ live: 1 }`, one cycle with
its own usage, then an unknown state. Two shells of 8 s and 28 s gave `{
live: 2 }`, `{ live: 1 }`, then unknown. All Paperclip tool calls were
admitted, and none was refused.

Not run: a live run through the real sidecar process,
`paperclip-runnerd` and Core (that path needs an API key, not a local
sign-in); Linux; `pnpm verify --full`. CI supplies the broad coverage.

## Risks

- **The feature is off.** No controller sends the opt-in, so the sidecar
is never armed. The new code that runs today is the loop that reads the
stream: with no arming it drops each cycle, and the background state
becomes unknown as before.
- **A follow-up for a command that finishes while Claude reports on an
earlier one is lost.** The Runner waits again only while a background
command is still running. The run ends as it does today. The reason is
in #21421.
- **A cycle that starts in the last second of a prompt turn** can have
its first messages in the prompt turn and the rest in the
provider-started turn. Tool calls stay safe: the admission rule of
#21278 refuses a call that the active turn did not announce.
- **The 5 second hold** is not tested under load. A cycle that is held
longer is dropped, and the run takes the path of today.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 19:50:22 -08:00
Nicky LeachandClaude Opus 5.5 4a78124a3e feat(runner): wait for a provider-started Claude turn when the controller asks (#21423)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - `paperclip-runnerd` ends a run when a turn ends. For a Claude turn
that ended with no result while a background shell is running, this
stops the shell and the run fails.
> - #21278 binds each tool call to its turn, and #21293 reports how many
background shells are running at the end of a turn. Part 4 declared the
contract.
> - This pull request makes `paperclip-runnerd` wait for a turn that
Claude starts by itself, when the controller asks for it, and admit that
turn into the same run.
> - The benefit is that the Runner has one place that decides to wait
and one place that ends a wait.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 5. Its base branch is `feat/prp-provider-turn-contract`.
After each part, a production run does the same as today, because no
controller sends the opt-in. Refs #19912 (design).

**What existing behavior does this improve?**

What `paperclip-runnerd` does after a Claude turn ends with no result
while a background shell is running. Nothing changes until a controller
sends the opt-in.

**Subsystem affected**

Paperclip Runner: the ACPX provider backend in `runner-core` (Rust).
Claude only.

**Current behavior**

- After each completed turn the Runner emits `run.terminal`. A turn with
a running background shell gets the provider restart, and the shell
stops.

**Proposed behavior**

Nothing changes unless the controller sends `providerStartedTurns: true`
in the payload of `turn.start`. No controller sends it yet.

With that field:

- The Runner keeps the provider and waits when all of these are true:
the agent is Claude; the turn completed; the run has no accepted
`paperclip_finish` or `paperclip_block` result; the terminal event
reported `backgroundTasks` with `live` above zero; each condition of
#21293 for a kept process is true; the sidecar accepted
`session.provider_turn` with `armed: true`.
- `turn.completed` then has `awaitingProviderTurn: true`, and no
`run.terminal` follows. The Runner holds that `run.terminal`.
- When the sidecar reports a turn that the provider started, the Runner
admits it only during a wait and publishes `turn.started` with `origin:
"provider"`. In each other case that report fails the session closed.
- When a wait ends with no turn active, the Runner publishes the held
`run.terminal`. This applies when the provider process exits, the
sidecar exits, the controller cancels or stops the run, the session is
closed or suspended, and after a Runner restart.
- The held `run.terminal` and the removal of the wait are saved in one
write, and the controller gets the event from the next poll. A crash
after a cancel or a stop cannot lose it, and it is never sent two times.
- A controller `turn.start` during a wait replaces the provider first,
as before, and the held event is dropped.
- A `run.attach` from a different run during a wait ends the wait when
the attach succeeds. The Runner then discards the held `run.terminal`,
because that event is of the old run and must not end the new run. This
is the same rule that a Runner restart applies. The wait is removed only
in the state that the attach saves for the new run. If the attach fails,
for example when the old session does not stop, the old run keeps the
wait and gets its held `run.terminal` one time, from the next poll or
from a cancel or a stop. A `run.attach` from the same run publishes the
held event to that run.
- The Runner waits at most 5 seconds for the sidecar to answer a request
to arm or to disarm a provider-started turn. This is the limit that turn
cancellation already uses. Before, the limit was the 30 seconds of an
ordinary request, and a cancel, a stop or a shutdown had to wait behind
it. No answer is handled as a lost sidecar: the session fails closed,
and the Runner still publishes the held `run.terminal` one time.
- The sidecar of this branch does not know `session.provider_turn`, so
the arming request fails and the Runner does not wait. The behavior
comes with part 6.

**Reason and benefit**

- The wait is one entry in the saved session state. One rule ends it at
each save, so each new way to leave the open session ends the wait with
no new code.
- A wait must not outlive the provider or the sidecar, and a cancel must
stop it.

## What Changed

- `acpx_provider_turn_wait.rs` (new, 230 lines with its unit tests):
`provider_turn_wait { turn_id, held_run_terminal }` in the saved state;
the rule that ends a wait at save; `settle_turn_events`, which decides
at one place to arm, to create the wait and to put
`awaitingProviderTurn` on `turn.completed`.
- `acpx_provider_turn.rs` (new, 119 lines): the offer of a turn at its
terminal, the arming request, and `admit_provider_turn`.
- `acpx_provider_session.rs`, `acpx_provider_backend.rs`,
`acpx_provider_state.rs`, `provider_events.rs`,
`acpx_park_preparation.rs`: the opt-in from `turn.start`; the record
that the run has a result; the provider-started turn in the state and in
the PRP events; cancel and stop with no turn; no turn-end restart and no
park during a wait.
- `acpx_sidecar_transport.rs` (11 lines): during a wait, a poll with no
event also checks that the sidecar process is alive. A child process can
keep the output stream of a dead sidecar open.
- `bin/fake-acpx-sidecar.rs` (232 lines): provider-started turns and
failures for tests.
- Tests (1,164 lines): `tests/acpx_quiet_turn_continuation.rs`,
`tests/support/acpx_quiet_turn_restart.rs`,
`src/live/runnerd-acpx-provider-turn-wait.test.ts` (the real daemon),
`src/native-session-runtime.provider-turn.test.ts` (the controller of
today).
- `runner/DURABLE_TRANSPORT.md`: the rules.

## State saved by the previous release

- One new optional field in the saved ACPX session state:
`providerTurnWait`. It is written only during a wait. In each other
state the saved file has the shape of the previous release.
- **State from the previous release, read by this release:** the field
is absent, so there is no wait. A unit test covers this.
- **State from this release, read by the previous release (a
rollback):** outside a wait, the file loads as before. During a wait,
the previous Runner refuses the file with its "ACPX provider state is
malformed" error, because its loader refuses each unknown field. Only a
run that waits at the moment of a rollback is in this case, and no run
waits before a controller sends the opt-in.
- **Runner restart during a wait:** the wait is restored. The provider
is a new process, so the wait is over, and the controller receives the
held `run.terminal` one time. A test with the real daemon covers this.
- **Controller of today or of the previous release:** `turn.start` has
no opt-in. A test runs a turn with a running shell and a turn with an
unknown background state, and compares the events, the commands after
the turn, the restart, and the park state before and after the restart.
They are the same.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed.
- `cargo test -p paperclip-runner-core --lib`: 644 passed, 4 ignored.
All 17 `acpx_*` suites passed.
- `src/live/runnerd-acpx-provider-turn-wait.test.ts`, 5 cases with the
real `paperclip-runnerd` and the fake sidecar through PRP: a wait ends
with exactly one `run.terminal` when the sidecar exits while a child
process keeps its output open, when the provider is lost, when the
controller cancels, and after a Runner restart; with no opt-in,
`run.terminal` follows the turn and the sidecar is never armed.
- Rust tests: wait and admit; no wait with a result; no wait with no
opt-in; an active goal neither marks nor arms a wait; a sidecar that
refuses to arm; a provider-started turn that the Runner did not wait for
fails the session closed; cancel and stop during a wait; provider and
sidecar loss; restart; the saved shape in both directions.
- `src/native-session-runtime.provider-turn.test.ts`: the real
`consumeTurn` of today gets the event sequence with no opt-in and
finalizes it as `native_finalization_missing`, as today.
- Attach during a wait, in `tests/support/acpx_quiet_turn_restart.rs`: a
different run attaches after the snapshot reports `warmAttachReady:
true`; the new run gets no `run.terminal`, the saved state has no wait,
and the old session is closed. A same-run attach gives exactly one
`run.terminal`. An attach that fails when the old sidecar is dead leaves
the wait with the old run, and the old run then gets exactly one
`run.terminal` from a poll, `run.cancel`, `turn.interrupt` or
`turn.stop`. Mutation checks: with no fix, the old `run.terminal` is in
the pending events of the new run; with the wait ended before the attach
is complete, the old run gets no `run.terminal` after a failed attach.
- No answer from the sidecar, in
`tests/support/acpx_quiet_turn_restart.rs`: the fake sidecar does not
answer the disarm request; `run.cancel` and `turn.interrupt` each return
`already_settled` in less than 10 seconds, exactly one `run.terminal`
reaches the run, `session.close` completes in less than 2 seconds with
no second `run.terminal`, and the provider process is gone. With no
answer to the arm request, the turn ends with its `run.terminal` and
with no wait. With the limit removed, each test fails at 30 seconds.
- Mutation checks, each made the named tests fail: the rule that ends a
wait at save; no `run.terminal` after a restart; arming with no opt-in;
the marker set with no wait; the sidecar process check removed.

Not run: a rollback with the binary of the previous release; a run with
the opt-in and a sidecar that answers the arming request with an error
(the code reads each error as not armed); Linux; the full suite. CI
supplies the broad coverage.

## Risks

- **The feature is off.** No controller sends the opt-in.
- **A rollback of the Runner during a wait** makes the previous Runner
refuse that saved session. See "State saved by the previous release".
- **The wait has no time limit in the Runner.** The controller must
close, suspend or cancel. Core will own that limit.
- **Park is refused during a wait.** The controller must end the wait
before it parks.
- **A new run can attach during a wait.** The attach is not blocked,
because the Runner has no time limit on the wait and a new run can send
only `run.attach`. After an attach that succeeds, the old run gets no
`run.terminal` from the Runner; the controller must not depend on that
event. The request that disarms a sidecar that the attach keeps has no
test, because no test can keep a session across an attach during a wait.
- **After a sidecar that does not answer, `runner.suspend` returns an
error at once** ("provider lifetime cleanup is not yet proven"), and
`session.close` completes. This is the behavior of today for each lost
sidecar; this PR does not change it.
- **The shared code that changed** runs for each Claude turn: the record
of a run result, and the settle step at each turn end.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 19:21:10 -08:00
DottaandClaude Opus 5.5 e82c4f0a31 fix(runner): prove a warm Runner's suspension on a slow sandbox host so the next run does not fail (#21472)
**DO NOT MERGE until the coordinator confirms the Daytona validation.**

This PR extends the failed-close retry that #21442 (merged) added. It is
based on `master`, which contains #21442.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A warm remote sandbox is reused by the next run of the same task.
Before the next run starts, the previous run's Runner must be retired:
suspended, with durable proof.
> - An idle warm close had 10 s for everything. On a loaded sandbox host
one liveness probe or one read of the durable state used that up. The
suspend was never sent, or it completed but was never proven.
> - The next run then replayed that cached close failure, failed before
any provider work, and the task showed a failed run with Retry
> - For a follow-up on a task blocked on its subtasks, the automatic
continuation was refused and the Retry stayed until another wake
> - This pull request bounds each probe, gives the suspend proof its own
read window, lets a later close prove the suspension again, and lets
that follow-up continue by itself
> - The benefit is that a slow host no longer turns a completed
suspension into a failed task, and the person sees no Retry for it

## Linked Issues or Issue Description

**What happened?**

Local release-swap cells on Daytona (G5c and G6c, 2026-11-13) each
showed one Retry banner: "previous run's session could not be retired
(runner did not durably suspend before checkpoint)". G6c had no fault
injection, so this is a real class.

- In the G6c case, the Runner completed its suspend and exited. The
server's proof read (a sandbox exec under load) ran out of the remaining
0.6 s of the close grace, so the suspension was "not proven".
- In the G5c case (an adopted Runner), the liveness probe
`adoptedRunner.isAlive()` took about 220 s and used the whole close
deadline. The suspend was never queued. Five adopted sessions failed the
same way in the same second.
- The successor run's retirement called `transport.close()`, which
returned the cached rejection of the predecessor's failed close. It did
not send a new suspend and did not read the state again.
- The successor failed before provider work. Its fresh-sandbox
continuation (`interrupted_preparation`) was refused as "blocked after
recovery", because the parent task was blocked on its subtasks before
the follow-up started. The task showed a failed run with Retry for 27 s,
until a blocker-stall wake.

**Expected behavior**

A completed suspend is proven, even on a slow host. A follow-up that
fails before any provider work on a task that was already blocked
continues by itself, and the task shows as working while that
continuation is queued.

**Steps to reproduce**

1. Close an idle warm Runner whose durable state read takes longer than
the close grace, or whose liveness probe stalls.
2. On master the close fails and the next close replays the same
failure.
3. The new tests in `runnerd-codex-transport-close-proof.test.ts` and
`heartbeat-native-predispatch-retry.test.ts` do this.

**Paperclip version or commit**

`master` at c6b497d831 (contains #21442).

## What Changed

- `runnerd-codex-transport.ts` (controller-side TypeScript):
- Each liveness probe in the close is bounded. A probe that does not
answer counts as possibly alive, never as gone, and the suspend goes
over the live connection. A connected Runner's probe gets a 1 s slice,
so drain and suspend keep their budget.
- Once the suspend command's completion is observed, what the
durable-state read shows decides how long the proof may take:

    | Read after the completion | Proof window | Maximum close time |
    | --- | --- | --- |
    | Suspended, right Runner | proven at once | unchanged |
| Another run's or Runner's state | ends at the close deadline, as on
master | close grace (10 s; 45 s for Pi) |
| Right Runner, not yet `suspended` | settle window of 3 s from the
first such read (`RUNNER_SUSPENSION_PROOF_SETTLE_MS`, test option
`closeProofSettleMs`) | close grace + 3 s |
| Unreadable or slow read | full proof window of 30 s from the
completion (`RUNNER_SUSPENSION_PROOF_READ_MS`, `closeProofReadMs`) |
close grace + 30 s |

All rows add the existing bounded exit-probe slices. runner-core saves
the `suspended` lifecycle before it delivers the suspend result
(`persist_lifecycle_before_command_delivery` in `durable/runner.rs`), so
a readable non-suspended state after the completion can only be a stale
remote read; the settle window covers a few read attempts and is not a
write-ordering wait.
- The settlement diagnostic names the branch that ended the window
(`proofWindowEnd`: `proven`, `wrong_identity`,
`not_suspended_after_settle`, `unreadable_after_window`,
`suspend_not_completed`).
- A close that failed only because readable proof was missing is retried
by the next close: it reads the durable state again with a fresh bound
and finishes settled. A close whose proof was refuted stays memoized, as
on master. This extends the failed-close retry from #21442.
  - The close failure carries per-step timings.
- `native-runtime/predecessor-retirement.ts`,
`heartbeat/run-completion.ts`: the predecessor-retirement fallback
records the task's status when the run failed.
- `modules/run-dispatch/domain/policy.ts`,
`modules/run-dispatch/adapters/postgres.ts`: a continuation of a run
that did no provider work, on a task that was already blocked then, is
not treated as blocked after recovery. Its dependency check follows the
wake it continues. The continuation keeps a person's comment wake.
- `task-interruption-presentation.ts`: while that run's failure handling
queues the continuation (up to 30 s), the task shows as working with no
Retry. The window ends when the continuation is scheduled, when the run
records `continuationScheduling: "refused"` (written by
`run-completion.ts` when the fallback does not schedule it), or after 30
s. The fallback record alone no longer ends it.

## Verification

Red on master, green on this branch:

- `runnerd-codex-transport-close-proof.test.ts`
- `proves a completed suspend on its own read window when the durable
state read outlasts the close grace`
- `suspends five adopted idle Runners whose liveness probes stall
together, instead of spending the close on the probe`
- `heartbeat-native-predispatch-retry.test.ts`
- `continues a follow-up on a blocked task whose predecessor could not
be retired`
- `does not continue a task that was blocked only after its run failed`
- `task-interruption-presentation.test.ts`
- `shows the task as working while its fresh-sandbox continuation is
being queued`
- `keeps the task working after the fallback is recorded while its
continuation is still being scheduled` (review fix)
  - `offers Retry when the continuation was refused or never queued`
- `runnerd-codex-transport.test.ts` (review fixes)
- `gives a suspend that completes near the close deadline its whole
proof window from the completion`
- `bounds the proof window from the observed completion, so a proof read
that never answers still ends`
- `proves a suspension whose suspended state appears 1.5 s after the
command completed`
- `fails a suspension whose state stays ready after the settle window,
not after the whole proof window`
- `fails at the close deadline when the durable state belongs to another
run`
- The three existing tests in
`runnerd-codex-transport-suspension.test.ts` that expect a conclusive
negative to fail within the close grace pass unchanged.

Local runs on this branch: `tsc --noEmit` is clean for `server` and
`packages/paperclip-runner`. Passing on this branch:
`runnerd-codex-transport-suspension`, `-close-proof`, `-adoption`,
`-remote-retention`, `runnerd-detach-durability` (all files), the
proof-window cases of `runnerd-codex-transport.test.ts`,
`task-interruption-presentation.test.ts` (72), `run-dispatch` policy and
Postgres adapter tests and `heartbeat-native-predispatch-retry.test.ts`
(166 together).

Conflict resolved: `task-interruption-presentation.ts` gained
`WAITING_CONTINUATION_MOVE_FORWARD_MS` on master (#21385). Both
constants are kept.

Runner content ID: the Runner change is controller-side TypeScript under
`packages/paperclip-runner/src` only. There is no Rust, `protocol/` or
`generated/` change. Existing Runner images work with this controller.

Daytona: this fix has not run on Daytona at all. The two banners above
are the evidence of the class (G5c and G6c ran on 588c6ba267, without
this fix).

## Risks

- Not proven on Daytona.
- A close can now take up to 30 s longer when the proof read is
unreadable or slow, and up to 3 s longer when the read is stale. This is
on the close path only, not before `turn.start`.
- A probe that does not answer now counts as possibly alive, so the
close sends a suspend instead of treating the Runner as gone.
- Ownership: this PR changes
`server/src/modules/run-dispatch/domain/policy.ts`,
`server/src/modules/run-dispatch/adapters/postgres.ts` and
`server/src/services/heartbeat/run-completion.ts`. CODEOWNERS does not
list these files. Their recent changes come from the restart-recovery
and Retry work (#21348, #21256, #20863, #21302). That area should review
the dispatch-policy change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 18:32:22 -06:00
Nicky LeachandClaude Opus 5.5 1332ab987f feat(prp): declare the contract for provider-started turns (#21422)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - The Runner, its sidecar and the controller speak two protocols with
schemas: PRP between the controller and the Runner, and the sidecar
protocol between `paperclip-runnerd` and the ACPX sidecar.
> - A turn that the provider starts needs new fields and one new command
in those protocols.
> - This pull request declares them and generates the readers again. It
changes no behavior.
> - The benefit is that the next two pull requests show behavior only,
with the contract already in review.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 4. Its base branch is `feat/claude-provider-cycle-stream`.
After each part, a production run does the same as today, because no
controller sends the opt-in. Refs #19912 (design).

**What existing behavior does this improve?**

Nothing in production. The protocol schemas.

**Subsystem affected**

Paperclip Runner: `protocol/` and the generated TypeScript and Rust
contract code.

**Current behavior**

- PRP has no way to say that a turn was started by the provider, or that
the Runner waits for such a turn.
- The sidecar protocol has no command to arm the sidecar and no event
for a turn that the provider starts.

**Proposed behavior**

- PRP (`schemas/provider-turn.schema.json`, new): `providerStartedTurns`
(boolean) in the payload of `turn.start`, `origin: "provider"` on
`turn.started`, and `awaitingProviderTurn: true` on `turn.completed`.
`command-v3` and `event-v3` refer to it.
- Sidecar protocol (`provider-schemas/acpx-sidecar.schema.json`): the
command `session.provider_turn` with `{ turnId, armed }`, the event
`runtime.turn_started` with `{ origin: "provider" }`, and the payload of
the diagnostic for a lost provider. The schema enforces that payload for
a `runtime.diagnostic` with the code `ACPX_PROVIDER_PROCESS_LOST`.
Diagnostics with other codes keep their open payload.
- `paperclip-runnerd` decodes `runtime.turn_started` and refuses it. No
sidecar sends it yet. The Runner of today refuses an unknown event too.
- No controller, Runner or sidecar uses the new fields in this pull
request.

**Reason and benefit**

The repository rule for Runner contracts says that new fields must be in
the schema with generated readers. A separate pull request lets a
reviewer read the contract alone.

## What Changed

- `protocol/schemas/provider-turn.schema.json` (new),
`command-v3.schema.json`, `event-v3.schema.json`,
`protocol/provider-schemas/acpx-sidecar.schema.json`,
`protocol/manifest.json`.
- `scripts/generate-acpx-sidecar-contract.mjs` and
`scripts/generate-protocol-schema-module.mjs`.
- Generated: `src/drivers/acpx/generated-sidecar-contract.ts`,
`src/protocol/generated/schema-bundle.ts`,
`src/protocol/generated/standalone-validators.ts`,
`runner/crates/runner-core/src/generated_acpx_sidecar_contract.rs`.
- `runner/crates/runner-core/src/acpx_event_payload.rs` (3 lines): the
Rust `match` on the event type must name the new event to compile.
- `test/protocol-contract.test.mjs`.
- The contract digests of the pack change (`prp.acpx-sidecar`,
`prp.command-v3`, `prp.event-v3`, `rust`, and the new
`prp.provider-turn`). No stored file refers to these digests. No
provider `commandDigest` and no profile version changes.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed. The contract tests: 32 passed.
- The sidecar suite and the tracker tests: 141 passed.

Not run: the full suite. CI supplies that coverage.

## Risks

- **The pack contract digest changes.** Older image packs are not used
again.
- **The opt-in field is accepted by the schema but has no effect yet.**
A controller that sends it before part 5 gets the behavior of today.
- **State saved by the previous release:** this pull request changes no
persisted shape.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 00:20:45 +00:00
Nicky LeachandClaude Opus 5.5 e5ad224cc5 feat(acpx): give Claude's own cycles to the adapter as one ordered stream (#21421)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - Claude Code starts a cycle of its own when a background task
finishes and no prompt is in flight. The Runner sees the raw messages of
that cycle.
> - #21293 only marks the background state as unknown when it sees such
activity.
> - This pull request gives each such cycle to the caller of the adapter
as one ordered stream: started, its events, ended with its result.
Nothing reads the stream yet.
> - The benefit is one place that knows where a cycle starts and ends,
with cycles that cannot overlap.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 3. Its base branch is
`feat/acp-session-update-event-converter`. After each part, a production
run does the same as today, because no controller sends the opt-in. Refs
#19912 (design).

**What existing behavior does this improve?**

Nothing in production yet. How the ACPX adapter sees the cycles that
Claude starts by itself.

**Subsystem affected**

Paperclip Runner: `src/drivers/acpx` (the adapter, the runtime host, the
Claude background task tracker). Claude only.

**Current behavior**

- The raw ACP observer gives each message to the background task
tracker.
- Activity of Claude with no client request in flight makes the
background state unknown. The messages of that cycle are not kept.

**Proposed behavior**

- `ClaudeProviderCycles` puts each cycle on one ordered stream. A cycle
starts at the first update of the own session with no client request and
no prompt in flight. It ends at the `usage_update` that has
`_meta._claude/origin`.
- A caller gets the stream with `providerCycles()`, reads it with
`next()`, and calls `adopt()` or `drop()` for each cycle. A permission
request waits for the decision of its own cycle. The stream records the
cycle of each tool call, and `decisionFor(toolCallId)` gives that
decision. The binding that a caller gives to `adopt()` can also carry
`onPlanProposal`, so a plan that an adopted cycle proposes goes to the
caller as it does for a normal turn.
- With no caller, a cycle is declined and the background state becomes
unknown, as before. The sidecar does not call `providerCycles()` in this
pull request, so production does the same as today.
- The tracker reports `{ live: N }` only while N shells are running, `{
live: 0 }` only if the process never had a background task and Claude
never acted between prompts, and unknown in each other case. A dropped,
failed or cancelled cycle makes the state unknown.
- An empty result that directly follows the end of a cycle (zero usage,
same context use and cost) is ignored. Claude Code sometimes sends it.
It must not become a cycle, and its usage must not be reported two
times.

**Reason and benefit**

- One producer and one reader make the order of cycles a property of the
structure.
- The tracker does not read the order of messages to decide which
finished task a cycle reported on. Captures showed that Claude Code can
put two notifications in one cycle, start a second cycle, or give a
notification to a cycle in progress, and the ACP stream does not say
which.

## What Changed

- `src/drivers/acpx/claude-provider-turns.ts` (new, 303 lines):
`ClaudeProviderCycles`.
- `acpx-runtime-adapter.ts` (78 added, 8 removed) and `runtime-host.ts`
(24 added): `providerCycles()` and `providerTurnsAvailable()`, and the
permission path that waits for the decision.
- `claude-background-tasks.ts` (16 added, 7 removed): the rule above.
- Tests (764 lines): `claude-provider-turns.test.ts`,
`claude-provider-turn-tasks.test.ts`, `claude-background-tasks.test.ts`,
a permission test in `acpx-runtime-adapter.test.ts`, two tests with the
real `claude-agent-acp`
(`acpx-runtime-adapter.claude-provider-turn.test.ts`,
`acpx-runtime-adapter.claude-overlapping-shells.test.ts`), and a
captured raw stream as a fixture.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed.
- 10 test files, 294 passed. This includes the test of #21293 with the
real `claude-agent-acp`, not changed: with no caller of the stream it
still gets `{ live: 1 }` at the end of the turn and an unknown state
after the cycle of Claude.
- Real `claude-agent-acp` with a local fake Anthropic API: a background
shell finishes, and the stream gives one cycle with its own usage. With
two shells where the second finishes before the first cycle sends its
first update, the first cycle ends with an unknown state.
- Mutation checks, each made the named tests fail: an end item before
the last message of a cycle; a drop that does not make the state
unknown; a finished task that reports `{ live: 0 }`; the empty result
not ignored; a permission request that does not wait.
- Live run with a real Claude account, not in CI, on the full change:
one background shell gave `{ live: 1 }`, one cycle, then unknown. Two
shells of 8 s and 28 s gave `{ live: 2 }`, `{ live: 1 }`, then unknown.

Not run: the full suite. CI supplies that coverage.

## Risks

- **The time at which the state becomes unknown changes a little.** With
no caller, a cycle makes the state unknown at its first message. Before,
it was at the first message of an activity kind. The result for the
Runner is the same: a restart at the end of the turn.
- **Limit, by design.** After a cycle, `{ live: N }` counts only running
shells. A command that finishes while Claude already reports on an
earlier one gets no cycle that the Runner waits for.
- **The rule for the empty result is narrow** and comes from captures of
one Claude Code version. The reason that Claude Code sends it is not
known.
- **A queue of 4,096 messages for one cycle.** A cycle that is larger
fails, and the state becomes unknown.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 15:48:53 -08:00
Nicky LeachandClaude Opus 5.5 1d112a504f feat(acpx): convert raw Claude session updates the way ACPX does (#21420)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - The Runner sees each raw ACP message before ACPX does. A cycle that
Claude starts by itself is not a part of an ACPX prompt, so ACPX gives
no events for it.
> - To show that cycle in the same form as a normal turn, the Runner
must convert the raw messages itself. ACPX 0.22 does not export its
converter.
> - This pull request adds a copy of that converter with a test that
compares it with ACPX. Nothing uses it yet.
> - The benefit is that a later pull request can show the cycle of
Claude with the same events as a normal turn.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 2. Its base branch is
`refactor/acpx-sidecar-runtime-event-step`. After each part, a
production run does the same as today, because no controller sends the
opt-in. Refs #19912 (design).

**What existing behavior does this improve?**

Nothing yet. It adds a module that a later part uses.

**Subsystem affected**

Paperclip Runner: `src/drivers/acpx`. One new direct dependency in the
Runner and the server.

**Current behavior**

- ACPX converts `session/update` messages to runtime events inside a
prompt. The function is private.
- The Runner has no way to make the same events from a raw message.

**Proposed behavior**

- `acp-session-update-event.ts` converts a raw `session/update` to the
event that ACPX 0.22 makes for the same update.
- It validates the message with `@agentclientprotocol/sdk` 1.9.0. This
is the version that ACPX 0.22 already installs.
- No production code imports the module in this pull request.

**Reason and benefit**

A copy with a comparison test is a smaller risk than a patch of ACPX.
The test fails when the two differ.

## What Changed

- `src/drivers/acpx/acp-session-update-event.ts` (new, 367 lines).
- `acp-session-update-event.test.ts` and
`acpx-runtime-adapter.claude-session-update-events.test.ts` (new, 363
lines).
- `packages/paperclip-runner/package.json` and `server/package.json`:
`@agentclientprotocol/sdk` 1.9.0 as a direct dependency. The server
needs the line because it copies the Runner build with no
`node_modules`.
- `pnpm-lock.yaml` is not in this pull request. The lockfile workflow
owns it.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed.
- The two new test files and the sidecar suite: 130 passed.
- The comparison test runs the real ACPX runtime with the pinned
`claude-agent-acp` and a local fake Anthropic API, and gives the same
raw updates to the copy. The events are the same.
- A second test fails if the SDK version of the Runner is not the
version that ACPX resolves. With 1.4.0 in `package.json` it fails.

Not run: the full suite. CI supplies that coverage.

## Risks

- **A new direct dependency.** It is already installed through ACPX, so
no new package is downloaded. The version test keeps the two the same.
- **The converter is a copy of ACPX code.** An ACPX upgrade must run the
comparison test.
- **The module is not used yet.** It is used in part 3.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 15:14:59 -08:00
Nicky LeachandClaude Opus 5.5 055a0a3c54 refactor(acpx-sidecar): extract the runtime event step and the test process helper (#21419)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - The last part of that stack adds a second kind of turn to the
sidecar. It needs two pieces of the code of a normal turn.
> - This pull request moves those two pieces into functions, and moves a
test helper to a file of its own. It changes no behavior.
> - The benefit is that the later pull request shows only new behavior,
not moved code.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 1. Its base branch is
`refactor/remove-direct-gateway-path`. After each part, a production run
does the same as today, because no controller sends the opt-in. Refs
#19912 (design).

**What existing behavior does this improve?**

Nothing for a user. The structure of the ACPX sidecar code.

**Subsystem affected**

Paperclip Runner: `src/cli/acpx-runtime-sidecar.ts` and its test.

**Current behavior**

- `pumpTurn` has the code that prepares each runtime event for the
Runner, and the code that sends the terminal frame of a turn, inline.
- The sidecar test file has the `SidecarProcess` helper that starts a
sidecar process.

**Proposed behavior**

- `createRuntimeEventStep` has the event code. `emitTurnTerminal` has
the terminal code. `pumpTurn` calls both.
- `src/test-support/acpx-sidecar-process.ts` has the helper, so a second
test file can use it.
- No protocol, event or timing change.

**Reason and benefit**

A later pull request adds a turn that Claude starts by itself. It uses
the same two functions, and its tests use the same helper.

## What Changed

- `src/cli/acpx-runtime-sidecar.ts`: 74 lines added and 56 removed. The
code moved with its comments, with no change.
- `src/cli/acpx-runtime-sidecar.test.ts` and
`src/test-support/acpx-sidecar-process.ts`: the helper moved, 138 lines
added and 132 removed.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed.
- `npx vitest run src/cli/acpx-runtime-sidecar.test.ts`: 125 passed. The
tests are the same as before, apart from the import of the helper.

Not run: the full suite. CI supplies that coverage.

## Risks

- **A refactor of the turn loop of each ACPX agent.** The moved code
runs for Claude, Codex, Pi, Kimi, Cursor, Copilot and Grok. The existing
sidecar suite covers it. No new test is necessary, because no behavior
changes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 14:39:55 -08:00
DottaandClaude Opus 5.5 342fb308bd perf(run-events): commit streaming text deltas in batches (#21460)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - One server process stores every event of every running native agent
in `heartbeat_run_events`
> - Text deltas (`item.delta`) are about 85% of that process's SQL
statements: each delta is one transaction of four statements on the
shared 10-connection pool
> - At 50 agents and 20 deltas per second, the process falls behind and
unrelated agent requests wait for the pool
> - This pull request commits text deltas in batches: one transaction
for all deltas that arrive within 250 ms, with one multi-row insert
> - The benefit is about 7 times fewer transactions and 5–6 times fewer
statements per delta, with no delay for the first text of an item

## Linked Issues or Issue Description

Refs #21435 (this pull request is stacked on it). Refs #21131. Item 1a
of the agent scaling plan in #21418.

**What existing behavior does this improve?**
The persistence of native text deltas. `native-session-runtime` awaited
`PaperclipControlPlanePort.appendEvent` for each event, and the port ran
one transaction per event.

**Subsystem affected**
Server run event ingest (`paperclip-control-plane-port.ts`,
`heartbeat-run-events.ts`) and the runner package's turn consumer
(`native-session-runtime.ts`).

**Current behavior**
Each delta takes one transaction: BEGIN, run lock, append, COMMIT. In
the load model with 50 agents (deltas only), that is about 4.0
statements and 1.0 transaction per delta.

**Proposed behavior**
The turn consumer starts the next append without waiting for a delta's
receipt. The port collects deltas and commits them in one transaction
with one insert. In the same load model, that is about 0.7 statements
and 0.14 transactions per delta.

**Reason and benefit**
Fewer transactions free the shared pool and the main thread for other
agents' requests. This is the largest per-agent cost in the scaling
plan.

**Breaking changes**
None. The row shape does not change. A port without the new capability
keeps the earlier serial behavior.

## What Changed

- `appendHeartbeatRunEventBatch` (`heartbeat-run-events.ts`) appends
native source events of one run and one source in one transaction. When
the events continue the stored source cursor (from #21435), it sends
four statements for the whole batch: the run lock, one replay probe, one
`insert ... select from jsonb_array_elements(...)` of every row, and the
commit. Any other batch (a replay, a gap, an event with a wake, or a run
without a cursor) appends its events one by one in the same transaction,
with the single-append checks.
- **Storage choice: one row per delta, written by one statement per
batch.** I did not store one row per batch. The transcript, live events,
run log export, chat progress, replay (`replayEvents`), recovery cursors
and the UI all read one row per delta. One statement per batch keeps
every reader unchanged, and rows written before this change read the
same.
- `PaperclipControlPlanePort` sets `pipelinesStreamingEvents`. An
`item.delta` appended with the new `streaming` option goes into a queue:
- It commits at once until a delta with text commits after the latest
other event. So the first text of each item, and the time to first
token, never wait for the timer.
- Later deltas commit after at most 250 ms, or at once when the batch
has 100 deltas.
- Every other event, `checkpointSession`, `completeRun`, `replayEvents`
and `provesNoRunProviderWork` first commit the queue. A non-streaming
event is never committed before the deltas that preceded it.
- A delta appended without `streaming` commits at once, so callers that
wait for each receipt are not slowed.
- The receipt of a delta settles only after its batch commits and its
observers (live event, executor callback) run, in order.
- If a batch fails, every delta in it rejects and nothing from it is
stored.
- `native-session-runtime` `consumeTurn` appends deltas with `streaming`
and does not wait for their receipts (at most 256 pending). Before any
other event, any result, and the governed-wait settlement, it calls
`flushStreamingEvents()` and then waits for all pending receipts, so a
boundary never waits for the timer. A failed delta stops the turn before
a later event is appended. The consumer joins every pending delta append
before it settles, also when the provider stream fails.
- If one observer of a committed batch fails, only that delta's receipt
rejects. The other rows are still published and observed.
- `ControlPlanePort` contract: optional `pipelinesStreamingEvents`,
`flushStreamingEvents()` and `AppendControlPlaneEventOptions.streaming`.

## Verification

- Red/green. The new tests failed before the change and pass after it:
- Runner: "starts text delta appends without waiting and flushes them
before any other event" (`native-session-runtime.test.ts`). The port in
the test has no timer, so the turn finishes only if the boundary
flushes. Before: the appends were serial, and later the boundary waited
for the timer.
- Runner: "joins pending delta appends before a failed turn settles".
Before: the turn settled while a delta commit still ran.
- Port: "observes every committed delta of a batch when one observer
fails".
- Port: "commits the first text delta at once and later deltas in one
batch before the next other event". It checks one transaction and one
insert for five deltas, flush before a non-streaming event, receipts
only after commit, and immediate commit of the first delta with text.
Before: each delta committed alone.
- Port: "flushes queued deltas on the timer and before a replay read,
and rejects every delta of a failed batch". It checks the
crash-before-commit case: a revoked writer fails the batch, nothing is
stored, and the resent deltas commit idempotently (the stored one is a
duplicate).
- Store: four new tests in `heartbeat-run-event-atomic-append.test.ts`:
one lock, one probe and one insert per batch; replay idempotency and
payload conflict; a batch that fails before commit stores nothing and
its resend commits; the one-by-one path for no cursor and for gaps.
- Focused suites pass: `heartbeat-run-event-atomic-append`,
`heartbeat-run-event-sequencing`, `paperclip-control-plane-port`
(including the package conformance suite),
`paperclip-semantic-conformance`, `native-observer-replay`,
`native-event-boundary-golden`, `live-recovery-work`,
`native-interaction-bridge`, `chat-native-progress-work`, and
`native-session-runtime.test.ts` in the runner package.
- `pnpm --filter @paperclipai/paperclip-runner run typecheck:typescript`
and `pnpm --filter @paperclipai/server run typecheck` pass. `pnpm
check:module-boundaries` passes.
- Load model (`scripts/benchmark-agent-scaling-load.ts` from #21418,
`AGENTS=50 SECONDS=20`, local embedded PostgreSQL). The load model calls
the store directly, so for this run I added a local `BATCH_MS` mode that
applies the port's batching rules (first delta at once, 250 ms timer,
100 cap, flush before other events) and does not wait for delta
receipts. That mode is not in this pull request. The machine was very
busy (load average 40–60), so absolute figures are noisy; per-delta
ratios are the stable result:

| Deltas only | Statements/s | Transactions/s | Deltas done of offered |
Statements per delta | Transactions per delta | Lateness p50 / p99 |
  | --- | --- | --- | --- | --- | --- | --- |
| One transaction per delta | 2,920 | 730 | 14,668 of 20,000 | 3.98 |
1.00 | 3,074 / 5,384 ms |
| Batched (250 ms) | 704 | 141 | 19,940 of 20,000 | 0.71 | 0.14 | 668 /
2,858 ms |
| One transaction per delta (repeat) | 1,712 | 428 | 8,575 of 20,000 |
3.99 | 1.00 | 6,558 / 11,464 ms |
| Batched (250 ms, repeat) | 856 | 172 | 20,000 of 20,000 | 0.86 | 0.17
| 187 / 556 ms |

With all sources on, the machine was saturated (event-loop utilization
0.96–0.98) in both modes and the results were not usable (764 and 12,800
of 20,000 deltas done).

## Risks

- **Ack and crash (read first).** In the live runnerd path the runner's
ack does not wait for the `heartbeat_run_events` row today.
`DurablePrpControlPlane.#event` acks after the in-memory journal commit.
Streaming events are not in the controller checkpoint. So a delta that
is acked but not yet appended is lost if the server crashes, and the
runner does not resend it. This is the earlier behavior. This pull
request makes that window up to 250 ms longer. The final text still
survives through `item.completed`, which commits immediately and in
order. This pull request does not change `durable-prp-control-plane.ts`.
- Live text in the UI arrives in steps of up to 250 ms after the first
text of an item.
- A delta failure now surfaces at the next non-delta event or at the end
of the turn, not at the delta. The turn stops before it appends a later
event.
- The governed-wait drain and the control-event replay still append one
event at a time. They do not carry deltas.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 15:34:01 -06:00
Nicky LeachandClaude Fable 5.2 0aaab644fe refactor(runner): remove the direct assigned-gateway path (#21311)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - An agent reaches the tools of its app connections through an MCP
gateway.
> - Before #21274, some native harnesses got the gateway URL and a
bearer token and connected to the gateway directly. The Runner has code
for that: an HTTP MCP server for ACPX providers, a server entry in the
Codex `config.toml`, a server entry in the OpenCode config, and the
lists that pass the three `PAPERCLIP_NATIVE_MCP_*` variables to a
sidecar.
> - #21274 gives each native harness its gateway tools through the PRP
relay. Core removes the three variables from each Runner environment.
The direct code path gets no binding.
> - Code that cannot run still needs maintenance, and it can still put a
token in a provider process if a variable arrives by another way.
> - This pull request deletes the direct path from the Runner.
> - The benefit is one path for assigned gateway tools, and no Runner
code that can give a gateway token to a provider.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is in a stack. Its base branch is
`fix/keep-quiet-claude-provider-between-turns` (#21293). It depends on
#21274, which is the bottom of the stack. On `master` without #21274,
the direct path is still in use.

**What existing behavior does this improve?**

The Runner code that connects a provider directly to the assigned MCP
gateway.

**Subsystem affected**

Paperclip Runner: the TypeScript drivers and the Rust `runner-core`
environment lists.

**Current behavior**

After #21274, Core never gives `PAPERCLIP_NATIVE_MCP_NAME`,
`PAPERCLIP_NATIVE_MCP_URL` or `PAPERCLIP_NATIVE_MCP_TOKEN` to a Runner.
The Runner still has the code that reads them and configures a direct
gateway server for ACPX providers, Codex and OpenCode.

**Proposed behavior**

The Runner has no code that reads the three variables to configure a
provider. If the variables are in the environment of a Runner, no
provider environment, sidecar, Codex `config.toml` or OpenCode config
gets them.

**Reason and benefit**

Less code, one path, and no way for a gateway token to reach a provider
process through the Runner.

## What Changed

- Deleted `packages/paperclip-runner/src/drivers/native-mcp.ts`
(`nativeMcpLaunchBinding`).
- `drivers/acpx/runtime-host.ts`: removed the native gateway server
entry and the name conflict check. The admission gate keeps its message.
A run with a binding id is admitted when the sidecar uses the PRP
transport with its tool bridge, as the production sidecar always does.
- `drivers/acpx/environment.ts`: the name and the URL are not in the
provider allowlist.
- `drivers/runtime-context-materializer.ts` and
`live/runnerd-codex-transport.ts`: the Codex home functions have no
`nativeMcp` parameter. The sidecar token pass-through is gone.
- `drivers/opencode/opencode-server-driver.ts`: removed the read of the
binding, its trace redaction value, and the gateway server entry in the
OpenCode config.
- `drivers/stdio-mcp.ts` and `runner-core/src/stdio_mcp.rs`: removed the
reservation of the gateway server name. No gateway server exists, so no
name can conflict. The names `paperclip` and `paperclip_runner_*` stay
reserved.
- `runner-core/src/acpx_sidecar_transport.rs` and `codex_provider.rs`:
the three variables are not in the pass-through lists.
- `control-plane/durable-prp-control-plane.ts`: the three variables are
not in the platform environment list.
- `packages/paperclip-runner/README.md`: the text about direct callers
and assigned servers.
- Kept: the three names stay in the reserved lists
(`src/configured-environment.ts`,
`runner-core/src/configured_environment.rs`). A user cannot configure
them. Tests in both languages now check all three names.
- No server code changed. The relay needs the gateway token and URL on
the control plane.

23 files, 73 lines added and 211 lines removed.

## State saved by the previous release

- This pull request changes no persisted shape. The provider server
lists, the launch profile digests, the protocol versions and the sidecar
contract are the same as with #21274.
- A Runner or sidecar that the previous release started keeps its own
code until its run ends, as #21274 describes.
- A kept Codex home that has the old gateway server in `config.toml` is
written again at the next start and at a recovery of the same run. A
test covers this.
- If the old variables reach a new Runner, no provider gets them. Tests
cover this for ACPX, Codex and OpenCode.

## Verification

From `packages/paperclip-runner`:

- `npx vitest run src/drivers/acpx/runtime-host.test.ts
src/live/runnerd-codex-transport.test.ts
src/drivers/acpx/environment.test.ts
src/drivers/opencode/opencode-server-driver.test.ts
src/live/runnerd-codex-transport-warm-attachment.test.ts
src/drivers/runtime-context-materializer.test.ts
src/drivers/agent-observability.test.ts src/drivers/stdio-mcp.test.ts
src/control-plane/durable-prp-control-plane.test.ts
src/configured-environment.test.ts`: `10 passed (10)` files, `807
passed` tests.
- `npx tsc --noEmit -p tsconfig.json` and `-p tsconfig.test.json`: no
errors.
- `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no
changes.
- `cargo test -p paperclip-runner-core --test acpx_sidecar_transport`:
20 passed. `cargo test -p paperclip-runner-core --lib`: 639 passed, 4
ignored.

Tests that failed with the old code and pass now:

- `runtime-host.test.ts`: "admits a relayed … run with a binding id and
ignores a previous-release gateway binding", for Claude, Codex and
Cursor.
- `runnerd-codex-transport.test.ts`: "drops a previous-release gateway
binding from the … runner environment", for ACPX Claude, Codex, Pi,
Cursor, Copilot and Kimi, and for OpenCode.
- `opencode-server-driver.test.ts`: the OpenCode config has only the
task bridge.
- `runnerd-codex-transport-warm-attachment.test.ts`: a restored Codex
home has no gateway server and no gateway credential in `config.toml`.
- `environment.test.ts`: a provider environment has none of the three
variables.
- Rust
`previous_release_gateway_binding_and_unrelated_secrets_never_reach_the_sidecar`:
a Runner that starts with the old variables gives none of them to the
sidecar, for five agents.

Mutation check: with the three names put back in the Rust pass-through
list, the Rust test fails.

Not run: a live run with a real provider. Not run: `pnpm verify` and the
full suite. CI supplies that coverage.

## Risks

- **This pull request is safe only after #21274.** The stack order keeps
that.
- **A developer who sets the three variables by hand for
`eval-session`** no longer gets direct gateway tools. No document
describes that use.
- **The warm attachment test uses the staged Runner binary from
preflight.** That binary does not have the Rust changes of this pull
request. The Codex path of that test does not use the changed lists.
- The evidence that no entry point supplies a binding comes from a read
of the code: Core removes the variables before it builds each Runner
environment, and a native run cannot use the in-process path.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change, reviewed the diff, and ran the verification and a mutation check
again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), traced each code path and
wrote the change and the tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 13:33:47 -08:00
Devin FoleyandPaperclip 12aa08721c feat(ssh): use the agent AI account in new environments (#21248)
Use the agent-selected AI account for new SSH environments. Preserve
remote-machine authentication for existing environments and expose it as
an advanced option. Keep managed credentials private while preserving
supported conversation state and credential refreshes across turns.

Validated with focused route, adapter, lifecycle, and real-process tests;
final CI passed on d634944df1.
Greptile Apex reviewed that revision at 5/5 with no unresolved threads.
External neutral security reports remain documented in the pull request.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-13 13:20:13 -08:00
DottaandClaude Opus 5.5 2a8b485140 fix(runner): accept object fields sent as JSON text and name wrong-type rejections (#21485)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip runner checks each tool call from the model against
the tool's input schema before it sends the call to the control plane
> - Some model hosts send an object argument as its JSON text. On the
DigitalOcean host, `paperclip_finish` arrived with `continuation` as a
string
> - The runner rejected these calls, and the repeated-rejection guard
then told the model that its arguments "did not arrive", which is wrong
when every field arrived with the wrong type
> - This pull request makes the runner parse such a string back into its
object when that is safe, records each coercion as a run event, and
makes the guard name the actual problem
> - The benefit is fewer stuck finish calls, and guidance that points
the model at the real fault

## Linked Issues or Issue Description

Refs #21398

**What happened?**
Chat-eval cohort 13 ran OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017` on master `df1b12ad8e`.
On the DigitalOcean host, 6 of the 8 remaining "Invalid tool input"
rejections were `paperclip_finish` calls with `continuation` sent as a
JSON string (`/continuation must be object,null`). Examples:
`gateways-profiles.rotate-token`, `instance-operations.restart-service`,
and `questions-and-approvals.multi-part-question`. The run diagnostic
recorded `continuationShape: "string"`. After the second identical
rejection, the guard from #21398 said "If you meant to send other
arguments, they did not arrive". Every argument had arrived; one had the
wrong type.

**Expected behavior**
When a schema field is object-typed, does not allow text, and the text
parses to a plain JSON object that makes the call valid, the runner
accepts the call with the parsed object and logs a run event. When a
call stays invalid, the guard says which required fields are missing and
which fields have the wrong type.

**Steps to reproduce**
1. Start the OpenCode MCP bridge.
2. Call `paperclip_finish` with `reportedWorkDisposition: "yielded"` and
`continuation` as `JSON.stringify({ kind: "response_wake", ... })`.
3. Before this change, the bridge returns `Invalid tool input:
/continuation must be object,null`.

## What Changed

- Added `packages/paperclip-runner/src/drivers/tool-input-coercion.ts`.
`admitToolInput` validates the arguments. If they are invalid, it parses
a string in an object-typed field back into its object. It does this
only when the field's schema declares `object`, does not allow `string`,
the text parses to a plain JSON object, and the whole coerced call
validates. It applies to any object-typed field at any depth, not only
`continuation`.
- The OpenCode MCP bridge and the runner tool bridge use
`admitToolInput`. They dispatch the coerced arguments, and they send one
`paperclip.runner.tool-input-coerced/v1` diagnostic per accepted
coercion. The OpenCode driver already records bridge diagnostics as run
events. The diagnostic names the tool and the field paths. It never
contains the values. The runner tool bridge has a new optional
`onDiagnostic` option.
- When parsed text is still invalid, the rejection describes what is
wrong inside the parsed object, for example `/options/mode must be
string`.
- The repeated-rejection guard now says "Required fields X did not
arrive" only for missing required fields. For fields with the wrong
type, it says "Every field you sent arrived, but one has the wrong type:
continuation must be an object or null, not text".

## Verification

- `cd packages/paperclip-runner && npx vitest run
src/drivers/opencode/mcp-bridge.test.ts
src/drivers/runner-tool-bridge.test.ts` (63 tests pass).
- Red/green: with the source changes reverted and the new tests kept, 5
tests fail: coercion acceptance and its run event (both bridges), no
coercion for text fields or non-object text, the wrong-type guard
wording, and the missing-field guard wording. With the change, all of
them pass.
- `npx tsc --noEmit -p packages/paperclip-runner` passes.
- A narrow paid run of `questions-and-approvals.multi-part-question`,
`runs-adapters.inspect-failed-run` and `budgets-costs.explain-hard-stop`
ran on a branch that combines this PR with the two sibling cohort-13
PRs. The results are in a PR comment.

## Risks

- Low risk. Coercion runs only after a call fails validation, and the
call is accepted only if the coerced form validates. A field that allows
text is never parsed, so a valid string never changes meaning.
- Previous-release story (upgrade-safe run state): this change adds no
persisted run, session, launch or pack state. It changes only how one
in-flight tool call is admitted. Saved state carries forward unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 15:06:37 -06:00
Nicky LeachandClaude Fable 5.2 0ebe6d846d perf(runner): keep a quiet Claude provider between turns of a run (#21293)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar. The
Runner stops and starts the sidecar and Claude at each turn boundary.
> - The reason was that a late tool call from an earlier turn could run
in a later turn. #21278 removes that reason for Claude: the sidecar
refuses a call that the active turn did not announce.
> - A second reason remains. Claude Code works by itself between
prompts. When a background task finishes, Claude starts a cycle of its
own. If the process stayed alive with a live background task, that cycle
could start between turns or mix with the next turn.
> - So the Runner can keep the process only when it knows that Claude
has no background work.
> - This pull request adds that knowledge and uses it. After a Claude
turn that completed with no background work, the Runner keeps the
process for the next turn of the same run.
> - The benefit is that a later turn of the same run starts with no
provider restart. This is also the base for the next step, which lets a
run continue when background work finishes.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is stacked on #21278. Its base branch is
`fix/bind-claude-tool-calls-to-turn`. Refs #19912 (design), #12419
(restart at turn start), #20145 (restart at park) and #20190 (restart at
turn end).

**What existing behavior does this improve?**

The provider restart between turns of the same run, for Claude through
ACPX.

**Subsystem affected**

Paperclip Runner: the ACPX sidecar (TypeScript) and the ACPX provider
session in `runner-core` (Rust). Claude only.

**Current behavior**

- When a completed turn settles, the Runner restarts the sidecar and
Claude (`restart_session_after_turn`, #20190).
- If that restart did not run, the next turn restarts them at its start
(`start_turn`).
- The Runner does not know if Claude has background tasks.

**Proposed behavior**

- The sidecar asks `claude-agent-acp` for its async task reports. This
is an optional capability of the pinned `claude-agent-acp` 0.73.0. The
patch does not change.
- A tracker reads those reports and the id of each background Bash
command. It reports `{ live: 0 }` only when it knows that this process
never had a background task and that Claude did not act between prompts.
In all other cases the state is unknown.
- The sidecar adds `backgroundTasks` to the terminal event of a
completed Claude turn, and to `session.snapshot`.
- The Runner keeps the process between two turns only when all of these
are true:
  - the agent is Claude;
  - the turn completed (not failed, cancelled or interrupted);
  - the terminal event reported `backgroundTasks: { "live": 0 }`;
  - each existing condition of the turn-end restart is true;
- before the next turn, a new `session.snapshot` reports the same
identity and run, no active turn, no pending work, and `{ "live": 0 }`;
  - the settled-turn ledger and the tool replay ledgers are not full.
- In each other case the Runner restarts as before.
- A kept process keeps its tool call-id tombstones and its settled turn
identities.

**Reason and benefit**

- A turn that follows a quiet turn in the same run starts without a
restart.
- The decision is safe by default: unknown state means a restart.

## What Changed

TypeScript (`packages/paperclip-runner/src`):

- `drivers/acpx/claude-background-tasks.ts` (new):
`ClaudeBackgroundTaskTracker`. It reads raw ACP messages. It reports
unknown when the capability is not negotiated, when a task finished,
when a live task is not a shell command, when an update is malformed,
when a limit is reached, when Claude acts with no client request in
flight, or when Claude calls `CronCreate` or `ScheduleWakeup`.
- `drivers/acpx/provider-failure.ts` and `acpx-runtime-adapter.ts`:
Claude sessions also advertise the `asyncTasks` capability. The raw ACP
observer gives each message to the tracker. ACPX 0.22 drops the
`async_task_*` updates before the turn event stream, so only the raw
observer sees them.
- `drivers/acpx/runtime-host.ts`: `backgroundTaskState()`.
- `cli/acpx-runtime-sidecar.ts`: the terminal event of a completed
Claude turn carries `backgroundTasks` in the same payload as
`goalAtTerminal`, with the same size limit. `session.snapshot` carries
it when no turn is active.
- `drivers/acpx/claude-tool-use-admission.ts`:
`hasRoomForAnotherTurn()`. When the admitted-id ledger cannot hold one
more full turn, the sidecar reports unknown, so the Runner restarts
before the ledger is full.

Rust (`packages/paperclip-runner/runner/crates/runner-core`):

- `acpx_provider_session.rs`: the session records a quiet turn from the
terminal event. `start_turn` continues in the same process only under
the conditions above, and then uses `prepare_turn`, which keeps the
tombstones.
- `acpx_provider_generation.rs`: `can_continue_after_turn`,
`continue_after_quiet_turn`, and the snapshot check
`background_work_is_quiet`.
- `acpx_park_preparation.rs`: the turn-end restart does not run for a
quiet Claude turn.
- `acpx_provider_backend.rs`: `session.reconciled` reports a replaced
process only when the turn start replaced it.
- `bin/fake-acpx-sidecar.rs`: new option `--background-tasks` for tests.
- `runner/DURABLE_TRANSPORT.md`: the rule for a kept sidecar.

Contract (`packages/paperclip-runner/protocol`):

- The `backgroundTasks` report is `$defs.backgroundTasks` in
`provider-schemas/acpx-sidecar.schema.json`: an object with only `live`,
a non-negative integer. The schema validates it on
`runtime.turn_terminal`.
- `scripts/generate-acpx-sidecar-contract.mjs` generates the TypeScript
type and reader and the Rust reader from it. The tracker, the sidecar
and the Rust session use the generated code.
- `fixtures/provider-boundary/acpx-background-tasks.json` has shared
test documents. The TypeScript reader, the Rust reader and the schema
are tested with the same accepted and refused shapes.
- This changes the contract digest of the provider pack
(`prp.acpx-sidecar` and `rust`). Older image packs are not reused. No
provider `commandDigest` and no profile version changes.

Not changed: park and run attachment. A process that ran a turn is still
restarted at park, and only a process that ran no turn is bound to the
next run. Other ACPX agents and Codex do not change. The
`claude-agent-acp` patch does not change.

## State saved by the previous release

- This pull request adds no persisted state. The three new fields of the
session are in memory only.
- A terminal event or a snapshot without `backgroundTasks` means
unknown. An older sidecar therefore gets the restart, as before.
- An older Runner with a new pack ignores the field and restarts at each
turn, as before.
- After a Runner restart, the in-memory state is gone. The next turn
restarts the provider, as before. A test covers this.

## Verification

From `packages/paperclip-runner`:

- `npx vitest run src/drivers/acpx/claude-background-tasks.test.ts
src/drivers/acpx/acpx-runtime-adapter.claude-background-tasks.test.ts
src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts
src/drivers/acpx/claude-tool-use-admission.test.ts
src/cli/acpx-runtime-sidecar.test.ts
src/drivers/acpx/acpx-runtime-adapter.test.ts
src/drivers/acpx/runtime-host.test.ts`: `7 passed (7)` files, `363
passed` tests.
- `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p
tsconfig.test.json`: no errors.
- `pnpm run typecheck:typescript` (it includes the contract checks): no
errors. `node --test test/acpx-sidecar-contract.test.mjs`: 17 passed.
- `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no
changes.
- `cargo test -p paperclip-runner-core --test
acpx_quiet_turn_continuation --test acpx_park_preparation --test
acpx_park_readiness --test acpx_provider_turns --test
acpx_provider_session --test acpx_warm_attachment --test
native_provider_backend`: 4, 33, 8, 31, 12, 6 and 47 passed. No test
failed.
- `cargo test -p paperclip-runner-core --lib`: 637 passed, 4 ignored.
- Mutation check: when the Rust code accepts each live count as quiet,
two tests in `acpx_park_preparation` fail.

New tests:

- **Real ACP runtime**
(`acpx-runtime-adapter.claude-background-tasks.test.ts`), with the
pinned `claude-agent-acp`, the pinned Claude Code client and a local
fake Anthropic API. A foreground Bash turn reports `{ live: 0 }`. A
background Bash in the same process reports `{ live: 1 }` at the
terminal event, and unknown after Claude ran its own cycle for the
finished task. Each turn has its own usage receipt. One process serves
all turns.
- **Tracker unit tests** (13): spawn, completion, failure, stop, pause,
a task that is not a shell, a subagent, no negotiation, malformed
updates, activity between prompts, scheduling tools, session load
replay, limits.
- **Sidecar tests**: the terminal event carries the state for Claude and
not for Codex. A full ledger reports unknown.
- **Rust** `tests/acpx_quiet_turn_continuation.rs` (4) and
`tests/support/acpx_quiet_turn_restart.rs` (7):
- a quiet Claude turn continues in the same process, and a replay of a
settled call id is refused;
  - live, unknown and malformed reports restart;
- background work that appears after the terminal event restarts at turn
start;
  - failed, cancelled and interrupted turns restart;
  - Codex with a quiet report restarts;
  - an active goal restarts;
  - 4,096 kept turns, then the next turn restarts;
- park after a kept process still prepares a new process, and the next
run attaches to it;
  - a Runner restart after a kept process resumes in a new process.
- The existing restart tests did not change. The fake sidecar sends no
report by default, so they are the tests for the unknown case.

Live run with a real Claude account, not in CI: the real ACP runtime
(ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code
client), a real tool bridge, and the real Anthropic API with model
`claude-sonnet-5-5`, on the code at the top of this stack. A turn that
started one background shell reported `{ live: 1 }`. A turn that started
two reported `{ live: 2 }`, then `{ live: 1 }` and `{ live: 0 }` as each
shell finished. A turn in which Claude waited for its background shell
in the foreground reported the unknown state.

Not run: server integration tests (the fake sidecar there sends no
report, so their process counts do not change), Linux, and a live run
through the real sidecar process and runnerd.

## Risks

- **All Claude sessions now negotiate the `asyncTasks` capability.**
`claude-agent-acp` then sends `async_task_*` updates and adds a marker
to a background Bash update. ACPX drops the updates. No other behavior
of `claude-agent-acp` changes, from a read of its code.
- **Park after a kept process pays the restart again**, as it did before
#20190. With real Claude that restart was about 1.5 s.
- **A small window remains** between the snapshot and the prompt of the
next turn. Claude can start a cycle of its own in that window only
without a background task, because a process that had a background task
is not kept.
- **Activity between prompts is found only when no client request is in
flight.** A request that never gets a response would hide such activity.
- **Background watchers cannot be told apart from other tasks**, so each
task that is not a shell command makes the state unknown.
- **Each kept turn adds one `session.snapshot` request.**
- A multi-turn run with real Claude now uses one provider process. This
path has tests with the fake sidecar and with the real ACP runtime. It
has no test with the real sidecar host.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change, reviewed the diff, and ran the TypeScript and Rust verification
again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), investigated the signals
and wrote the change and the tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 12:54:43 -08:00
7773b52f05 fix(codex): preserve sandbox certificate trust in Codex shell commands (#20364)
<!-- Written in ASD-STE100 style: short sentences, active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner and the Codex adapters run agent shell commands
inside a sandboxed Codex session with a strict environment allowlist
> - The controller builds `shell_environment_policy.include_only` before
the sandbox injects certificate-trust variables such as `SSL_CERT_FILE`
and `GIT_SSL_CAINFO`
> - The allowlist drops the injected variables, so managed `git` and
`gh` commands cannot verify HTTPS against a runtime-provided CA
> - This pull request allows the certificate-trust variable names
through every environment policy stage, from the Rust process supervisor
to the shell allowlist
> - The benefit is that managed `git` and `gh` work again in sandboxes
that require a runtime CA, while provider-credential isolation and the
bounded allowlist stay in place

## Linked Issues or Issue Description

- Fixes: #20019
- Refs #20038 — community pull request for the same defect. This change
reviews the approach independently. The commit credits the community
author as co-author.

## What Changed

- Add `CERTIFICATE_ENVIRONMENT_KEYS` (`SSL_CERT_FILE`, `SSL_CERT_DIR`,
`GIT_SSL_CAINFO`, `NODE_EXTRA_CA_CERTS`, `DENO_CERT`,
`REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`) to
`packages/paperclip-runner/src/configured-environment.ts`
- Native Codex shell policy (`codex-security-config.ts`): always put the
certificate names in `include_only`; set inheritance to `"all"` with
`ignore_default_excludes=true`, bounded by the explicit allowlist
- Pass the certificate names through the app-server environment filter,
the ACPX spawn environment filter, the ACPX runtime sandbox
`include_only`, the runner platform environment keys, and the Rust
process supervisor base key list
- Legacy and ACPX-engine Codex shell allowlists
(`packages/adapter-utils/src/codex-shell-environment.ts`): always
include the certificate names
- Add a synthetic private-CA HTTPS journey test: a real `git ls-remote`
fails TLS verification without the injected CA and succeeds with it,
through the policy-filtered child environment
- Document the contract in `doc/sandbox-shell-environment.md` and link
it from `doc/DEVELOPING.md`

## Verification

- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/drivers/codex/codex-security-config.test.ts
src/configured-environment.test.ts
src/drivers/codex/codex-shell-certificate-trust.test.ts
src/drivers/codex/codex-app-server-driver.context-control.test.ts
src/drivers/acpx/runtime-sandbox.test.ts` — 104 tests pass
- `pnpm --filter @paperclipai/adapter-utils exec vitest run
src/codex-shell-environment.test.ts src/acpx-engine/execute.test.ts` —
270 tests pass
- `pnpm --filter @paperclipai/adapter-codex-local exec vitest run
src/server/execute.test.ts` — 10 tests pass
- Codex driver lifecycle, conformance, recovery, and boundaries suites
pass (103 tests); environment-builder suites pass (307 tests)
- Red proof: with only the source fix reverted, the new regressions
fail. The journey test shows `GIT_SSL_CAINFO` is undefined in the shell
environment, and `git` fails TLS verification against the private-CA
fixture
- `cargo check` passes for `paperclip-runner-core`; `rustfmt --check`
passes for the edited Rust file
- The journey test needs `git` and `openssl`; it skips when either tool
is absent

## Risks

- The native shell policy now always sets
`shell_environment_policy.inherit="all"` with
`ignore_default_excludes=true`. Before, it set `"none"` when no
credentials or task keys were projected. The explicit `include_only`
allowlist still bounds every inherited name, so no new secret names can
cross
- The allowlist grows by seven non-secret certificate-trust names. Their
values point at CA files. The sandbox filesystem policy still controls
access to those files; this change grants no filesystem access
- No persisted run, session, or launch identity changes shape. Sessions
saved by the previous release resume unchanged; the new policy applies
when the next launch builds its arguments
- Provider credentials keep their existing exclusions at every stage

## Model Used

- Claude Fable 5 (Anthropic, model ID `claude-fable-5`), extended
thinking, agentic tool use, run through Claude Code. The implementation
was written and reviewed independently against the community design in
#20038; the community author is credited as co-author in the commit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending: checks run after this
PR opens)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending review)
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: everlastingcrown <everlastingcrown@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-11-13 14:13:01 -06:00
Nicky LeachandClaude Fable 5.2 7866e9ef05 fix(acpx): admit a Claude tool call only for a tool use its turn announced (#21278)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar. Claude
calls Paperclip tools through a Runner-owned MCP endpoint, the tool
bridge.
> - The bridge identifies a call by its JSON-RPC request id. That id
does not say which turn sent the call. The sidecar binds a call to the
turn that is active when the call arrives.
> - Because of this, the Runner stops and starts the Claude process at
each turn boundary. A late call from an earlier turn then cannot run in
a later turn. This restart also stops all background work that Claude
started.
> - Claude Code already sends the missing identity. Each MCP
`tools/call` has `_meta["claudecode/toolUseId"]`. It is the id of the
ACP `tool_call` that the provider announces first, inside one turn.
> - This pull request uses that id. The sidecar admits a Claude tool
call only if the active turn announced its tool use.
> - The benefit is that a call from an earlier turn is refused by its
identity. This is the first step to keep the Claude process alive
between turns. This pull request does not change the restart.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

Refs #19912 (the design for turn-scoped ACPX MCP authority), #12419 (the
restart at turn start) and #20190 (the restart at turn end).

This pull request is the bottom of a stack. #21274 is merged, so the
base branch is `master`. It uses no code from #21274. The stack order
lets a later pull request remove the direct gateway path after both.

**What existing behavior does this improve?**

How the Runner decides which turn a Claude tool call belongs to.

**Subsystem affected**

Paperclip Runner: the tool bridge and the ACPX sidecar, for the Claude
provider only.

**Current behavior**

- The bridge keys a call by the JSON-RPC request id
(`runner-tool-bridge.ts`).
- The sidecar gives the call to the active turn (`waitForTool` in
`acpx-runtime-sidecar.ts`).
- After `session/cancel`, Claude Code does not stop an MCP request that
is in flight. A test run showed two calls from two turns live at the
same time on one MCP session. Request ids cannot tell them apart.

**Proposed behavior**

- The bridge can read a provider tool-use id from `params._meta` and
give it to the handler. It keys replays by that id.
- For Claude, the sidecar admits a call only when the turn event stream
of the active turn contains the ACP `tool_call` with the same id, and no
earlier call used that id.
- The announcement and the call use different channels. If the call
arrives first, the sidecar waits for the announcement for a maximum of 5
seconds. The wait stops when the turn ends or the call is cancelled.
- A call with no id, an id from an earlier turn, an id that was never
announced, or an id that was used before gets a tool error. There is no
fallback.
- The call id that the sidecar sends to runnerd is `tool-use:<id>` for
Claude.

**Reason and benefit**

- A late call from an earlier turn cannot run in a later turn. The
identity of the call prevents this, not only the process restart.
- A later change can stop the restart between turns.

## What Changed

- `packages/paperclip-runner/src/drivers/runner-tool-bridge.ts`
- New option `providerToolUseIdMetaKey`. Without it the bridge does not
change.
- With it, the bridge reads `params._meta[key]`. It refuses a value that
is not a bounded string. It gives a valid value to the handler as
`providerToolUseId`.
- The replay key is the tool-use id. A retry with a new request id
replays the first result. The same id with different arguments is
refused.
  - `notifications/cancelled` still finds the call by its request id.
-
`packages/paperclip-runner/src/drivers/acpx/claude-tool-use-admission.ts`
(new): `ClaudeToolUseAdmission`.
- `observe` records an id only from an event with `type === "tool_call"`
and `tag === "tool_call"`. A `tool_call_update` does not count, so a
late update about an old tool use cannot admit it.
- `admit` applies the rule above. An id is used when it is admitted.
Limits: 4,096 announcements for each turn and 8,192 admitted ids. At a
limit, new calls are refused.
- `packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts`
- The sidecar creates the admission object only for `agent === "claude"`
and sets the bridge option only for Claude.
- `pumpTurn` gives each turn event to `observe`. `waitForTool` calls
`admit` and then checks the turn and run again.
- The internal `get_task_context` call for a completion continuation
does not go through admission.
- The Claude call id does not use the per-process namespace, because a
tool-use id is unique.
- No change to Rust, the sidecar contract, the `claude-agent-acp` patch,
the server, or restart, park and turn lifecycle code. Other ACPX agents
and Cursor Cloud use the same code path as before.

Why 5 seconds: the measured time between the announcement and the call
is 1.1 ms to 14.7 ms. The limit must also include queue and event-loop
delay. 5 seconds is far above that, and a call that cannot be attributed
still gets a fast answer.

## Verification

Commands, from `packages/paperclip-runner`:

- `npx vitest run
src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts
src/drivers/acpx/claude-tool-use-admission.test.ts
src/drivers/runner-tool-bridge.test.ts
src/drivers/runner-tool-bridge.claude-tool-use.test.ts
src/drivers/runner-tool-bridge.claude-schema.test.ts
src/cli/acpx-runtime-sidecar.test.ts
src/drivers/acpx/acpx-runtime-adapter.test.ts
src/drivers/cursor-cloud/cursor-cloud-driver.test.ts`: `8 passed (8)`
files, `384 passed` tests.
- `npx tsc --noEmit -p tsconfig.json`: no errors.

New tests:

- **Real ACP runtime** (`acpx-runtime-adapter.claude-tool-use.test.ts`).
It uses the real `openCodexAcpxRuntime`, ACPX 0.22, the pinned and
patched `claude-agent-acp`, the pinned Claude Code client and a real
tool bridge, against a local fake Anthropic API.
- Turn 1: two parallel tool uses are admitted as `tool-use:<id>`. The
model receives both results. The turn completes.
- Turn 2 in the same session: a replay of a turn 1 id returns the cached
result and does not dispatch again. The same id with different arguments
is refused. `admit` refuses the old id.
- In five runs each call arrived 6.9 ms to 13.2 ms after its
announcement. No call waited.
- **Qualification** (`runner-tool-bridge.claude-tool-use.test.ts`). The
pinned Claude Code client sends `_meta["claudecode/toolUseId"]` on each
`tools/call`, with the id that the API issued. This test must pass
before a Claude Code version change.
- **Admission unit tests** (19): announced then called, called then
announced, never announced, turn end during the wait, cancel, an id from
the previous turn, reuse, two concurrent calls with one id, two turns
with live calls, a subagent call, a bad id.
- **Sidecar process tests** (2), with a fake host, for Claude and for
Codex. The Codex run shows no change.
- **Bridge unit tests** (5).

Mutation checks:

- Change the announcement predicate so that it matches nothing: the real
ACP test and one unit test fail.
- Remove the bridge option in the real ACP test: the test fails.
- Count only `tool_call_update`: the unit test "counts only an opening
ACP tool_call event as an announcement" fails.

Live run with a real Claude account, not in CI: the real ACP runtime
(ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code
client), a real tool bridge, and the real Anthropic API with model
`claude-sonnet-5-5`, on the code at the top of this stack. In three
sessions, 8 Paperclip tool calls were admitted and none was refused.
This includes calls in a prompt turn, in a cycle that Claude started
itself, and in a later turn of the same process. Each call arrived 0.8 s
to 1.7 s after its announcement, so no call waited for a late
announcement.

Not run: a turn through the real sidecar process with the real host.
That needs a qualified provider installation, the runtime sandbox and
the provider lifetime lease. Not run: the Linux builds of the pinned
client and a test under load. Not run: `pnpm verify` and the full suite.
CI supplies that coverage.

## Risks

- **This changes each Paperclip tool call that Claude makes.** If the
announcement does not reach the turn event stream in production, the
sidecar refuses the call after 5 seconds. The real ACP test covers the
stream of ACPX 0.22 with the pinned provider. It does not cover the real
sidecar host.
- **The 5 second limit is not tested under load.** A slow host can delay
the announcement. The call is then refused, and Claude sees a tool
error.
- **A Claude Code version that stops sending the id breaks all Claude
tool calls.** The qualification test finds this before the version
changes.
- **The admitted-id ledger is in sidecar memory.** Today the sidecar
restarts at each turn boundary, so the ledger is small. The limit of
8,192 ids refuses new calls when it is full.
- **Subagent calls and calls from a cycle that Claude starts itself**
are covered by unit tests and fake-host tests, not by the real ACP
stream.
- **An admitted call that is in flight when its turn ends** is refused
by the existing turn cleanup. Claude Code can leave the HTTP request
open. The bridge stops it only on its own cancel or close, as before.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change, reviewed the diff, asked for the real ACP runtime test, and ran
the verification and a mutation check again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 11:56:42 -08:00
DottaandClaude Opus 5.5 38821e061f perf(runner): serialise only changed journal records for a checkpoint (#21437)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - One server process controls every running agent, and its main thread
limits how many agents can run at the same time
> - Each agent has a controller journal. The controller saves a
checkpoint of this journal after each control event and before each
frame it sends
> - Each checkpoint converted every command and every event of the
journal to JSON again, also when only one record changed. The cost grew
with the journal length (up to 500 commands and 4,096 events)
> - This pull request keeps the JSON of each record and uses it again
while the record does not change
> - The benefit is that a checkpoint converts only the records that
changed, so the main thread does less work for each agent

## Linked Issues or Issue Description

Refs #21411 (this change sits on top of its checkpoint queue).

**What existing behavior does this improve?**
The controller journal checkpoint in
`packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts`.
Each flush calls `captureCheckpoint`, which runs `JSON.stringify` on
every command and every recovery event of the journal.

**Current behavior**
A flush after one new event converts the full journal to JSON. With
4,096 events, one flush made about 5 MB of new JSON strings and used
about 14 ms of CPU on the event loop. A hosted CPU profile with about 50
agents showed about 3% of the main thread in this path.

**Proposed behavior**
A flush converts only the records that changed since the last flush. The
other records use the JSON that the controller made before. The saved
records are byte-identical to the records of the full conversion.

**Reason and benefit**
The target is 1,000 agents in one controller process. Checkpoint CPU
must not grow with journal length. After this change, one flush with
4,096 events makes about 2.5 KB of new JSON and uses about 2 ms of CPU.

## What Changed

- Add `CheckpointRecordCache` to `durable-prp-control-plane.ts`. It
keeps the JSON and the part name of each command and event record, keyed
by the record object. It uses the JSON again only when the record has
the same keys, in the same order, with the same values (`===`). Commands
change some fields in place (`status`, `result`, `controllerSeq`), and
this comparison finds these changes.
- `captureCheckpoint` accepts the cache as an optional fourth argument.
Only `setCheckpointPersistence` and `flushCheckpoint` pass it. All other
callers (decode, legacy import, bootstrap inspection) do the full
conversion as before.
- `checkpointChanges` uses loops instead of array copies. Records that
did not change keep the same string, so the comparison is by reference.
- `queueCommand` stores a copy of the command payload
(`structuredClone`). The journal must own its nested objects, because
the cache compares nested objects by reference.
- `durableRecoveryInternals` exposes `captureCheckpoint` and
`createCheckpointRecordCache` for tests.
- Regions changed: `checkpointChanges` and the new cache above
`captureCheckpoint`; `captureCheckpoint`; `durableRecoveryInternals`;
one field of `DurablePrpControlPlane`; the `captureCheckpoint` calls in
`setCheckpointPersistence` and `flushCheckpoint`; the `payload` field in
`queueCommand`. The `#event` path is not changed.
- No change to what a checkpoint saves, when a flush is awaited, or the
order of event commit, acknowledgement and checkpoint. The "at most one
waiting checkpoint" behavior from #21411 is not changed.

## Verification

- New `durable-prp-checkpoint-records.test.ts`:
- A journal with 499 commands and 4,000 events gets one new command. The
test counts `JSON.stringify` calls on journal records during the flush.
Before: 4,499. After: 0 calls on existing records, at most 2 in total
(the size check of the new command and its checkpoint record). The delta
contains only `command:500`. The test then restores a new controller
from the checkpoint and compares its records with a full conversion
without the cache. I saw this test fail on `origin/master`.
- A property-style test with 5 seeds and 600 random steps each: add,
settle and remove commands (with `controllerSeq` renumbering), add,
replay and evict events, streaming and usage events, `structuredClone`
and spread copies of the state, header counters, the same value written
again, and a key-order change. After each step, the records with the
cache must be byte-identical to the records without the cache, in the
same order. I checked that this test fails when the field comparison is
removed (cache by object only).
- Existing `durable-prp-control-plane.test.ts` passes. On this loaded
machine, some 5-second timing tests time out with and without this
change (they also time out on `origin/master`), and pass when I run them
alone.
- Benchmark: `PAPERCLIP_CHECKPOINT_FLUSH_BENCHMARK=1 pnpm exec vitest
run src/control-plane/durable-prp-checkpoint.benchmark.test.ts` (from
`packages/paperclip-runner`). Each case commits one new event and
flushes with a no-op save, 200 times. CPU time is noisy (load average
about 40), the byte and record counts are exact.

| Journal | CPU per flush, before | CPU per flush, after | JSON bytes
per flush, before | after | Records converted per flush, before | after
|
| --- | --- | --- | --- | --- | --- | --- |
| 10 commands, 10 events | 207 µs | 64–80 µs | 20,975 | 2,465 | 20 | 1 |
| 500 commands, 10 events | 1,874 µs | 505–511 µs | 375,068 | 2,466 |
510 | 1 |
| 10 commands, 4,096 events | 14,013 µs | 1,655–2,343 µs | 5,174,786 |
2,473 | 4,106 | 1 |

- `pnpm check:module-boundaries` passes. `tsc -p
packages/paperclip-runner/tsconfig.json --noEmit` passes.

## Risks

- The cache assumes that nested `payload`, `result` and `envelope`
objects of journal records are not edited in place after they enter the
journal. I checked the module: results are stored with
`structuredClone`, envelopes with `structuredClone`, and events are
replaced, not edited. This change also copies the payload in
`queueCommand`. If a later change edits a nested object in place, the
checkpoint could keep old JSON. The property test shows top-level edits
are found.
- Some per-record work is still proportional to the journal (a Map of
part names and a field comparison for each record). It is about 0.4 µs
for each record. Removing it needs dirty tracking in every state
mutator, which this change does not do.
- No change to the saved format, so old and new servers read the same
checkpoints during a deploy overlap.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
file edits) in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 13:21:51 -06:00
DottaandClaude Opus 5.5 d9b977baf2 fix(runner): answer a live Runner's commands while it replays its backlog after a reconnect (#21442)
**DO NOT MERGE until the coordinator confirms the Daytona validation.**

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A native run on a remote sandbox is driven by a Runner process. The
server talks to it over PRP and saves the controller state in a
checkpoint.
> - After a release swap, the incoming server reconnects to Runners that
kept working while no server was attached. Each Runner first replays the
events it built up.
> - The server answered commands one at a time and only after every
event queued ahead of them. It wrote one checkpoint and one
acknowledgement per event. A recovery read sent at reconnect therefore
timed out at 30 s behind the replay, although the Runner was alive and
had answered.
> - A tool call that the previous server started and never answered
blocked every later command, and a failed close was replayed forever.
> - This pull request lets read and barrier answers pass the replay,
extends a wait while the replay still commits, acknowledges a replayed
burst once per batch, delivers every pending command at attach, settles
an orphaned tool call at attach, and retries a close whose drain timed
out.
> - The benefit is that a live Runner is reached after a late reconnect
instead of being declared unanswered and then replaced, which lost its
turn.

## Linked Issues or Issue Description

**What happened?**

Hosted staging, release swaps on 2026-11-13 (06:05Z to 09:45Z). For each
run that crossed a swap we measured the gap from the last Runner output
to the moment the new server recorded the Runner's process identity:

- gap under 150 s: 0 of 9 runs failed
- gap over 420 s: 14 of 15 runs failed
- median gap: 180 s for runs that resumed, 470 s for runs that failed

Authentication took under 1 s. The identity was recorded 4 to 7 minutes
later, because it is written only after the Runner answers `run.attach`,
`runner.drain` and the thread read. Those answers waited behind the
replayed event backlog. The failure codes on the first attempt were:

- `PRP command session.snapshot timed out` and
`codex_history_read_failed: thread/turns/list read failed` (all 10
`ingress` runs in the sample)
- `PRP command runner.drain timed out (runner connected; pending ahead 2
[semantic_tool.result] oldest 1023s)` on the retry

A separate class, `cleanup_quarantined` (23 failures in 3 hours, no swap
involved), showed "prior session cleanup remains incomplete" on every
retry, because a close that failed once kept replaying the same
rejection.

A live Daytona capture showed the stale `semantic_tool.result`: Codex
waited for the result of an exec tool call that the stopped server
owned. It was not an unacknowledged frame.

**Expected behavior**

A live Runner answers recovery commands within its deadline after a late
reconnect. A tool call left by an earlier controller is settled once
(with its durable receipt, or as interrupted with an unknown outcome,
never executed again). A close that failed only because its journal
drain timed out reaches its real outcome on the next close.

**Steps to reproduce**

1. Start a control plane with a commit callback that takes 50 ms per
event.
2. Connect a Runner that replays 1,000 events, then send
`session.snapshot`.
3. On master the snapshot times out at its 30 s deadline. The new test
`answers a command issued during a 1,000-event replay within its
deadline` does this.

**Paperclip version or commit**

`master` at 426e8a35dc.

## What Changed

All changes are controller-side TypeScript under
`packages/paperclip-runner/src`.

- `durable-prp-control-plane.ts`: a `command_result` for
`session.snapshot`, `session.goal.get`, `runner.drain` or
`semantic_tool.result` that reports completed or rejected is applied
when it arrives, not after the events queued ahead of it. Failed
results, lifecycle, attachment and turn commands keep their place.
- `durable-prp-control-plane.ts`: the welcome carries every consecutive
pending command (up to 32, within the existing byte bound). A lifecycle
command or `run.attach` that is not first is never batched.
- `durable-prp-control-plane.ts`: an ordinary event inside a replayed
burst is committed at once but checkpointed and acknowledged once per
batch (when the queue drains, when a critical event arrives, or after 64
events). An ACK is still sent only after the checkpoint that contains
its cursor. The owner fence in `flushCheckpoint` is unchanged. Critical
events keep their own checkpoint and ACK.
- `runnerd-codex-transport.ts`: `waitForRunnerCommand` also extends a
deadline while the controller keeps committing the Runner's events. A
Runner with no committed event for 30 s still times out at the original
deadline. A Runner that keeps emitting but never answers times out after
at most 5 more minutes.
- `runnerd-codex-transport.ts`: recovery of a semantic tool input that
an earlier generation dispatched and never answered now also runs when
the controller attaches, not only after startup completes.
`paperclip_finish` and `paperclip_block` still wait for startup.
- `runnerd-codex-transport.ts`: a close that failed only because the
controller journal drain timed out is no longer cached. The next close
drains again with a fresh bound and finishes as settled or as a typed
`NativeSessionCloseUnrecoverableError`.

## Verification

Red on master, green on this branch:

- `durable-prp-control-plane.test.ts`
- `answers a command issued during a 1,000-event replay within its
deadline`
- `keeps a failed command result behind the events the Runner sent
before it`
- `delivers every command pending at attach so an earlier generation's
result cannot hold the recovery probe`
  - `never delivers a lifecycle command ahead of its turn at attach`
- `acknowledges a replayed burst once per batch, never ahead of its
durable checkpoint`
- `still acknowledges a lifecycle event inside a burst before the next
frame`
- `answers a snapshot from a Runner whose provider stays wedged until
its whole replay is acknowledged` (models the Daytona capture: the Codex
Runner stops reading provider output while it holds unacknowledged
events)
- `runnerd-codex-transport-reattach-carried-commands.test.ts`
- `keeps waiting past the deadline while the controller is still
committing the Runner's replayed events`
- `times out a Runner that keeps replaying but never answers once the
replay extension is spent`
- `does not extend a deadline for a Runner whose replay has stopped
advancing`
- `runnerd-codex-transport-orphaned-tool-at-attach.test.ts`: `settles an
earlier generation's in-flight tool call at attach even when that attach
never finishes startup`
- `runnerd-detach-durability.test.ts`: new case `close_timeout` in
`joins an output flush before controller retirement permits a successor
writer`

Measured answer latency for a snapshot sent during a replay (50 ms
commit callback): 100 / 1,000 / 10,000 events took 13.95 s / more than
900 s / did not finish on master, and 26 ms / 26 ms / 170 ms with this
branch.

Local runs on this branch: `tsc --noEmit` is clean for
`packages/paperclip-runner` and `server`.
`durable-prp-control-plane.test.ts`: 227 of 227 pass.
`runnerd-codex-transport-reattach-carried-commands`, `-adoption`,
`-orphaned-tool-at-attach` and `runnerd-detach-durability`: all pass.
`runnerd-codex-transport-settlement` and `-deadlines`: the run on this
shared machine (load 50 to 96) had 8 tests fail. A rerun of those two
files alone (load 40 to 72) still had 7 failures: one timing assertion
(`expected 7419 to be less than 3500`) and six settlement cases that
ended with `native_cleanup_maintenance_unproven` after timeouts. The set
of failing cases changed between the two runs. The CI Runner support
suite ran both files on this head and passed, so I rely on CI for them.

Runner content ID: this PR changes only controller-side TypeScript under
`packages/paperclip-runner/src`. There is no Rust, `protocol/` or
`generated/` change. The Runner content ID changes because its hash
covers that source, but existing Runner images work with this
controller. The Daytona cells below ran this controller against an image
built for an earlier content ID.

Daytona results that exist (local servers with Daytona sandboxes,
release-swap cells). They ran on a combined branch with this PR and
three others (the reattach-first PR (`fix/reattach-live-runs-first`),
the adopt-live-Runner PR (`fix/adopt-live-runner-on-retry`), and #21362,
which is merged), so they cannot be credited to this PR alone:

- G5 baseline (master d1d71a7755; 480 s delay injected on the first
identity write per run): one Claude run reproduced the staging chain:
authentication 419 ms, identity +423.5 s, attempt 1 `native session
provider recovery timed out after 900000ms`, then the retry failed with
`runner_harness_state_mismatch: resumed_not_suspended`.
- G5c (same cell, combined branch 588c6ba267): 6 of 6 tasks finished
hands-off. That chain did not recur. All 6 runs that crossed swap 2 were
adopted on the same Runner process, with no retry and no 900 s timeout.
- Caveat: G5c ran before the reattach-phase timing fix in the
reattach-first PR (`fix/reattach-live-runs-first`). The command fixes in
this PR were present in G5c except the test-only fix for the
wedged-provider test.
- G6c (12 live runs and 20 queued, two graceful swaps, no injection): 0
failed runs at either swap; 11 of 11 survivors at both. Swap 2 was slow
(see the reattach-first PR (`fix/reattach-live-runs-first`)).

## Risks

- Not proven: the Runner-side stall. A Codex Runner stops reading
provider output while no controller is attached (bounded 256-line
channel, provider stdout pipe full). This PR makes the controller
consume the replay quickly, so the stall clears in about 2 to 6 s once
the controller drives it. The durable fix is in the Runner (keep
draining provider output to the durable outbox while disconnected, save
once per ACK batch, run commands ahead of ACKs). That needs a new Runner
image and is not in this PR.
- Not proven: why a connected Runner with an empty outbox sometimes did
not answer for more than 30 s. The G5 capture points to the server
stalling past its own deadline (a `session.open` answered and
acknowledged 8 minutes before the server reported it timed out), not to
the Runner.
- Not done: cleanup takeover for a same-run retry when
`remoteCleanupScope` is absent, and for a predecessor run that shares
the warm sandbox. This PR only stops the replayed close failure.
- Time to reach a wedged Codex Runner still grows with its backlog:
about backlog x per-event commit / database concurrency (40 runs x 1,000
events x 20 ms / 8 is about 100 s).
- Behavior shift: a crash during a batched burst replays at most the
unacknowledged batch (up to 64 events). The Runner still holds those
events, and commit order is unchanged.
- A Runner that keeps emitting but never answers now fails after at most
5 more minutes instead of 30 s.
- The predecessor-retirement PR (`fix/retire-predecessor-session`)
builds on the close path changed here (`retry a close whose controller
journal drain timed out`).

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code. The fixes were written by Claude agents in the
same session family.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:34:31 -06:00