feat(runner): let a caller ask the transport for provider-started turns (#21607)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - Today the run ends when the turn ends. The command stops, and the
run fails with `native_finalization_missing`.
> - The Runner can already keep Claude while a background shell is
running and give the cycle of Claude to the controller as a turn that
the provider started (#21423, #21316). Core can already wait for that
turn (#21453, #21454, #21455). No run does this yet, because no
controller asks the Runner for it.
> - The request goes through the transport of the Runner package. This
pull request gives the transport an option for it and the behavior that
the option needs.
> - No caller sets the option in this pull request. It changes no
production run.
> - The benefit is a small base for the two pull requests after it: the
recovery after a controller restart, and the switch that turns the
behavior on.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

#21456 turned the behavior on and had grown too large to review. It is
now three stacked pull requests:

1. **This pull request:** the transport can ask the Runner for
provider-started turns. No caller asks.
2. #21612: the recovery after a controller restart during a wait or
during a provider-started turn.
3. #21456: Core sets the option for native ACPX Claude runs. Only that
part changes what a production run does.

Refs #19912 (design), #21456.

**What existing behavior does this improve?**

None in production. It adds the support that parts 2 and 3 use.

**Subsystem affected**

Paperclip Runner package, the controller side:
`src/live/runnerd-codex-transport.ts`. ACPX Claude only, and only for a
caller that sets the new option.

**Current behavior**

- The transport has no way to ask the Runner for provider-started turns.

**Proposed behavior**

- The transport options have a new field, `providerStartedTurns`. The
transport uses it only for ACPX with the agent `claude`. In each other
case the field has no effect.
- With the option, the transport sends `providerStartedTurns: true` in
`turn.start`. It does not send it for a compaction turn.
- With the option, a tool call in a provider-started turn waits until
the transport gave each earlier event to the driver. Without this, a
`paperclip_finish` of that turn could reach the driver before the
`turn.started` of that turn, and the driver would refuse it.
- With the option, the transport gives a `run.terminal` of the Runner to
the driver as a marked internal notification (`paperclip/runTerminal`).
The driver uses it only during a wait, so a wait ends at once when the
Runner says that the run is over.
- With no option, the transport does the same as before this pull
request. Live sessions, evals, the CLI runner and the probes do not set
the option.

**Reason and benefit**

- The opt-in is a choice of the caller, not a property of the agent.
Only a consumer that can wait for a provider-started turn must ask for
one. `CapabilityLiveSession` completes a turn at the first
`turn/completed`, so it must not get the behavior because its agent is
Claude.

## What Changed

- `src/live/runnerd-codex-transport.ts` (31 lines added, 1 removed): the
option, the field in `turn.start`, the order of the tool call, and the
`paperclip/runTerminal` notification.
- `src/live/provider-turn-tool-order.ts` (new, 16 lines): finds the
source sequence of a tool call that belongs to a provider-started turn.
- `src/live/runnerd-acpx-provider-turn.test.ts` (new, 239 lines): 7
cases with the real `paperclip-runnerd`, the fake ACPX sidecar, and the
real consumer of Core.
- `src/live/live-session.test.ts` (12 lines added): a Claude live
session gives the transport no such option.
- `runner/crates/runner-core/src/bin/fake-acpx-sidecar.rs` (test binary
only): the mode `provider-result-resolved`. In this mode the
provider-started turn calls `paperclip_finish` and waits for the answer
of Core through the Runner (`tool.resolve`), as the real sidecar does.
The mode `provider-result` is not changed.

## State saved by the previous release

- No persisted state changes in this part.
- **A Runner that does not know the field** ignores it.
- **Rollback of the Runner package:** the previous transport does not
send the field, so the Runner does not wait.

## Verification

At the head of this branch (`2f2cb14b9c`), with nothing from parts 2 and
3:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript` and `pnpm run
build:typescript` in `packages/paperclip-runner`; `pnpm typecheck` and
`node scripts/verify-runner-vendor-dependencies.mjs` in `server`; `node
--test scripts/__tests__/runner-release-crate-cache.test.mjs`; `node
--test test/acpx-sidecar-contract.test.mjs` (18 passed): all passed.
- `src/live/runnerd-acpx-provider-turn.test.ts`, 3 runs, 7 of 7 passed
in each. The cases:
- background work, then the turn of Claude calls `paperclip_finish`;
Core answers; the Runner gives the answer to the sidecar with exactly
one `tool.resolve`; the turn completes; the run finalizes with one
`run.result.accepted`;
- a Claude run with no option from the caller takes the path of today:
no `providerStartedTurns` in `turn.start`, no `paperclip/runTerminal`,
no wait, and no `session.provider_turn` to the sidecar;
- the limit ends the wait, the run takes the path of today, and the
provider processes had exited before the cleanup of the test;
- the provider process is lost during the wait, and the run settles in
seconds;
  - the sidecar exits during the wait, and the run settles in seconds;
  - a turn with no background work does not wait;
- an ACPX agent that is not Claude is never opted in, also when the
caller sets the option.
- Each case asserts the result of the close of the session: clean in 6
cases, and a named failure in the case where the sidecar is killed. Each
clean case also asserts that there is no
`native_session_settlement_incomplete` diagnostic and no "conflicting
duplicate tool result" in the durable state.
- `src/live/live-session.test.ts`,
`src/live/runnerd-codex-transport.test.ts`,
`src/live/runnerd-acpx-provider-turn-wait.test.ts`,
`src/native-session-runtime.provider-turn.test.ts`: 434 passed.
- Rust, because the fake sidecar changed: `cargo fmt --check`, `cargo
build`, the `acpx_*` suites (244 passed) and the `acpx` unit tests (102
passed).
- Mutation checks, run on the full change before the split: with the
option ignored, 6 of the 7 assertions of the case with no option fail;
with the old mode of the fake, the case with the result fails at the
close of the session.

Not run: the full suite. CI supplies that coverage.

## Risks

- **No production effect.** No caller sets the option. Core sets it in
part 3.
- **The transport is shared by each provider.** The three new behaviors
are behind one condition: the option, ACPX, and the agent `claude`. With
the condition false, the code path is the path of today.
- **The tool-call ordering uses a 10 ms poll** while earlier events are
not delivered, only in a provider-started turn.
- **A controller restart during a wait or during a provider-started turn
is not handled in this part.** Part 2 handles it. Part 3 must not merge
before part 2.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Opus 5.5 authored and GitHub committed 2026-11-14 08:16:24 -08:00
1 parent 7db92ea319
commit 68d5c9c120
5 files changed
+361 -4

No files matched your search

@@ -269,7 +269,12 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
.any(|token| token == "provider-turn-after-disarm")
{
if let Some(run) = pending_provider_run.take() {
next_sequence = write_provider_turn(&mut stdout, next_sequence, &run)?;
next_sequence = write_provider_turn(
&mut stdout,
next_sequence,
&run,
&mut source_pending_completion,
)?;
}
}
if armed
@@ -299,7 +304,12 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
continue;
}
if let (true, Some(run)) = (armed, pending_provider_run.take()) {
next_sequence = write_provider_turn(&mut stdout, next_sequence, &run)?;
next_sequence = write_provider_turn(
&mut stdout,
next_sequence,
&run,
&mut source_pending_completion,
)?;
provider_rearmable = tokens.iter().any(|token| token == "provider-terminal-live");
} else {
provider_rearmable = false;
@@ -547,8 +557,19 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
next_sequence,
&source_run,
&mut pending_provider_run,
&mut source_pending_completion,
)?;
}
if command == "tool.resolve" && request["params"]["callId"] == "call-provider-finish" {
let turn = source_pending_completion
.take()
.ok_or("missing provider completion")?;
if request["params"]["turnId"] != turn {
return Err("provider completion identity mismatch".into());
}
next_sequence = finish_provider_turn(&mut stdout, next_sequence, &source_run)?;
continue;
}
if command == "tool.resolve" {
let turn = source_pending_completion
.take()
@@ -1084,6 +1105,7 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
next_sequence,
"run-1",
&mut pending_provider_run,
&mut None,
)?;
}
if command == "turn.start" && mode == "turns-multiple-tool-results-terminal" {
@@ -1574,13 +1596,14 @@ fn offer_provider_turn(
sequence: u64,
default_run: &str,
pending: &mut Option<String>,
pending_finish: &mut Option<String>,
) -> io::Result<u64> {
let tokens = provider_turn_tokens();
if !tokens.iter().any(|token| token == "provider-turn") {
return Ok(sequence);
}
if tokens.iter().any(|token| token == "provider-turn-unarmed") {
return write_provider_turn(output, sequence, default_run);
return write_provider_turn(output, sequence, default_run, pending_finish);
}
*pending = Some(default_run.to_owned());
Ok(sequence)
@@ -1590,6 +1613,7 @@ fn write_provider_turn(
output: &mut impl Write,
mut sequence: u64,
default_run: &str,
pending_finish: &mut Option<String>,
) -> io::Result<u64> {
let tokens = provider_turn_tokens();
if !tokens.iter().any(|token| token == "provider-turn") {
@@ -1650,6 +1674,42 @@ fn write_provider_turn(
)?;
sequence += 2;
}
if tokens
.iter()
.any(|token| token == "provider-result-resolved")
{
write_json(
output,
&frame(
sequence,
"runtime.tool_called",
json!({"callId":"call-provider-finish","operationId":"paperclip_finish","input":valid_reserved_completion_result()}),
),
)?;
*pending_finish = Some(turn_id);
return Ok(sequence + 1);
}
finish_provider_turn(output, sequence, default_run)
}
fn finish_provider_turn(
output: &mut impl Write,
mut sequence: u64,
default_run: &str,
) -> io::Result<u64> {
let tokens = provider_turn_tokens();
let value = |prefix: &str, fallback: &str| {
tokens
.iter()
.find_map(|token| token.strip_prefix(prefix).map(str::to_owned))
.unwrap_or_else(|| fallback.to_owned())
};
let turn_id = value("provider-turn-id=", "provider-1");
let run_id = value("provider-run=", default_run);
let frame = |sequence: u64, event_type: &str, payload: Value| {
json!({"protocolVersion":GENERATED_ACPX_SIDECAR_PROTOCOL_VERSION,"sequence":sequence,
"eventType":event_type,"runId":run_id,"turnId":turn_id,"payload":payload})
};
write_json(
output,
&frame(
@@ -53,6 +53,18 @@ it("admits Pi live sessions without candidate opt-in while preserving the exact
} finally { await session.shutdown("test complete"); }
});
it("leaves provider-started turns off for a Claude ACPX live session", async () => {
const factory = fakeTransportFactory(providerState());
const options: Parameters<CapabilityLiveTransportFactory>[0][] = [];
const service = new CapabilityLiveSessionService({ transportFactory: next => { options.push(next); return factory(next); } });
const session = await service.create({ provider: "acpx", acpxAgent: "claude", requestedModel: "claude-sonnet-4-6" });
try {
await session.sendMessage("hello");
expect(options[0]).toMatchObject({ provider: "acpx", acpxAgent: "claude" });
for (const option of options) expect(option).not.toHaveProperty("providerStartedTurns");
} finally { await session.shutdown("test complete"); }
});
class AsyncNotifications implements AsyncIterable<CodexRpcNotification> {
#values: CodexRpcNotification[] = [];
#waiters: Array<(value: IteratorResult<CodexRpcNotification>) => void> = [];
@@ -0,0 +1,16 @@
import type { DurableRecoveryCommittedEvent } from "../control-plane/prp-transport-types.js";
const record = (value: unknown): Record<string, unknown> =>
value !== null && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};
export function providerTurnToolCallSourceSeq(
events: readonly Pick<DurableRecoveryCommittedEvent, "sourceSeq" | "sourceEventId" | "eventType" | "envelope">[],
sourceEventId: string,
): number | null {
const index = events.findIndex(event => event.sourceEventId === sourceEventId);
for (let at = index - 1; at >= 0; at -= 1) {
if (events[at]!.eventType !== "turn.started") continue;
return record(record(events[at]!.envelope.payload).payload).origin === "provider" ? events[index]!.sourceSeq : null;
}
return null;
}
@@ -0,0 +1,239 @@
import { createHash } from "node:crypto";
import { execFileSync, spawn } from "node:child_process";
import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises";
import { closeSync, openSync } from "node:fs";
import { createServer } from "node:http";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { expect, it, vi } from "vitest";
import type { DurablePrpControlPlane } from "../control-plane/durable-prp-control-plane.js";
import { QUALIFIED_ACPX_PROFILES, resolveQualifiedAcpxProfile } from "../drivers/acpx/qualified-profiles.js";
import { createCapabilityRunnerdCodexTransport, readRunnerdArtifactBinding } from "./runnerd-codex-transport.js";
import { createRunnerProviderLaunch, runnerLaunchRequirementsDigest } from "./runner-provider-launch.js";
import { defaultCapabilityRunnerdBinary } from "../../test/support/runnerd-transport.js";
import type { PrpEvent } from "../protocol/replay-contract.js";
import { createNativeSessionBackend } from "../backends/native-backend-factory.js";
import type { NativeExecutionInput } from "../contracts/native-execution.js";
import { executeNativeSession } from "../native-session-runtime.js";
const sha = (value: string | Buffer) => `sha256:${createHash("sha256").update(value).digest("hex")}`;
const quote = (value: string) => `'${value.replaceAll("'", "'\"'\"'")}'`;
const kill = (pid: number) => { try { process.kill(-pid, "SIGKILL"); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error; } };
const alive = (pid: number) => {
try { process.kill(pid, 0); } catch (error) { if ((error as NodeJS.ErrnoException).code === "ESRCH") return false; throw error; }
try { return !execFileSync("ps", ["-p", String(pid), "-o", "stat="], { encoding: "utf8" }).trim().startsWith("Z"); }
catch (error) { if ((error as { status?: number }).status === 1) return false; throw error; }
};
function qualifiedProfile(agent: "claude" | "codex") {
const qualified = resolveQualifiedAcpxProfile(agent, "explicit-source-model");
return { driverKind: qualified.driverKind, protocolVersion: qualified.protocolVersion, acpxVersion: qualified.acpxVersion,
agent: qualified.agent, agentProfileVersion: qualified.agentProfileVersion,
agentServerPackage: qualified.agentServerPackage, agentServerVersion: qualified.agentServerVersion,
agentRuntimePackage: qualified.agentRuntimePackage, agentRuntimeVersion: qualified.agentRuntimeVersion,
commandDigest: qualified.commandDigest };
}
let scenario = 0;
const durableState = async (root: string) => (await Promise.all(["control-plane/control-plane-state.json", "runner/runner-state.json"]
.map(path => readFile(join(root, path), "utf8").catch(() => "")))).join("\n");
async function runProviderTurnScenario(tokens: string, options: { agent?: "claude" | "codex"; capMs?: number; providerStartedTurns?: boolean } = {}) {
const agent = options.agent ?? "claude";
const providerStartedTurns = options.providerStartedTurns ?? agent === "claude";
const turnStarts: Array<Record<string, unknown>> = [];
const notificationMethods: string[] = [];
const diagnostics: string[] = [];
const n = ++scenario;
const root = await realpath(await mkdtemp(join(tmpdir(), "runner-acpx-provider-turn-")));
const journal = join(root, "calls"), fault = join(root, "fault"), wrapper = join(root, "sidecar"), script = join(root, "sidecar-entry");
const background = join(root, "background");
const binary = defaultCapabilityRunnerdBinary(), sidecar = resolve(import.meta.dirname, "../../runner/target/debug/fake-acpx-sidecar");
await mkdir(join(root, "acpx"), { mode: 0o700 }); await mkdir(join(root, "diagnostics"), { mode: 0o700 });
await writeFile(script, "qualified fixture entry");
await writeFile(background, tokens);
await writeFile(wrapper, `#!/bin/sh\nexec ${quote(sidecar)} --mode source-reconcile --profile-digest ${quote(QUALIFIED_ACPX_PROFILES[agent].commandDigest)} --journal ${quote(journal)} --source-fault ${quote(fault)} --background-tasks ${quote(background)} --spawn-owned-child\n`, { mode: 0o700 });
let authority: DurablePrpControlPlane | null = null;
const server = createServer();
server.on("upgrade", (request, socket, head) => authority ? authority.handleUpgrade(request, socket, "/runner", head) : socket.destroy());
await new Promise<void>(done => server.listen(0, "127.0.0.1", done));
const address = server.address(); if (!address || typeof address === "string") throw new Error("missing fixture listener");
const identity = { runnerInstanceId: `runner-provider-turn-${n}`, environmentLeaseId: `lease-provider-turn-${n}`, runId: `run-provider-turn-${n}`,
normalizedSessionId: `session-provider-turn-${n}`, turnId: `turn-provider-turn-${n}`, itemId: `item-provider-turn-${n}` };
const nativeIdentity = { companyId: `company-${n}`, agentId: "agent", issueId: "issue", runId: identity.runId, sessionId: identity.normalizedSessionId };
const execution: NativeExecutionInput = { schema: "paperclip.native-execution-input.v1",
binding: { companyId: nativeIdentity.companyId, agentId: nativeIdentity.agentId, issueId: nativeIdentity.issueId, runId: identity.runId, executionWorkspaceId: "workspace" },
task: { identifier: "TASK-1", title: "Install and test", description: null, prompt: "Install and test", workMode: "standard" },
workspace: { cwd: root, repoUrl: null, repoRef: null, branchName: null },
session: { normalizedSessionId: identity.normalizedSessionId, driverKind: "acpx_runtime", protocolVersion: 1, lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null } },
provider: { kind: "acpx", agent, model: "explicit-source-model", permissionPolicy: "interactive", profile: qualifiedProfile(agent) },
completionContract: { id: "contract", sha256: sha("contract"), schemaVersion: "paperclip.completion-contract.v1",
contract: { revision: "acpx-provider-turns-v1", objective: "Install and test", criteria: [{ id: "objective", requirement: "Install and test" }] } },
interactionResponses: [], credentialBindings: [] };
const runnerPids = new Set<number>();
const bundle = createCapabilityRunnerdCodexTransport({
provider: "acpx", acpxAgent: agent, ...(providerStartedTurns ? { providerStartedTurns: true } : {}),
onDiagnostic: message => diagnostics.push(message), runnerBinary: binary, stateDirectory: root, prpIdentity: identity,
runnerArtifact: readRunnerdArtifactBinding(binary), codexSourceRecoveryVersion: 1, acpxSourceRecoveryVersion: 1,
lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null }, runnerProcessReconnectGraceMs: null, runnerReconnectGraceMs: 3_000,
controlPlaneRegistration: async next => {
authority = next;
const queueCommand = next.queueCommand.bind(next);
next.queueCommand = (type, payload, ...rest) => {
if (type === "turn.start") turnStarts.push(structuredClone(payload ?? {}));
return queueCommand(type, payload, ...rest);
};
return { connectUrl: `ws://127.0.0.1:${address.port}/runner`, release: () => { authority = null; } }; },
prepareLaunch: async launch => ({ configurationId: `initial-${launch.identity.runId}`, requirementsDigest: runnerLaunchRequirementsDigest(launch),
provider: createRunnerProviderLaunch(launch, { options: {}, root, cwd: root, codexHome: root, codexCommand: "", providerNodeCommand: wrapper, opencodeProxyPath: "", acpxSidecarPath: script }) }),
runnerProcessLauncher: spec => {
const stderr = openSync(join(root, "native.stderr"), "a", 0o600);
const child = spawn(binary, [...spec.args, "--acpx-launch-authority-digest", sha("fixture authority"),
"--acpx-sidecar-command", wrapper, "--acpx-sidecar-command-sha256", sha(execFileSync("cat", [wrapper])),
"--acpx-sidecar-script", script, "--acpx-sidecar-script-sha256", sha("qualified fixture entry")],
{ env: { PATH: process.env.PATH, HOME: root, ...spec.environment }, stdio: ["ignore", "ignore", stderr], detached: true });
closeSync(stderr);
if (child.pid) runnerPids.add(child.pid);
return { child, completion: new Promise((done, reject) => { child.once("error", reject); child.once("exit", (code, signal) => done({ code, signal, stdout: "", stderr: "" })); }) };
},
});
const notifications = bundle.transport.notifications.bind(bundle.transport);
bundle.transport.notifications = () => ({ async *[Symbol.asyncIterator]() {
for await (const notification of notifications()) { notificationMethods.push(notification.method); yield notification; }
} });
const calls = async () => (await readFile(journal, "utf8").catch(() => "")).trim().split("\n").filter(Boolean)
.flatMap(line => { try { return [JSON.parse(line)]; } catch { return []; } });
const appended: PrpEvent[] = [];
const completeRun = vi.fn(async () => undefined);
const resolveMissingResult = vi.fn(async () => null);
try {
const backend = createNativeSessionBackend(execution, { codexTransportFactory: () => bundle.transport,
environment: { PAPERCLIP_WORKSPACE_CWD: root }, dynamicTools: [] });
const started = Date.now();
const outcome = await executeNativeSession({
input: execution, backend, runnerInstanceId: identity.runnerInstanceId, controlPlaneInstanceId: "control-provider-turn",
controlPlane: {
async openRun() {}, async checkpointSession() {},
async appendEvent(value: PrpEvent) {
appended.push(structuredClone(value));
return { cursor: value.sourceSeq, highestContiguousSourceSeq: value.sourceSeq, disposition: "committed" };
},
async replayEvents() { return { events: [], highestContiguousSourceSeq: 0 }; },
completeRun,
},
timeoutMs: 30_000, resolveMissingResult,
providerTurnWait: options.capMs === undefined ? undefined : { capMs: options.capMs },
}).then(() => "finalized", (error: unknown) => String(error instanceof Error ? error.message : error));
const elapsedMs = Date.now() - started;
const closeOutcome = await bundle.transport.close().then(() => "closed", (error: unknown) => String(error instanceof Error ? error.message : error));
const durable = await durableState(root);
const journalled = await calls();
const providerPids = journalled.flatMap(entry => [entry.processStart, entry.childPid]).filter((pid): pid is number => typeof pid === "number");
await vi.waitFor(() => { for (const pid of providerPids) expect(alive(pid)).toBe(false); }, { timeout: 5_000, interval: 50 }).catch(() => undefined);
const providerProcesses = providerPids.map(pid => ({ pid, exited: !alive(pid) }));
return { outcome, appended, completeRun, resolveMissingResult, journalled, providerProcesses, root, elapsedMs, turnStarts, notificationMethods, closeOutcome, diagnostics, durable };
} catch (error) {
const diagnostics = (await readFile(join(root, "native.stderr"), "utf8").catch(() => ""))
+ (await readFile(join(root, "diagnostics", "runnerd.stderr.log"), "utf8").catch(() => ""));
throw new Error(`${String(error)}\n${diagnostics}`, { cause: error });
} finally {
await bundle.transport.close().catch(() => undefined);
for (const pid of runnerPids) kill(pid);
for (const entry of await calls()) if (entry.processStart) kill(entry.processStart);
server.closeAllConnections(); await new Promise<void>(done => server.close(() => done()));
await rm(root, { recursive: true, force: true });
}
}
const sidecarCommands = (journalled: Array<Record<string, unknown>>) => journalled.filter(entry => typeof entry.command === "string").map(entry => entry.command);
const unsettled = (messages: string[]) => messages.filter(message => message.startsWith("native_session_settlement_incomplete"));
const expectCleanClose = (run: Awaited<ReturnType<typeof runProviderTurnScenario>>) => {
expect(run.closeOutcome).toBe("closed");
expect(unsettled(run.diagnostics)).toEqual([]);
expect(run.durable).not.toContain("conflicting duplicate");
};
it.skipIf(process.platform === "win32")("continues a run in the provider's own turn after background work and settles its result", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-result-resolved");
expect(run.outcome).toBe("finalized");
expect(run.completeRun).toHaveBeenCalledOnce();
expect(run.resolveMissingResult).not.toHaveBeenCalled();
const awaited = run.appended.find(event => event.eventType === "turn.completed" && event.payload.awaitingProviderTurn === true);
expect(awaited).toBeDefined();
const started = run.appended.filter(event => event.eventType === "turn.started");
expect(started).toHaveLength(2);
expect(started[1]).toMatchObject({ turnId: "provider-1", payload: { origin: "provider" } });
expect(run.appended.filter(event => event.payload.kind === "usage").map(event => event.turnId)).toContain("provider-1");
expect(sidecarCommands(run.journalled)).toContain("session.provider_turn");
expect(run.turnStarts).toEqual([expect.objectContaining({ providerStartedTurns: true })]);
expect(run.notificationMethods).toContain("paperclip/runTerminal");
expect(run.appended.filter(event => event.eventType === "run.result.accepted")).toHaveLength(1);
expect(run.appended.filter(event => event.eventType === "run.result.proposed").map(event => event.turnId)).toEqual(["provider-1"]);
expect(run.appended.filter(event => event.eventType === "turn.completed").map(event => event.turnId)).toContain("provider-1");
const resolved = run.journalled.filter(entry => entry.command === "tool.resolve");
expect(resolved.map(entry => [entry.params.turnId, entry.params.callId, entry.params.error ?? null])).toEqual([["provider-1", "call-provider-finish", null]]);
expectCleanClose(run);
}, 60_000);
it.skipIf(process.platform === "win32")("keeps a Claude run on today's path when the caller does not ask for provider-started turns", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-result", { providerStartedTurns: false });
expect(run.outcome).toContain("native_finalization_missing");
expect(run.turnStarts).toHaveLength(1);
expect(run.turnStarts[0]).not.toHaveProperty("providerStartedTurns");
expect(run.notificationMethods).not.toContain("paperclip/runTerminal");
expect(run.appended.some(event => event.payload.awaitingProviderTurn === true)).toBe(false);
expect(run.appended.some(event => event.payload.origin === "provider")).toBe(false);
expect(sidecarCommands(run.journalled)).not.toContain("session.provider_turn");
expectCleanClose(run);
}, 60_000);
it.skipIf(process.platform === "win32")("ends the wait at the cap and takes today's missing-result path when the provider never starts its turn", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-turn-hold", { capMs: 300 });
expect(run.outcome).toContain("native_finalization_missing");
expect(run.completeRun).not.toHaveBeenCalled();
expect(run.resolveMissingResult).toHaveBeenCalledWith(expect.objectContaining({
terminalEvent: expect.objectContaining({ eventType: "turn.completed", payload: expect.objectContaining({ awaitingProviderTurn: true }) }) }));
expect(run.appended.some(event => event.payload.status === "provider_turn_wait_ended")).toBe(true);
expect(run.appended.some(event => event.payload.origin === "provider")).toBe(false);
expect(run.providerProcesses.length).toBeGreaterThan(1);
expect(run.providerProcesses.filter(entry => !entry.exited)).toEqual([]);
expectCleanClose(run);
}, 60_000);
it.skipIf(process.platform === "win32")("ends the wait in seconds, not at the cap, when the provider process is lost while awaiting", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-turn-hold provider-lost-after-arm", { capMs: 30_000 });
expect(run.outcome).toContain("native_finalization_missing");
expect(run.completeRun).not.toHaveBeenCalled();
expect(run.appended.some(event => event.payload.awaitingProviderTurn === true)).toBe(true);
expect(run.elapsedMs).toBeLessThan(15_000);
expectCleanClose(run);
}, 60_000);
it.skipIf(process.platform === "win32")("ends the wait in seconds, not at the cap, when the sidecar exits while awaiting", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-turn-hold provider-sidecar-exit-after-arm", { capMs: 30_000 });
expect(run.completeRun).not.toHaveBeenCalled();
expect(run.appended.some(event => event.payload.awaitingProviderTurn === true)).toBe(true);
expect(run.elapsedMs).toBeLessThan(15_000);
expect(run.closeOutcome).toBe("provider_transport_failed: runner did not durably suspend before checkpoint");
expect(unsettled(run.diagnostics)).toEqual([expect.stringContaining('"proofWindowEnd":"suspend_not_completed"')]);
expect(run.durable).not.toContain("conflicting duplicate");
}, 60_000);
it.skipIf(process.platform === "win32")("leaves a Claude turn without background work exactly as before", async () => {
const run = await runProviderTurnScenario("quiet provider-turn");
expect(run.outcome).toContain("native_finalization_missing");
expect(run.appended.some(event => event.payload.awaitingProviderTurn === true)).toBe(false);
expect(run.appended.some(event => event.payload.status === "provider_turn_wait_ended")).toBe(false);
expect(sidecarCommands(run.journalled)).not.toContain("session.provider_turn");
expectCleanClose(run);
}, 60_000);
it.skipIf(process.platform === "win32")("never opts a non-Claude ACPX agent in", async () => {
const run = await runProviderTurnScenario("live provider-turn provider-result", { agent: "codex", providerStartedTurns: true });
expect(run.outcome).toContain("native_finalization_missing");
expect(run.turnStarts[0]).not.toHaveProperty("providerStartedTurns");
expect(run.notificationMethods).not.toContain("paperclip/runTerminal");
expect(run.appended.some(event => event.payload.awaitingProviderTurn === true)).toBe(false);
expect(sidecarCommands(run.journalled)).not.toContain("session.provider_turn");
expectCleanClose(run);
}, 60_000);
@@ -1,3 +1,4 @@
import { providerTurnToolCallSourceSeq } from "./provider-turn-tool-order.js";
import type { RunnerAttachmentDownloadGrant } from "../contracts/harness-driver.js";
import { readProviderFailure } from "../contracts/provider-failure.generated.js";
import { confirmAntigravityProbeNativeRetirement } from "../drivers/acpx/antigravity-probe-retirement.js";
@@ -104,7 +105,7 @@ import {
releaseMaterializedNativeRuntimeSkills,
} from "../drivers/runtime-context-materializer.js";
import { resolvePackagedRunnerBinary, resolvePackagedRunnerBinaryAsync } from "./runner-binary.js";
import { RUNNERD_CANONICAL_ITEM, RUNNERD_RUNTIME_ACTIVITY } from "../drivers/codex/codex-driver-values.js";
import { RUNNERD_CANONICAL_ITEM, RUNNERD_RUN_TERMINAL, RUNNERD_RUNTIME_ACTIVITY } from "../drivers/codex/codex-driver-values.js";
import { crashPoint } from "../crash-point.js";
import { isTerminalOperation, settleEarlierToolCalls, staleTerminalCallResult } from "./terminal-call-freshness.js";
@@ -1708,6 +1709,7 @@ export interface CapabilityRunnerdCodexTransportOptions extends RunnerExecutionD
recoveryLaunch?: { requirements: RunnerProviderLaunchRequirements };
opencodePermissionMode?: NativeOpenCodePermissionMode;
acpxAgent?: QualifiedAcpxAgent;
providerStartedTurns?: boolean;
/** Explicit evaluation-only candidate selection, never derived from persisted session input. */
acpxCandidateProfile?: "pi" | "cursor" | "copilot" | "kimi";
/** Server/operator authority; never derived from persisted agent or provider input. */
@@ -7712,6 +7714,10 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
this.#closedSignal.then(() => { throw new SemanticToolNotDispatchedError(); }),
]).catch(() => { throw new SemanticToolNotDispatchedError(); });
}
if (this.#providerStartedTurns()) {
const providerTurnCallSeq = providerTurnToolCallSourceSeq(core?.store.state.committedEvents ?? [], call.sourceEventId);
if (providerTurnCallSeq !== null) await this.#deliverNotificationsBefore(providerTurnCallSeq);
}
try {
this.#throwIfFailed();
} catch {
@@ -7778,6 +7784,26 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
return this.#bindSemanticToolOutcome(call, outcome);
}
#providerStartedTurns() {
return this.options.providerStartedTurns === true && this.options.provider === "acpx" && this.options.acpxAgent === "claude";
}
async #deliverNotificationsBefore(sourceSeq: number) {
while (this.#eventSourceSeq < sourceSeq - 1 || this.#deferredTurnStartEvents.some(event => event.sourceSeq < sourceSeq)) {
if (this.#closed || this.#failure) throw new SemanticToolNotDispatchedError();
this.#pumpEvents();
await new Promise<void>(resolveWait => setTimeout(resolveWait, 10));
}
let delivered!: () => void;
const barrier = new Promise<void>(resolveBarrier => { delivered = resolveBarrier; });
this.#queue.pushControl({ id: `provider-turn-order:${sourceSeq}`, method: "paperclip/providerTurnOrder", params: {} }, () => delivered());
await Promise.race([
barrier,
this.#failureSignal,
this.#closedSignal.then(() => { throw new SemanticToolNotDispatchedError(); }),
]).catch(() => { throw new SemanticToolNotDispatchedError(); });
}
#bindSemanticToolOutcome(call: { operationId: string; callId: string }, outcome: { result: unknown; isError?: boolean }) {
if (call.operationId === "call_api") {
const result = record(outcome.result);
@@ -7945,6 +7971,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
...((this.options.provider ?? "codex") === "codex" && params.effort !== undefined ? { effort: params.effort } : {}),
...(reasoningMode === undefined ? {} : { reasoningMode }),
...(skills.length ? { skills } : {}),
...(!compactionTurn && this.#providerStartedTurns() ? { providerStartedTurns: true } : {}),
turnId: pendingTurnId,
},
commandDeadline,
@@ -8549,6 +8576,9 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
});
continue;
}
if (event.eventType === "run.terminal" && this.#providerStartedTurns()) {
this.#queue.push({ method: "paperclip/runTerminal", params: { [RUNNERD_RUN_TERMINAL]: true, threadId: this.#threadId } });
}
const sessionUpdatePayload = record(eventPayload);
const canonicalMethod = runnerdCanonicalNotificationMethod(
event.eventType,