refactor(acpx-sidecar): extract the runtime event step and the test process helper (#21419)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs Claude Code through the ACPX sidecar.
> - Claude can start a shell command in the background and end its turn.
When the command finishes, Claude Code starts a cycle of its own to
continue. Today the Runner stops Claude at the end of the turn, so the
command stops and the run fails with `native_finalization_missing`.
> - A stack of pull requests lets the Runner accept that cycle as a new
turn of the same run, when a controller asks for it.
> - The last part of that stack adds a second kind of turn to the
sidecar. It needs two pieces of the code of a normal turn.
> - This pull request moves those two pieces into functions, and moves a
test helper to a file of its own. It changes no behavior.
> - The benefit is that the later pull request shows only new behavior,
not moved code.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one part of a change that was first opened as one
large pull request (#21316). The parts, in stack order after #21311:

1. #21419: sidecar preparation (a refactor).
2. #21420: converter for raw ACP session updates.
3. #21421: cycle stream at the adapter.
4. #21422: contract for provider-started turns.
5. #21423: `paperclip-runnerd` waits for a provider-started turn.
6. #21316: the sidecar adopts a cycle.

This is part 1. Its base branch is
`refactor/remove-direct-gateway-path`. After each part, a production run
does the same as today, because no controller sends the opt-in. Refs
#19912 (design).

**What existing behavior does this improve?**

Nothing for a user. The structure of the ACPX sidecar code.

**Subsystem affected**

Paperclip Runner: `src/cli/acpx-runtime-sidecar.ts` and its test.

**Current behavior**

- `pumpTurn` has the code that prepares each runtime event for the
Runner, and the code that sends the terminal frame of a turn, inline.
- The sidecar test file has the `SidecarProcess` helper that starts a
sidecar process.

**Proposed behavior**

- `createRuntimeEventStep` has the event code. `emitTurnTerminal` has
the terminal code. `pumpTurn` calls both.
- `src/test-support/acpx-sidecar-process.ts` has the helper, so a second
test file can use it.
- No protocol, event or timing change.

**Reason and benefit**

A later pull request adds a turn that Claude starts by itself. It uses
the same two functions, and its tests use the same helper.

## What Changed

- `src/cli/acpx-runtime-sidecar.ts`: 74 lines added and 56 removed. The
code moved with its comments, with no change.
- `src/cli/acpx-runtime-sidecar.test.ts` and
`src/test-support/acpx-sidecar-process.ts`: the helper moved, 138 lines
added and 132 removed.

## Verification

At the head of this branch, from `packages/paperclip-runner`:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run
build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs
test/protocol-contract.test.mjs`, the server check `node
scripts/verify-runner-vendor-dependencies.mjs`, `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt
--check` and `cargo build -p paperclip-runner-core --bins` (no new
warning): all passed.
- `npx vitest run src/cli/acpx-runtime-sidecar.test.ts`: 125 passed. The
tests are the same as before, apart from the import of the helper.

Not run: the full suite. CI supplies that coverage.

## Risks

- **A refactor of the turn loop of each ACPX agent.** The moved code
runs for Claude, Codex, Pi, Kimi, Cursor, Copilot and Grok. The existing
sidecar suite covers it. No new test is necessary, because no behavior
changes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Opus 5.5 authored and GitHub committed 2026-11-13 14:39:55 -08:00
1 parent e824c5cc16
commit 055a0a3c54
3 files changed
+212 -188

No files matched your search

@@ -26,6 +26,7 @@ import { classifyClaudeStatusBanner } from "../drivers/acpx/claude-status-banner
import { acpxGoalProjection } from "../drivers/acpx/session-goals.js";
import { afterEach, describe, expect, it, vi } from "vitest";
import { initializeRequest, SidecarProcess } from "../test-support/acpx-sidecar-process.js";
describe("live sidecar pending request snapshots", () => {
it("preserves explicit turn cost through the sidecar's closed usage projection", () => {
@@ -115,6 +116,12 @@ import {
const children = new Set<SidecarProcess>();
function startSidecar(): SidecarProcess {
const sidecar = new SidecarProcess();
children.add(sidecar);
return sidecar;
}
afterEach(async () => {
vi.unstubAllEnvs();
await Promise.all([...children].map((child) => child.close()));
@@ -2174,138 +2181,6 @@ describe("qualified ACPX runtime sidecar", () => {
});
});
function initializeRequest(
id: number,
agent: string,
model = "gpt-5.6-sol",
) {
return {
protocolVersion: ACPX_SIDECAR_PROTOCOL_VERSION,
id,
command: "initialize",
params: { agent, model },
};
}
function startSidecar(): SidecarProcess {
const sidecar = new SidecarProcess();
children.add(sidecar);
return sidecar;
}
class SidecarProcess {
readonly #child: ChildProcessWithoutNullStreams;
readonly #frames: Array<Record<string, unknown>> = [];
readonly #signals: Array<() => void> = [];
#stderr = "";
#closed = false;
readonly #exit: Promise<{code:number|null;signal:NodeJS.Signals|null}>;
constructor(options: {loader?:string;environment?:Record<string,string>} = {}) {
this.#child = spawn(
options.loader ? process.execPath : fileURLToPath(new URL("../../node_modules/.bin/tsx", import.meta.url)),
[...(options.loader ? ["--import","tsx","--import",options.loader] : []), fileURLToPath(new URL("./acpx-runtime-sidecar.ts", import.meta.url))],
{ stdio: ["pipe", "pipe", "pipe"], env: { PATH: process.env.PATH, LANG: "C.UTF-8", ...options.environment } },
);
this.#exit = new Promise(resolve => this.#child.once("exit", (code, signal) => resolve({code,signal})));
let stdout = "";
this.#child.stdout.setEncoding("utf8");
this.#child.stdout.on("data", (chunk: string) => {
stdout += chunk;
for (;;) {
const newline = stdout.indexOf("\n");
if (newline < 0) break;
const line = stdout.slice(0, newline);
stdout = stdout.slice(newline + 1);
if (!line.trim()) continue;
this.#frames.push(JSON.parse(line) as Record<string, unknown>);
for (const signal of this.#signals.splice(0)) signal();
}
});
this.#child.stderr.setEncoding("utf8");
this.#child.stderr.on("data", (chunk: string) => {
this.#stderr += chunk;
});
}
write(value: Record<string, unknown>): void {
this.#child.stdin.write(`${JSON.stringify(value)}\n`);
}
closeOutput(): void {
this.#child.stdout.destroy();
}
async shutdown(trigger: "stdin" | "SIGTERM" | "SIGINT") {
if (trigger === "stdin") this.#child.stdin.end();
else this.#child.kill(trigger);
return this.exited();
}
async exited() {
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([
this.#exit,
new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error("Sidecar did not exit without forced cleanup")), 5000);
}),
]);
} finally { clearTimeout(timer); }
}
stderr(): string {
return this.#stderr;
}
pending(predicate: (frame: Record<string, unknown>) => boolean): number {
return this.#frames.filter(predicate).length;
}
async next(
predicate: (frame: Record<string, unknown>) => boolean,
): Promise<Record<string, unknown>> {
const deadline = Date.now() + 5_000;
for (;;) {
const index = this.#frames.findIndex(predicate);
if (index >= 0) return this.#frames.splice(index, 1)[0]!;
const remaining = deadline - Date.now();
if (remaining <= 0) {
throw new Error(
`Timed out waiting for sidecar frame. stderr=${JSON.stringify(this.#stderr)}`,
);
}
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => {
const index = this.#signals.indexOf(signal);
if (index >= 0) this.#signals.splice(index, 1);
reject(new Error("Timed out waiting for sidecar output"));
}, remaining);
const signal = () => {
clearTimeout(timer);
resolve();
};
this.#signals.push(signal);
});
}
}
async close(): Promise<void> {
if (this.#closed || this.#child.exitCode !== null || this.#child.signalCode !== null) return;
this.#closed = true;
this.#child.stdin.end();
const exit = new Promise<void>((resolve) => {
this.#child.once("exit", () => resolve());
});
const timeout = new Promise<void>((resolve) => {
setTimeout(() => {
if (this.#child.exitCode === null) this.#child.kill("SIGKILL");
resolve();
}, 2_000).unref();
});
await Promise.race([exit, timeout]);
}
}
type ToolCallResponseBody = { result: { content: Array<{ text: string }>; isError?: boolean } };
@@ -714,6 +714,68 @@ async function dispatch(
throw new Error("unreachable ACPX sidecar command");
}
function createRuntimeEventStep(
currentTurnId: string,
toolEvidence?: AcpxToolEvidence,
): { accept(event: AcpRuntimeEvent): void; settle(): void } {
// ACP tool updates are deltas. Preserve the opening event's identity and
// display metadata for later progress/completion frames before they cross
// the sidecar boundary, matching the in-process ACPX driver path.
const normalizeToolEvent = createAcpxToolEventNormalizer<AcpRuntimeEvent>();
const piMessages = initializedAgent === "pi" ? createPiMessageProjection<AcpRuntimeEvent>() : null;
const kimiMessages = initializedAgent === "kimi" ? createKimiMessageProjection<AcpRuntimeEvent>() : null;
const normalizeMessage = kimiMessages ? kimiMessages.normalize : piMessages ? piMessages.normalize : initializedAgent === "grok"
? createGrokMessageNormalizer<AcpRuntimeEvent>() : (event: AcpRuntimeEvent) => event;
// Claude subagent lineage reads the bounded `_meta` and `rawInput` before
// they are dropped at this boundary. Only display data leaves: a
// schema-checked delegation rich event and a child tool's parent id.
const trackSubagents = initializedAgent === "claude"
? createClaudeSubagentLineageTracker<AcpRuntimeEventShape>((id) => stableProviderIdentity(id, "tool"))
: null;
const accept = (event: AcpRuntimeEvent): void => {
toolUseAdmission?.observe(currentTurnId, event);
toolEvidence?.tool(event);
// Claude hook lifecycle rides on session_info_update `_meta`; publish it
// as schema-validated display activity rather than a raw status event.
const hook = claudeHookCanonicalEvent(event);
if (hook) {
try { validateAcpxRichEvent(hook); emit("runtime.rich_event", { ...hook }, currentTurnId); } catch { /* display only */ }
return;
}
const sanitized = sanitizeRuntimeEvent(
normalizeMessage(normalizeToolEvent(withSidecarToolDetail(event, boundRuntimeEventForNormalization(event)))),
);
const lineage = trackSubagents && event.type === "tool_call" && typeof sanitized.toolCallId === "string"
? trackSubagents({
type: "tool_call",
toolCallId: sanitized.toolCallId,
tag: event.tag,
status: event.status,
rawInput: event.rawInput,
rawOutput: event.rawOutput,
_meta: boundedAcpMeta((event as { _meta?: unknown })._meta),
})
: null;
if (lineage?.paperclipParentToolCallId) sanitized.parentToolCallId = lineage.paperclipParentToolCallId;
emit("runtime.event", sanitized, currentTurnId);
if (lineage?.paperclipSubagent) {
// Rust names the tool call (and each child's parentItemId) with
// acpx_opaque_item_id; the delegation must use the same identity.
const delegationId = acpxOpaqueToolItemId(String(sanitized.toolCallId));
for (const mapped of canonicalProviderEventsFromAcpxRuntimeEvent(lineage, `${currentTurnId}:subagent`, currentTurnId)) {
const rich = withDelegationIdentity(mapped, delegationId);
try {
validateAcpxRichEvent(rich);
emit("runtime.rich_event", { ...rich }, currentTurnId);
} catch {
// Display-only lineage must never fail the provider turn.
}
}
}
};
return { accept, settle: () => { piMessages?.settle(); kimiMessages?.settle(); } };
}
/** Turns whose native plan approval was declined after capturing the proposal. */
const planProposalTurns = new Set<string>();
@@ -729,63 +791,10 @@ async function pumpTurn(
): Promise<void> {
let terminal: Record<string, unknown>;
try {
// ACP tool updates are deltas. Preserve the opening event's identity and
// display metadata for later progress/completion frames before they cross
// the sidecar boundary, matching the in-process ACPX driver path.
const normalizeToolEvent = createAcpxToolEventNormalizer<AcpRuntimeEvent>();
const piMessages = initializedAgent === "pi" ? createPiMessageProjection<AcpRuntimeEvent>() : null;
const kimiMessages = initializedAgent === "kimi" ? createKimiMessageProjection<AcpRuntimeEvent>() : null;
const normalizeMessage = kimiMessages ? kimiMessages.normalize : piMessages ? piMessages.normalize : initializedAgent === "grok"
? createGrokMessageNormalizer<AcpRuntimeEvent>() : (event: AcpRuntimeEvent) => event;
// Claude subagent lineage reads the bounded `_meta` and `rawInput` before
// they are dropped at this boundary. Only display data leaves: a
// schema-checked delegation rich event and a child tool's parent id.
const trackSubagents = initializedAgent === "claude"
? createClaudeSubagentLineageTracker<AcpRuntimeEventShape>((id) => stableProviderIdentity(id, "tool"))
: null;
for await (const event of runtimeTurn.events) {
toolUseAdmission?.observe(currentTurnId, event);
toolEvidence?.tool(event);
// Claude hook lifecycle rides on session_info_update `_meta`; publish it
// as schema-validated display activity rather than a raw status event.
const hook = claudeHookCanonicalEvent(event);
if (hook) {
try { validateAcpxRichEvent(hook); emit("runtime.rich_event", { ...hook }, currentTurnId); } catch { /* display only */ }
continue;
}
const sanitized = sanitizeRuntimeEvent(
normalizeMessage(normalizeToolEvent(withSidecarToolDetail(event, boundRuntimeEventForNormalization(event)))),
);
const lineage = trackSubagents && event.type === "tool_call" && typeof sanitized.toolCallId === "string"
? trackSubagents({
type: "tool_call",
toolCallId: sanitized.toolCallId,
tag: event.tag,
status: event.status,
rawInput: event.rawInput,
rawOutput: event.rawOutput,
_meta: boundedAcpMeta((event as { _meta?: unknown })._meta),
})
: null;
if (lineage?.paperclipParentToolCallId) sanitized.parentToolCallId = lineage.paperclipParentToolCallId;
emit("runtime.event", sanitized, currentTurnId);
if (lineage?.paperclipSubagent) {
// Rust names the tool call (and each child's parentItemId) with
// acpx_opaque_item_id; the delegation must use the same identity.
const delegationId = acpxOpaqueToolItemId(String(sanitized.toolCallId));
for (const mapped of canonicalProviderEventsFromAcpxRuntimeEvent(lineage, `${currentTurnId}:subagent`, currentTurnId)) {
const rich = withDelegationIdentity(mapped, delegationId);
try {
validateAcpxRichEvent(rich);
emit("runtime.rich_event", { ...rich }, currentTurnId);
} catch {
// Display-only lineage must never fail the provider turn.
}
}
}
}
const step = createRuntimeEventStep(currentTurnId, toolEvidence);
for await (const event of runtimeTurn.events) step.accept(event);
const result = await runtimeTurn.result;
if (result.status === "completed") { piMessages?.settle(); kimiMessages?.settle(); }
if (result.status === "completed") step.settle();
await drainExtensions();
try {
const usageAfter = await readSidecarHostStatusWithin(activeHost);
@@ -836,6 +845,15 @@ async function pumpTurn(
if (declinedPlan) {
terminal = { status: "completed", stopReason: "end_turn" };
}
emitTurnTerminal(currentTurnId, terminal, activeHost, declinedPlan);
}
function emitTurnTerminal(
currentTurnId: string,
terminal: Record<string, unknown>,
activeHost: AcpxRuntimeHost,
declinedPlan: boolean,
): void {
// A terminal frame is also runnerd's permission to recycle this provider.
// Publish it only after no callback can inherit this turn's mutable binding.
// runnerd can settle this turn offline using the same observation as session.goal.get.
@@ -0,0 +1,131 @@
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
import { fileURLToPath } from "node:url";
import { ACPX_SIDECAR_PROTOCOL_VERSION } from "../drivers/acpx/sidecar-protocol.js";
export function initializeRequest(
id: number,
agent: string,
model = "gpt-5.6-sol",
) {
return {
protocolVersion: ACPX_SIDECAR_PROTOCOL_VERSION,
id,
command: "initialize",
params: { agent, model },
};
}
export class SidecarProcess {
readonly #child: ChildProcessWithoutNullStreams;
readonly #frames: Array<Record<string, unknown>> = [];
readonly #signals: Array<() => void> = [];
#stderr = "";
#closed = false;
readonly #exit: Promise<{code:number|null;signal:NodeJS.Signals|null}>;
constructor(options: {loader?:string;environment?:Record<string,string>} = {}) {
this.#child = spawn(
options.loader ? process.execPath : fileURLToPath(new URL("../../node_modules/.bin/tsx", import.meta.url)),
[...(options.loader ? ["--import","tsx","--import",options.loader] : []), fileURLToPath(new URL("../cli/acpx-runtime-sidecar.ts", import.meta.url))],
{ stdio: ["pipe", "pipe", "pipe"], env: { PATH: process.env.PATH, LANG: "C.UTF-8", ...options.environment } },
);
this.#exit = new Promise(resolve => this.#child.once("exit", (code, signal) => resolve({code,signal})));
let stdout = "";
this.#child.stdout.setEncoding("utf8");
this.#child.stdout.on("data", (chunk: string) => {
stdout += chunk;
for (;;) {
const newline = stdout.indexOf("\n");
if (newline < 0) break;
const line = stdout.slice(0, newline);
stdout = stdout.slice(newline + 1);
if (!line.trim()) continue;
this.#frames.push(JSON.parse(line) as Record<string, unknown>);
for (const signal of this.#signals.splice(0)) signal();
}
});
this.#child.stderr.setEncoding("utf8");
this.#child.stderr.on("data", (chunk: string) => {
this.#stderr += chunk;
});
}
write(value: Record<string, unknown>): void {
this.#child.stdin.write(`${JSON.stringify(value)}\n`);
}
closeOutput(): void {
this.#child.stdout.destroy();
}
async shutdown(trigger: "stdin" | "SIGTERM" | "SIGINT") {
if (trigger === "stdin") this.#child.stdin.end();
else this.#child.kill(trigger);
return this.exited();
}
async exited() {
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([
this.#exit,
new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error("Sidecar did not exit without forced cleanup")), 5000);
}),
]);
} finally { clearTimeout(timer); }
}
stderr(): string {
return this.#stderr;
}
pending(predicate: (frame: Record<string, unknown>) => boolean): number {
return this.#frames.filter(predicate).length;
}
async next(
predicate: (frame: Record<string, unknown>) => boolean,
): Promise<Record<string, unknown>> {
const deadline = Date.now() + 5_000;
for (;;) {
const index = this.#frames.findIndex(predicate);
if (index >= 0) return this.#frames.splice(index, 1)[0]!;
const remaining = deadline - Date.now();
if (remaining <= 0) {
throw new Error(
`Timed out waiting for sidecar frame. stderr=${JSON.stringify(this.#stderr)}`,
);
}
await new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => {
const index = this.#signals.indexOf(signal);
if (index >= 0) this.#signals.splice(index, 1);
reject(new Error("Timed out waiting for sidecar output"));
}, remaining);
const signal = () => {
clearTimeout(timer);
resolve();
};
this.#signals.push(signal);
});
}
}
async close(): Promise<void> {
if (this.#closed || this.#child.exitCode !== null || this.#child.signalCode !== null) return;
this.#closed = true;
this.#child.stdin.end();
const exit = new Promise<void>((resolve) => {
this.#child.once("exit", () => resolve());
});
const timeout = new Promise<void>((resolve) => {
setTimeout(() => {
if (this.#child.exitCode === null) this.#child.kill("SIGKILL");
resolve();
}, 2_000).unref();
});
await Promise.race([exit, timeout]);
}
}