From 1d112a504f7b4a32f3e01a1735dba3805bebff50 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Fri, 13 Nov 2026 15:14:59 -0800 Subject: [PATCH] feat(acpx): convert raw Claude session updates the way ACPX does (#21420) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude Code through the ACPX sidecar. > - Claude can start a shell command in the background and end its turn. When the command finishes, Claude Code starts a cycle of its own to continue. Today the Runner stops Claude at the end of the turn, so the command stops and the run fails with `native_finalization_missing`. > - A stack of pull requests lets the Runner accept that cycle as a new turn of the same run, when a controller asks for it. > - The Runner sees each raw ACP message before ACPX does. A cycle that Claude starts by itself is not a part of an ACPX prompt, so ACPX gives no events for it. > - To show that cycle in the same form as a normal turn, the Runner must convert the raw messages itself. ACPX 0.22 does not export its converter. > - This pull request adds a copy of that converter with a test that compares it with ACPX. Nothing uses it yet. > - The benefit is that a later pull request can show the cycle of Claude with the same events as a normal turn. ## Linked Issues or Issue Description No public issue exists. The issue is described here. This pull request is one part of a change that was first opened as one large pull request (#21316). The parts, in stack order after #21311: 1. #21419: sidecar preparation (a refactor). 2. #21420: converter for raw ACP session updates. 3. #21421: cycle stream at the adapter. 4. #21422: contract for provider-started turns. 5. #21423: `paperclip-runnerd` waits for a provider-started turn. 6. #21316: the sidecar adopts a cycle. This is part 2. Its base branch is `refactor/acpx-sidecar-runtime-event-step`. After each part, a production run does the same as today, because no controller sends the opt-in. Refs #19912 (design). **What existing behavior does this improve?** Nothing yet. It adds a module that a later part uses. **Subsystem affected** Paperclip Runner: `src/drivers/acpx`. One new direct dependency in the Runner and the server. **Current behavior** - ACPX converts `session/update` messages to runtime events inside a prompt. The function is private. - The Runner has no way to make the same events from a raw message. **Proposed behavior** - `acp-session-update-event.ts` converts a raw `session/update` to the event that ACPX 0.22 makes for the same update. - It validates the message with `@agentclientprotocol/sdk` 1.9.0. This is the version that ACPX 0.22 already installs. - No production code imports the module in this pull request. **Reason and benefit** A copy with a comparison test is a smaller risk than a patch of ACPX. The test fails when the two differ. ## What Changed - `src/drivers/acpx/acp-session-update-event.ts` (new, 367 lines). - `acp-session-update-event.test.ts` and `acpx-runtime-adapter.claude-session-update-events.test.ts` (new, 363 lines). - `packages/paperclip-runner/package.json` and `server/package.json`: `@agentclientprotocol/sdk` 1.9.0 as a direct dependency. The server needs the line because it copies the Runner build with no `node_modules`. - `pnpm-lock.yaml` is not in this pull request. The lockfile workflow owns it. ## Verification At the head of this branch, from `packages/paperclip-runner`: - `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p tsconfig.test.json`, `pnpm run typecheck:typescript`, `pnpm run build:typescript`, `node --test test/acpx-sidecar-contract.test.mjs test/protocol-contract.test.mjs`, the server check `node scripts/verify-runner-vendor-dependencies.mjs`, `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`, `cargo fmt --check` and `cargo build -p paperclip-runner-core --bins` (no new warning): all passed. - The two new test files and the sidecar suite: 130 passed. - The comparison test runs the real ACPX runtime with the pinned `claude-agent-acp` and a local fake Anthropic API, and gives the same raw updates to the copy. The events are the same. - A second test fails if the SDK version of the Runner is not the version that ACPX resolves. With 1.4.0 in `package.json` it fails. Not run: the full suite. CI supplies that coverage. ## Risks - **A new direct dependency.** It is already installed through ACPX, so no new package is downloaded. The version test keeps the two the same. - **The converter is a copy of ACPX code.** An ACPX upgrade must run the comparison test. - **The module is not used yet.** It is used in part 3. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change and the split, reviewed the diff, and ran a part of the verification again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), wrote the change and the tests and cut this slice from the full change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 --- packages/paperclip-runner/package.json | 1 + .../acpx/acp-session-update-event.test.ts | 152 ++++++++ .../drivers/acpx/acp-session-update-event.ts | 367 ++++++++++++++++++ ...apter.claude-session-update-events.test.ts | 211 ++++++++++ server/package.json | 1 + 5 files changed, 732 insertions(+) create mode 100644 packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.test.ts create mode 100644 packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.ts create mode 100644 packages/paperclip-runner/src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts diff --git a/packages/paperclip-runner/package.json b/packages/paperclip-runner/package.json index 47fd7bfed0c..c89d6b11040 100644 --- a/packages/paperclip-runner/package.json +++ b/packages/paperclip-runner/package.json @@ -179,6 +179,7 @@ "dependencies": { "@agentclientprotocol/claude-agent-acp": "0.73.0", "@agentclientprotocol/codex-acp": "1.6.2", + "@agentclientprotocol/sdk": "1.9.0", "@openai/codex": "0.160.0", "acpx": "0.22.0", "ajv": "^8.20.0", diff --git a/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.test.ts b/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.test.ts new file mode 100644 index 00000000000..aabec79a87a --- /dev/null +++ b/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.test.ts @@ -0,0 +1,152 @@ +import { existsSync, readFileSync, realpathSync } from "node:fs"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; + +import { createAcpRuntime, createAgentRegistry, createFileSessionStore, type AcpRuntimeEvent } from "acpx/runtime"; +import { afterAll, beforeAll, describe, expect, it } from "vitest"; + +import { ACP_SESSION_UPDATE_EVENT_KINDS, acpRuntimeEventFromValidatedSessionUpdate, createAcpSessionUpdateEventConverter } from "./acp-session-update-event.js"; + +const FIXTURES: Array> = [ + { sessionUpdate: "tool_call", toolCallId: "toolu_1", title: "Terminal", kind: "execute", status: "pending", rawInput: { command: "ls", args: ["-la", 3] }, _meta: { claudeCode: { toolName: "Bash" } } }, + { sessionUpdate: "tool_call", toolCallId: "toolu_2", title: " ", kind: "not-a-kind", rawInput: "plain input", locations: [{ path: "/w/a.ts", line: 3 }] }, + { sessionUpdate: "tool_call", toolCallId: "toolu_3", title: "Read", rawInput: { file_path: "/w/x" }, content: [{ type: "content", content: { type: "text", text: "body" } }] }, + { sessionUpdate: "tool_call", toolCallId: "toolu_4", rawInput: { path: "/w/p", pattern: "q" }, rawOutput: 7 }, + { sessionUpdate: "tool_call_update", toolCallId: "toolu_1", status: "completed", rawOutput: { stdout: "x".repeat(700) } }, + { sessionUpdate: "tool_call_update", toolCallId: "toolu_1", content: [{ type: "content", content: { type: "text", text: "dup" } }, { type: "content", content: { type: "text", text: "dup" } }, { type: "diff", path: "/w/d" }, { type: "terminal", terminalId: "t1" }, { type: "resource_link", name: "n", uri: "u" }, { type: "resource", resource: { uri: "file:///r" } }, { type: "audio", mimeType: "audio/wav" }, { type: "unknown" }] }, + { sessionUpdate: "tool_call_update", toolCallId: "toolu_2", rawOutput: { error: "boom" }, rawInput: { cmd: "make" } }, + { sessionUpdate: "tool_call_update", toolCallId: "toolu_3", status: "failed", _meta: { claudeCode: { nonExecutionKind: "permission-rule" } } }, + { sessionUpdate: "agent_message_chunk", content: { type: "text", text: "hello" } }, + { sessionUpdate: "agent_message_chunk", messageId: " m-1 ", content: { type: "text", text: "" }, _meta: { origin: " a ", kind: 3, source: "s", secret: "x" } }, + { sessionUpdate: "agent_message_chunk", content: { type: "text", text: "" } }, + { sessionUpdate: "agent_message_chunk", content: { type: "image", data: "", mimeType: "image/png" } }, + { sessionUpdate: "agent_message_chunk", text: "legacy text" }, + { sessionUpdate: "agent_thought_chunk", content: { type: "text", text: "thinking" } }, + { sessionUpdate: "usage_update", used: 10, size: 100, cost: { amount: 0.5, currency: "USD" } }, + { sessionUpdate: "usage_update", used: 10, _meta: { "_claude/origin": { kind: "task-notification" }, usage: { input_tokens: 6, output_tokens: 4, cache_read_input_tokens: 0, cache_creation_input_tokens: 1 } } }, + { sessionUpdate: "usage_update", cost: { currency: " " }, _meta: { usage: { inputTokens: "1" } } }, + { sessionUpdate: "available_commands_update", availableCommands: [{ name: "init", description: "Init", input: { hint: "x" } }, { name: " " }, "bad", { name: "plain" }] }, + { sessionUpdate: "available_commands_update", availableCommands: [] }, + { sessionUpdate: "current_mode_update", currentModeId: "plan" }, + { sessionUpdate: "current_mode_update" }, + { sessionUpdate: "config_option_update", id: "effort", currentValue: "high" }, + { sessionUpdate: "config_option_update", configOptionId: "model" }, + { sessionUpdate: "config_option_update", configOptions: [] }, + { sessionUpdate: "session_info_update", title: "A title" }, + { sessionUpdate: "session_info_update", summary: "summary text", _meta: { claudeCode: { hook: { phase: "started" } } } }, + { sessionUpdate: "plan", entries: [{ content: "step one", status: "in_progress", priority: "high" }, { content: "bad", status: "nope" }, { content: "two", status: "pending", priority: "urgent" }] }, + { sessionUpdate: "plan", entries: [] }, + { sessionUpdate: "plan", entries: [{ content: " ", status: "pending" }, { content: "later", status: "completed" }] }, + { sessionUpdate: "plan", entries: [{ status: "pending" }] }, + { sessionUpdate: "user_message_chunk", content: { type: "text", text: "echo" } }, + { sessionUpdate: "async_task_spawned", asyncTaskId: "b1", taskType: "shell" }, + { sessionUpdate: "async_task_state_update", asyncTaskId: "b1", state: "completed" }, + { sessionUpdate: "not_a_kind" }, +]; + +const FAKE_AGENT = ` +import { createInterface } from "node:readline"; +import { readFileSync } from "node:fs"; +const updates = JSON.parse(readFileSync(process.argv[2], "utf8")); +const send = (message) => process.stdout.write(JSON.stringify(message) + "\\n"); +createInterface({ input: process.stdin }).on("line", (line) => { + const message = JSON.parse(line); + if (message.id === undefined) return; + if (message.method === "initialize") return send({ jsonrpc: "2.0", id: message.id, result: { protocolVersion: 1, agentCapabilities: { loadSession: false }, authMethods: [] } }); + if (message.method === "session/new") return send({ jsonrpc: "2.0", id: message.id, result: { sessionId: "fake-session" } }); + if (message.method === "session/prompt") { + for (const update of updates) send({ jsonrpc: "2.0", method: "session/update", params: { sessionId: "fake-session", update } }); + return setTimeout(() => send({ jsonrpc: "2.0", id: message.id, result: { stopReason: "end_turn" } }), 50); + } + send({ jsonrpc: "2.0", id: message.id, result: {} }); +}); +`; + +describe("Paperclip's copy of the ACPX session update converter", () => { + let root = ""; + const raw: unknown[] = []; + const acpxEvents: AcpRuntimeEvent[] = []; + const converted: Array = []; + + beforeAll(async () => { + root = await mkdtemp(join(tmpdir(), "acp-session-update-event-")); + const agentPath = join(root, "fake-agent.mjs"); + const fixturePath = join(root, "updates.json"); + await writeFile(agentPath, FAKE_AGENT); + await writeFile(fixturePath, JSON.stringify(FIXTURES)); + const runtime = createAcpRuntime({ + cwd: root, + sessionStore: createFileSessionStore({ stateDir: join(root, "state") }), + agentRegistry: createAgentRegistry({ overrides: { fake: [process.execPath, agentPath, fixturePath] } }), + permissionMode: "deny-all", + onAcpMessage: (direction, message) => { + if (direction === "inbound" && (message as { method?: unknown }).method === "session/update") raw.push(structuredClone(message)); + }, + }); + try { + const handle = await runtime.ensureSession({ sessionKey: "fake", agent: "fake", mode: "persistent", cwd: root }); + const turn = runtime.startTurn({ handle, text: "emit", mode: "prompt", requestId: "fixture-turn" }); + for await (const event of turn.events) acpxEvents.push(event); + await turn.result; + } finally { + await runtime.shutdown?.(); + } + const converter = createAcpSessionUpdateEventConverter(); + try { + for (const message of raw) converted.push(await converter.convert(message)); + } finally { + converter.close(); + } + }, 30_000); + + afterAll(async () => { + await rm(root, { recursive: true, force: true }); + }); + + it("produces exactly the events ACPX yields for every fixture update, in order", () => { + expect(raw).toHaveLength(FIXTURES.length); + expect(converted.filter((event) => event !== null)).toEqual(acpxEvents); + }); + + it("covers every kind ACPX parses and returns null for every kind it ignores", () => { + const byKind = new Map(); + raw.forEach((message, index) => { + const kind = String((message as { params: { update: { sessionUpdate: string } } }).params.update.sessionUpdate); + if (converted[index] !== null) byKind.set(kind, (byKind.get(kind) ?? 0) + 1); + }); + expect([...byKind.keys()].sort()).toEqual([...ACP_SESSION_UPDATE_EVENT_KINDS].sort()); + for (const ignored of ["user_message_chunk", "async_task_spawned", "async_task_state_update", "not_a_kind"]) { + expect(byKind.has(ignored)).toBe(false); + } + }); + + it("ignores messages that are not session updates", () => { + expect(acpRuntimeEventFromValidatedSessionUpdate({ method: "session/request_permission", params: { update: {} } })).toBeNull(); + expect(acpRuntimeEventFromValidatedSessionUpdate({ method: "session/update", params: {} })).toBeNull(); + expect(acpRuntimeEventFromValidatedSessionUpdate(null)).toBeNull(); + }); +}); + +describe("the ACP SDK the converter validates with", () => { + it("is the exact package instance acpx resolves", () => { + const runner = createRequire(import.meta.url); + const acpx = createRequire(runner.resolve("acpx/runtime")); + const packageRoot = (entry: string) => { + let directory = dirname(realpathSync(entry)); + while (!existsSync(join(directory, "package.json")) || !JSON.parse(readFileSync(join(directory, "package.json"), "utf8")).name) { + directory = dirname(directory); + } + return directory; + }; + const ours = packageRoot(runner.resolve("@agentclientprotocol/sdk")); + const theirs = packageRoot(acpx.resolve("@agentclientprotocol/sdk")); + expect(ours).toBe(theirs); + const manifest = (root: string) => JSON.parse(readFileSync(join(root, "package.json"), "utf8")) as { name: string; version: string }; + expect(manifest(ours)).toMatchObject({ name: "@agentclientprotocol/sdk", version: manifest(theirs).version }); + const declared = (JSON.parse(readFileSync(new URL("../../../package.json", import.meta.url), "utf8")) as { dependencies: Record }) + .dependencies["@agentclientprotocol/sdk"]; + expect(declared).toBe(manifest(theirs).version); + }); +}); diff --git a/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.ts b/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.ts new file mode 100644 index 00000000000..25fe40f89fd --- /dev/null +++ b/packages/paperclip-runner/src/drivers/acpx/acp-session-update-event.ts @@ -0,0 +1,367 @@ +import { client, methods, type AnyMessage } from "@agentclientprotocol/sdk"; +import type { AcpRuntimeEvent } from "acpx/runtime"; + +type Payload = Record; + +export const ACP_SESSION_UPDATE_EVENT_KINDS = [ + "tool_call", + "tool_call_update", + "agent_message_chunk", + "agent_thought_chunk", + "usage_update", + "available_commands_update", + "current_mode_update", + "config_option_update", + "session_info_update", + "plan", +] as const; + +const TOOL_OUTPUT_SUMMARY_MAX_CHARS = 500; +const ORIGIN_META_KEYS = ["origin", "kind", "source"] as const; +const TOOL_KINDS = new Set(["read", "edit", "delete", "move", "search", "execute", "fetch", "think", "switch_mode", "other"]); +const USAGE_BREAKDOWN_FIELDS: ReadonlyArray = [ + ["inputTokens", ["inputTokens", "input_tokens"]], + ["outputTokens", ["outputTokens", "output_tokens"]], + ["cachedReadTokens", ["cachedReadTokens", "cacheReadInputTokens", "cache_read_input_tokens"]], + ["cachedWriteTokens", ["cachedWriteTokens", "cacheCreationInputTokens", "cache_creation_input_tokens"]], + ["thoughtTokens", ["thoughtTokens", "thought_tokens"]], + ["totalTokens", ["totalTokens", "total_tokens"]], +]; + +function isRecord(value: unknown): value is Payload { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function asTrimmedString(value: unknown): string { + return typeof value === "string" ? value.trim() : ""; +} + +function asString(value: unknown): string | undefined { + return typeof value === "string" ? value : undefined; +} + +function asOptionalString(value: unknown): string | undefined { + return asTrimmedString(value) || undefined; +} + +function asOptionalFiniteNumber(value: unknown): number | undefined { + return typeof value === "number" && Number.isFinite(value) ? value : undefined; +} + +function sanitizeOriginMeta(value: unknown): Record | undefined { + if (!isRecord(value)) return undefined; + const out: Record = {}; + for (const key of ORIGIN_META_KEYS) { + const entry = value[key]; + if (typeof entry !== "string") continue; + const trimmed = entry.trim(); + if (!trimmed) continue; + out[key] = trimmed; + } + return Object.keys(out).length > 0 ? out : undefined; +} + +function extractTextDeltaOrigin(payload: Payload): { messageId?: string; meta?: Record } { + const messageId = asOptionalString(payload.messageId); + const meta = sanitizeOriginMeta(payload._meta); + return { ...(messageId ? { messageId } : {}), ...(meta ? { meta } : {}) }; +} + +function resolveTextChunk(payload: Payload, stream: "output" | "thought", tag: string): AcpRuntimeEvent | null { + const origin = extractTextDeltaOrigin(payload); + const contentRaw = payload.content; + if (isRecord(contentRaw)) { + const contentType = asTrimmedString(contentRaw.type); + if (contentType && contentType !== "text") return null; + const text = asString(contentRaw.text); + if (typeof text === "string" && (text.length > 0 || origin.messageId)) { + return { type: "text_delta", text, stream, tag, ...origin } as AcpRuntimeEvent; + } + } + const text = asString(payload.text); + if (typeof text !== "string" || (text.length === 0 && !origin.messageId)) return null; + return { type: "text_delta", text, stream, tag, ...origin } as AcpRuntimeEvent; +} + +function readFirstString(record: Payload, keys: readonly string[]): string | undefined { + for (const key of keys) { + const value = asOptionalString(record[key]); + if (value) return value; + } + return undefined; +} + +function readFirstStringArray(record: Payload, keys: readonly string[]): string[] | undefined { + for (const key of keys) { + const value = record[key]; + if (!Array.isArray(value)) continue; + const entries = value.map((entry) => asOptionalString(entry)).filter((entry): entry is string => entry !== undefined); + if (entries.length > 0) return entries; + } + return undefined; +} + +function summarizeToolInput(rawInput: unknown): string | undefined { + if (rawInput == null) return undefined; + if (typeof rawInput === "string" || typeof rawInput === "number" || typeof rawInput === "boolean") return String(rawInput); + if (!isRecord(rawInput)) return undefined; + const command = readFirstString(rawInput, ["command", "cmd", "program"]); + const args = readFirstStringArray(rawInput, ["args", "arguments"]); + if (command) return [command, ...(args ?? [])].join(" "); + return readFirstString(rawInput, ["path", "file", "filePath", "filepath", "target", "uri", "url", "query", "pattern", "text", "search"]); +} + +function truncateToolSummary(value: string): string { + if (value.length <= TOOL_OUTPUT_SUMMARY_MAX_CHARS) return value; + return `${value.slice(0, 499)}…`; +} + +const TOOL_CONTENT_TEXT_READERS: Record string | undefined> = { + text: (record) => asString(record.text), + audio: (record) => `[audio] ${asOptionalString(record.mimeType) || "audio"}`, + resource_link: (record) => asOptionalString(record.title) || asOptionalString(record.name) || asOptionalString(record.uri), + resource: (record) => { + const resource = isRecord(record.resource) ? record.resource : undefined; + return asString(resource?.text) || asOptionalString(resource?.uri); + }, + diff: (record) => `diff ${asOptionalString(record.path) || "file"}`, + terminal: (record) => { + const terminalId = asOptionalString(record.terminalId) || asOptionalString(record.id); + return terminalId ? `[terminal] ${terminalId}` : "[terminal]"; + }, +}; + +function readToolContentText(value: unknown): string | undefined { + const record = isRecord(value) ? value : undefined; + if (!record) return undefined; + if (record.type === "content") return readToolContentText(record.content); + const type = String(record.type); + return Object.hasOwn(TOOL_CONTENT_TEXT_READERS, type) ? TOOL_CONTENT_TEXT_READERS[type]!(record) : undefined; +} + +function summarizeToolContent(content: unknown): string | undefined { + if (!Array.isArray(content)) return undefined; + const fragments = content.map((entry) => readToolContentText(entry)?.trim()).filter((entry): entry is string => Boolean(entry)); + if (fragments.length === 0) return undefined; + return truncateToolSummary([...new Set(fragments)].join("\n")); +} + +function summarizeToolOutput(rawOutput: unknown): string | undefined { + if (rawOutput == null) return undefined; + if (typeof rawOutput === "string" || typeof rawOutput === "number" || typeof rawOutput === "boolean") { + return truncateToolSummary(String(rawOutput)); + } + const record = isRecord(rawOutput) ? rawOutput : undefined; + if (!record) return undefined; + return truncateToolSummary(readFirstString(record, ["text", "message", "error", "stdout", "stderr", "content"]) ?? "") || undefined; +} + +function createToolCallEvent(payload: Payload, tag: "tool_call" | "tool_call_update"): AcpRuntimeEvent { + const title = asTrimmedString(payload.title) || "tool call"; + const status = asTrimmedString(payload.status); + const inputSummary = summarizeToolInput(payload.rawInput); + const outputSummary = summarizeToolContent(payload.content) ?? summarizeToolOutput(payload.rawOutput); + const toolCallId = asOptionalString(payload.toolCallId); + const rawKind = asOptionalString(payload.kind); + const kind = rawKind && TOOL_KINDS.has(rawKind) ? rawKind : undefined; + const summaryText = status ? `${title} (${status})` : title; + const detailSummary = tag === "tool_call_update" ? outputSummary ?? inputSummary : inputSummary ?? outputSummary; + const event: Payload = { type: "tool_call", text: detailSummary ? `${summaryText}: ${detailSummary}` : summaryText, tag, title }; + if (toolCallId) event.toolCallId = toolCallId; + if (status) event.status = status; + if (kind) event.kind = kind; + const snapshot = JSON.parse(JSON.stringify(payload)) as Payload; + if (Array.isArray(snapshot.locations)) event.locations = snapshot.locations; + if (Object.prototype.hasOwnProperty.call(snapshot, "rawInput")) event.rawInput = snapshot.rawInput; + if (Object.prototype.hasOwnProperty.call(snapshot, "rawOutput")) event.rawOutput = snapshot.rawOutput; + if (Array.isArray(snapshot.content)) event.content = snapshot.content; + if (isRecord(snapshot._meta)) event._meta = snapshot._meta; + return event as unknown as AcpRuntimeEvent; +} + +function normalizeUsageCost(value: unknown): { amount?: number; currency?: string } | undefined { + if (!isRecord(value)) return undefined; + const amount = asOptionalFiniteNumber(value.amount); + const currency = asTrimmedString(value.currency); + if (amount == null && !currency) return undefined; + return { ...(amount != null ? { amount } : {}), ...(currency ? { currency } : {}) }; +} + +function normalizeUsageBreakdown(value: unknown): Record | undefined { + if (!isRecord(value)) return undefined; + const breakdown: Record = {}; + for (const [key, aliases] of USAGE_BREAKDOWN_FIELDS) { + for (const alias of aliases) { + const found = asOptionalFiniteNumber(value[alias]); + if (found != null) { + breakdown[key] = found; + break; + } + } + } + return Object.keys(breakdown).length > 0 ? breakdown : undefined; +} + +function usageUpdateEvent(payload: Payload): AcpRuntimeEvent { + const used = asOptionalFiniteNumber(payload.used); + const size = asOptionalFiniteNumber(payload.size); + const meta = isRecord(payload._meta) ? payload._meta : undefined; + const cost = normalizeUsageCost(payload.cost); + const breakdown = normalizeUsageBreakdown(meta?.usage); + return { + type: "status", + text: used != null && size != null ? `usage updated: ${used}/${size}` : "usage updated", + tag: "usage_update", + ...(used != null ? { used } : {}), + ...(size != null ? { size } : {}), + ...(cost ? { cost } : {}), + ...(breakdown ? { breakdown } : {}), + ...(meta ? { _meta: meta } : {}), + } as AcpRuntimeEvent; +} + +function availableCommandsUpdateEvent(payload: Payload): AcpRuntimeEvent { + const raw = Array.isArray(payload.availableCommands) ? payload.availableCommands : []; + const availableCommands: Array<{ name: string; description?: string; hasInput: boolean }> = []; + for (const entry of raw) { + if (!isRecord(entry)) continue; + const name = asTrimmedString(entry.name); + if (!name) continue; + const description = asTrimmedString(entry.description); + availableCommands.push({ name, ...(description ? { description } : {}), hasInput: entry.input != null }); + } + return { + type: "status", + text: availableCommands.length > 0 ? `available commands updated (${availableCommands.length})` : "available commands updated", + tag: "available_commands_update", + availableCommands, + } as AcpRuntimeEvent; +} + +const STATUS_TEXT_RESOLVERS: Record string> = { + available_commands_update: (payload) => { + const commands = Array.isArray(payload.availableCommands) ? payload.availableCommands : []; + return commands.length > 0 ? `available commands updated (${commands.length})` : "available commands updated"; + }, + current_mode_update: (payload) => { + const mode = asTrimmedString(payload.currentModeId) || asTrimmedString(payload.modeId) || asTrimmedString(payload.mode); + return mode ? `mode updated: ${mode}` : "mode updated"; + }, + config_option_update: (payload) => { + const id = asTrimmedString(payload.id) || asTrimmedString(payload.configOptionId); + const value = asTrimmedString(payload.currentValue) || asTrimmedString(payload.value) || asTrimmedString(payload.optionValue); + if (id && value) return `config updated: ${id}=${value}`; + return id ? `config updated: ${id}` : "config updated"; + }, + session_info_update: (payload) => asTrimmedString(payload.summary) || asTrimmedString(payload.message) || "session updated", +}; + +function statusUpdateEvent(tag: string, payload: Payload): AcpRuntimeEvent | null { + const resolver = STATUS_TEXT_RESOLVERS[tag]; + const text = resolver ? resolver(payload) : null; + if (!text) return null; + return { type: "status", text, tag, ...(isRecord(payload._meta) ? { _meta: payload._meta } : {}) } as AcpRuntimeEvent; +} + +function normalizePlanEntries(value: unknown): Array<{ content: string; status: string; priority?: string }> { + if (!Array.isArray(value)) return []; + const entries: Array<{ content: string; status: string; priority?: string }> = []; + for (const entry of value) { + if (!isRecord(entry)) continue; + const content = asTrimmedString(entry.content); + const status = asTrimmedString(entry.status); + if (!content || !(status === "pending" || status === "in_progress" || status === "completed")) continue; + const priority = asTrimmedString(entry.priority); + entries.push({ content, status, ...(priority === "high" || priority === "medium" || priority === "low" ? { priority } : {}) }); + } + return entries; +} + +function planUpdateEvent(payload: Payload): AcpRuntimeEvent | null { + const raw = payload.entries; + const entries = normalizePlanEntries(raw); + if (Array.isArray(raw) && raw.length === 0) return { type: "status", text: "plan updated", tag: "plan", entries: [] } as AcpRuntimeEvent; + const firstContent = asTrimmedString((Array.isArray(raw) ? raw : []).find((entry) => isRecord(entry))?.content); + const text = (firstContent ? `plan: ${firstContent}` : null) ?? (entries.length > 0 ? `plan: ${entries[0]?.content}` : null); + if (!text) return null; + return { type: "status", text, tag: "plan", ...(entries.length > 0 ? { entries } : {}) } as AcpRuntimeEvent; +} + +const PARSERS: Record AcpRuntimeEvent | null> = { + tool_call: (payload) => createToolCallEvent(payload, "tool_call"), + tool_call_update: (payload) => createToolCallEvent(payload, "tool_call_update"), + agent_message_chunk: (payload) => resolveTextChunk(payload, "output", "agent_message_chunk"), + agent_thought_chunk: (payload) => resolveTextChunk(payload, "thought", "agent_thought_chunk"), + usage_update: usageUpdateEvent, + available_commands_update: availableCommandsUpdateEvent, + current_mode_update: (payload) => statusUpdateEvent("current_mode_update", payload), + config_option_update: (payload) => statusUpdateEvent("config_option_update", payload), + session_info_update: (payload) => statusUpdateEvent("session_info_update", payload), + plan: planUpdateEvent, +}; + +export function acpRuntimeEventFromValidatedSessionUpdate(message: unknown): AcpRuntimeEvent | null { + if (!isRecord(message) || asTrimmedString(message.method) !== "session/update") return null; + const params = message.params; + if (!isRecord(params) || !isRecord(params.update)) return null; + const payload = params.update; + const kind = asTrimmedString(payload.sessionUpdate); + return Object.hasOwn(PARSERS, kind) ? PARSERS[kind]!(payload) : null; +} + +const BARRIER_METHOD = "_paperclip/session_update_barrier"; + +export type AcpSessionUpdateEventConverter = { + convert(message: unknown): Promise; + close(): void; +}; + +export function createAcpSessionUpdateEventConverter(): AcpSessionUpdateEventConverter { + let enqueue: ((message: AnyMessage) => void) | null = null; + let closeInput: (() => void) | null = null; + const pending = new Map void>(); + let converted: AcpRuntimeEvent | null = null; + let sequence = 0; + const readable = new ReadableStream({ + start(controller) { + enqueue = (message) => controller.enqueue(message); + closeInput = () => controller.close(); + }, + }); + const writable = new WritableStream(); + client({ name: "paperclip-provider-turn" }) + .onNotification(methods.client.session.update, ({ params }) => { + converted = acpRuntimeEventFromValidatedSessionUpdate({ jsonrpc: "2.0", method: "session/update", params }); + }) + .onNotification(BARRIER_METHOD, (params: unknown) => params as { sequence: number }, ({ params }) => { + const settle = pending.get(params.sequence); + pending.delete(params.sequence); + const event = converted; + converted = null; + settle?.(event); + }) + .connect({ readable, writable }); + let closed = false; + return { + convert(message) { + const params = isRecord(message) ? message.params : undefined; + const kind = isRecord(params) && isRecord(params.update) ? params.update.sessionUpdate : undefined; + if (closed || !isRecord(message) || message.method !== "session/update" || message.id !== undefined || !isRecord(params) + || !(ACP_SESSION_UPDATE_EVENT_KINDS as readonly unknown[]).includes(kind)) { + return Promise.resolve(null); + } + const id = ++sequence; + const result = new Promise((resolve) => pending.set(id, resolve)); + enqueue!({ jsonrpc: "2.0", method: "session/update", params: structuredClone(params) } as AnyMessage); + enqueue!({ jsonrpc: "2.0", method: BARRIER_METHOD, params: { sequence: id } } as AnyMessage); + return result; + }, + close() { + if (closed) return; + closed = true; + for (const settle of pending.values()) settle(null); + pending.clear(); + closeInput?.(); + }, + }; +} diff --git a/packages/paperclip-runner/src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts b/packages/paperclip-runner/src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts new file mode 100644 index 00000000000..8738a45e7d1 --- /dev/null +++ b/packages/paperclip-runner/src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts @@ -0,0 +1,211 @@ +import { spawn, type ChildProcess } from "node:child_process"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { createServer } from "node:http"; +import { createRequire } from "node:module"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import type { AcpRuntimeEvent } from "acpx/runtime"; +import { expect, it, vi } from "vitest"; + +import { ACP_SESSION_UPDATE_EVENT_KINDS, createAcpSessionUpdateEventConverter } from "./acp-session-update-event.js"; +import { openCodexAcpxRuntime } from "./acpx-runtime-adapter.js"; +import type { VerifiedAcpxCommandLease } from "./installation-integrity.js"; +import { resolveQualifiedAcpxProfile } from "./qualified-profiles.js"; +import type { AcpxRuntimePortOpenOptions } from "./runtime-host.js"; + +type Block = { type: string; text?: string; tool_use_id?: string; content?: unknown }; +type Message = { role: string; content: string | Block[] }; +type Raw = { message: Record; afterPromptResponse: boolean }; + +const textOf = (content: string | Block[] | undefined): string => typeof content === "string" ? content + : (content ?? []).map(block => block.text ?? "").join("\n"); + +it("converts the real Claude ACP stream exactly as ACPX does for every update kind it emits", async () => { + const root = await mkdtemp(join(tmpdir(), "claude-session-update-events-")); + const home = join(root, "home"), cwd = join(root, "workspace"), config = join(home, ".claude"); + await Promise.all([mkdir(config, { recursive: true }), mkdir(cwd)]); + await writeFile(join(cwd, "README.md"), "fixture\n"); + let sequence = 0; + const api = createServer(async (request, response) => { + let raw = ""; + for await (const chunk of request) raw += chunk; + if (request.method === "GET") { + response.writeHead(200, { "content-type": "application/json" }); + response.end(JSON.stringify({ data: [], has_more: false })); + return; + } + if (request.url?.includes("count_tokens")) { + response.writeHead(200, { "content-type": "application/json" }); + response.end(JSON.stringify({ input_tokens: 1 })); + return; + } + const body = JSON.parse(raw || "{}") as { model?: string; stream?: boolean; messages?: Message[]; tools?: Array<{ name: string }> }; + const model = body.model ?? "claude-sonnet-5"; + if (body.stream !== true) { + response.writeHead(200, { "content-type": "application/json" }); + response.end(JSON.stringify({ id: "msg_fixture", type: "message", role: "assistant", model, + content: [{ type: "text", text: "OK" }], stop_reason: "end_turn", usage: { input_tokens: 1, output_tokens: 1 } })); + return; + } + const conversation = (body.messages ?? []).filter(message => message.role === "user" || message.role === "assistant"); + const last = conversation.at(-1)?.content; + const results = Array.isArray(last) ? last.filter(block => block.type === "tool_result") : []; + const names = new Set((body.tools ?? []).map(tool => tool.name)); + const frames: Array> = [{ type: "message_start", message: { id: `msg_${++sequence}`, type: "message", + role: "assistant", model, content: [], stop_reason: null, stop_sequence: null, usage: { input_tokens: 3, output_tokens: 0 } } }]; + let index = 0; + const thinking = (value: string) => frames.push( + { type: "content_block_start", index, content_block: { type: "thinking", thinking: "", signature: "" } }, + { type: "content_block_delta", index, delta: { type: "thinking_delta", thinking: value } }, + { type: "content_block_delta", index, delta: { type: "signature_delta", signature: "fixture-signature" } }, + { type: "content_block_stop", index: index++ }); + const text = (value: string) => frames.push( + { type: "content_block_start", index, content_block: { type: "text", text: "" } }, + { type: "content_block_delta", index, delta: { type: "text_delta", text: value } }, + { type: "content_block_stop", index: index++ }); + const tool = (name: string, input: Record) => frames.push( + { type: "content_block_start", index, content_block: { type: "tool_use", id: `toolu_01Kinds${sequence}${index}`, name, input: {} } }, + { type: "content_block_delta", index, delta: { type: "input_json_delta", partial_json: JSON.stringify(input) } }, + { type: "content_block_stop", index: index++ }); + const prompt = textOf(last); + if (results.length > 0) { + text("Step complete."); + } else if (/KINDS-MCP/.test(prompt) && names.has("mcp__paperclip__probe")) { + thinking("I should call the probe."); + text("Calling the probe."); + tool("mcp__paperclip__probe", { tag: "kinds" }); + } else if (/KINDS-FOREGROUND/.test(prompt)) { + tool("Bash", { command: "echo foreground-kinds", description: "foreground echo" }); + } else if (/KINDS-BACKGROUND/.test(prompt)) { + tool("Bash", { command: "sleep 1; echo background-kinds", description: "background sleep", run_in_background: true }); + } else if (/KINDS-PLAN/.test(prompt) && names.has("TodoWrite")) { + tool("TodoWrite", { todos: [{ content: "First step", status: "in_progress", activeForm: "Doing the first step" }, { content: "Second step", status: "pending", activeForm: "Doing the second step" }] }); + } else { + text("Nothing to run."); + } + const usesTool = frames.some(frame => (frame as { content_block?: { type?: string } }).content_block?.type === "tool_use"); + frames.push({ type: "message_delta", delta: { stop_reason: usesTool ? "tool_use" : "end_turn", stop_sequence: null }, usage: { output_tokens: 2 } }, + { type: "message_stop" }); + response.writeHead(200, { "content-type": "text/event-stream" }); + for (const frame of frames) response.write(`event: ${frame.type}\ndata: ${JSON.stringify(frame)}\n\n`); + response.end(); + }); + const mcp = createServer(async (request, response) => { + let raw = ""; + for await (const chunk of request) raw += chunk; + if (request.method !== "POST") { response.writeHead(request.method === "DELETE" ? 200 : 405); response.end(); return; } + const message = JSON.parse(raw || "{}") as { id?: number; method?: string; params?: { protocolVersion?: string } }; + const reply = (result: unknown) => { + response.writeHead(200, { "content-type": "application/json", "mcp-session-id": "kinds" }); + response.end(JSON.stringify({ jsonrpc: "2.0", id: message.id, result })); + }; + if (message.method === "initialize") return reply({ protocolVersion: message.params?.protocolVersion ?? "2025-03-26", capabilities: { tools: { listChanged: false } }, serverInfo: { name: "kinds", version: "1" } }); + if (message.id === undefined) { response.writeHead(202); response.end(); return; } + if (message.method === "tools/list") return reply({ tools: [{ name: "probe", description: "Fixture probe.", inputSchema: { type: "object", properties: { tag: { type: "string" } } } }] }); + if (message.method === "tools/call") return reply({ content: [{ type: "text", text: "probe output" }] }); + reply({}); + }); + await new Promise(resolve => api.listen(0, "127.0.0.1", resolve)); + await new Promise(resolve => mcp.listen(0, "127.0.0.1", resolve)); + const apiPort = (api.address() as { port: number }).port; + const mcpPort = (mcp.address() as { port: number }).port; + let capture: Raw[] | null = null; + let promptAnswered = false; + const entry = createRequire(import.meta.url).resolve("@agentclientprotocol/claude-agent-acp/dist/index.js"); + const command: VerifiedAcpxCommandLease = { + spawn: (args, input) => { + const child = spawn(process.execPath, [entry, ...(args ?? [])], { ...input, detached: true, stdio: ["pipe", "pipe", "pipe"] }) as ChildProcess; + let buffer = ""; + child.stdout!.on("data", (chunk: Buffer) => { + buffer += chunk.toString(); + for (let newline = buffer.indexOf("\n"); newline >= 0; newline = buffer.indexOf("\n")) { + const line = buffer.slice(0, newline); + buffer = buffer.slice(newline + 1); + let message: Record; + try { message = JSON.parse(line) as Record; } catch { continue; } + if (message.method === undefined && (message.result as { stopReason?: unknown } | undefined)?.stopReason !== undefined) promptAnswered = true; + if (message.method === "session/update") capture?.push({ message, afterPromptResponse: promptAnswered }); + } + }); + return child; + }, + close: async () => undefined, + }; + const options = { + command, + profile: resolveQualifiedAcpxProfile("claude", "claude-sonnet-5"), + cwd, + stateDirectory: join(root, "state"), + providerSessionKey: "claude-session-update-events", + permissionMode: "approve-all", + permissionPolicy: { autoApprove: ["read", "write", "execute", "other"], defaultAction: "approve" }, + launchEnvironment: { HOME: home, CLAUDE_CONFIG_DIR: config, PATH: "/usr/bin:/bin", TMPDIR: root, + ANTHROPIC_API_KEY: "local-fixture-only", ANTHROPIC_BASE_URL: `http://127.0.0.1:${apiPort}`, + CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1", DISABLE_TELEMETRY: "1", DISABLE_ERROR_REPORTING: "1", ENABLE_TOOL_SEARCH: "false" }, + credentialFenceFds: [42, 43] as const, + activateCredentialFenceOwner: async () => undefined, + systemInstructions: "Follow the fixture.", + mcpServers: [{ name: "paperclip", url: `http://127.0.0.1:${mcpPort}/mcp`, bearerToken: "fixture", runnerOwned: false }], + retainFailedAdmissionCleanup: vi.fn(), + } as unknown as AcpxRuntimePortOpenOptions; + const converter = createAcpSessionUpdateEventConverter(); + const seen = new Map(); + let port: Awaited> | undefined; + const runTurn = async (turnId: string, text: string) => { + capture = []; + promptAnswered = false; + const yielded: AcpRuntimeEvent[] = []; + const turn = port!.startTurn({ text, requestId: turnId, onPermissionRequest: async () => ({ outcome: "allow_once" }) }); + for await (const event of turn.events) yielded.push(event); + expect(await turn.result).toMatchObject({ status: "completed" }); + const raw = capture as Raw[]; + capture = null; + const converted: Array<{ event: AcpRuntimeEvent | null; raw: Raw }> = []; + for (const message of raw) converted.push({ event: await converter.convert(message.message), raw: message }); + const produced = converted.filter(entry => entry.event !== null); + expect(produced.map(entry => entry.event).slice(0, yielded.length)).toEqual(yielded); + expect(produced.slice(yielded.length).every(entry => entry.raw.afterPromptResponse)).toBe(true); + for (const [index, entry] of converted.entries()) { + const kind = String((entry.raw.message.params as { update: { sessionUpdate: unknown } }).update.sessionUpdate); + const counts = seen.get(kind) ?? { acpx: 0, paperclip: 0 }; + const producedIndex = produced.indexOf(entry); + if (entry.event !== null) counts.paperclip += 1; + if (producedIndex >= 0 && producedIndex < yielded.length) counts.acpx += 1; + seen.set(kind, counts); + void index; + } + return yielded; + }; + try { + port = await openCodexAcpxRuntime(options, { + awaitProviderOwnership: async () => undefined, + awaitProviderExit: child => new Promise(resolve => child.once("exit", () => resolve())), + }); + await runTurn("turn-mcp", "KINDS-MCP: think, say something and call the probe."); + await runTurn("turn-foreground", "KINDS-FOREGROUND: run a foreground command."); + await runTurn("turn-background", "KINDS-BACKGROUND: run a background command."); + await runTurn("turn-plan", "KINDS-PLAN: write a plan."); + for (const required of ["tool_call", "tool_call_update", "agent_message_chunk", "agent_thought_chunk", "usage_update", "available_commands_update"]) { + expect(seen.get(required)?.acpx ?? 0, required).toBeGreaterThan(0); + } + for (const [kind, counts] of seen) { + if ((ACP_SESSION_UPDATE_EVENT_KINDS as readonly string[]).includes(kind)) { + expect(counts.paperclip, kind).toBeGreaterThan(0); + } else { + expect(counts, kind).toEqual({ acpx: 0, paperclip: 0 }); + } + } + if (process.env.PAPERCLIP_PRINT_SESSION_UPDATE_KINDS) console.log(JSON.stringify(Object.fromEntries(seen))); + } finally { + converter.close(); + try { await port?.close({ reason: "session update conversion verified" }); } + finally { + api.closeAllConnections(); + mcp.closeAllConnections(); + await new Promise(resolve => api.close(() => resolve())); + await new Promise(resolve => mcp.close(() => resolve())); + await rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }); + } + } +}, 90_000); diff --git a/server/package.json b/server/package.json index ee5e7622502..73a88325b8b 100644 --- a/server/package.json +++ b/server/package.json @@ -50,6 +50,7 @@ "@1password/sdk": "0.5.0", "@agentclientprotocol/claude-agent-acp": "0.73.0", "@agentclientprotocol/codex-acp": "1.6.2", + "@agentclientprotocol/sdk": "1.9.0", "@ai-sdk/amazon-bedrock": "5.0.113", "@ai-sdk/anthropic": "4.0.78", "@ai-sdk/google": "4.0.93",