From 16816d45e60cf51750d1f4a0e7ddae3f642fb527 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Thu, 29 Oct 2026 15:13:29 -0700 Subject: [PATCH] fix(runner): keep OpenCode thread/read within runnerd's frame limit (#18218) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs OpenCode agents through runnerd and the OpenCode app-server proxy (`packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts`). > - When runnerd restores an OpenCode session, it sends `thread/resume` and then `thread/read`. It uses `thread/read` only to find an active provider turn. > - The proxy answered `thread/read` with the full retained OpenCode message history (`GET /session/{id}/message`) in one JSON-RPC line. > - runnerd rejects a provider stdout line that is larger than 4 MiB (`CODEX_APP_SERVER_MAX_FRAME_BYTES`). A long OpenCode session therefore can never resume. Each restore fails with `harness stdout frame exceeded 4194304 bytes`, and the run stops with `provider_frame_too_large`. > - This pull request makes the proxy return turn metadata only, the same as the Codex metadata-only read. > - The benefit is that long OpenCode sessions resume again, and resume no longer reads the full message history. ## Linked Issues or Issue Description **What happened?** A Paperclip Runner OpenCode task on staging tried to resume its session in a new sandbox. The restored OpenCode state was approximately 41.5 MB. The `run.attach` command failed with `failed to resume OpenCode provider: failed to start opencode provider: harness stdout frame exceeded 4194304 bytes`. The server recorded `native_session_recovery_failed` with error code `provider_frame_too_large`, and it stopped automatic replay. Each new attempt has the same failure, because the message history only grows. **Steps to reproduce** 1. Run an OpenCode task until its session holds more than 4 MiB of message parts. 2. Let the run continue in a new runner process, so that runnerd must send `thread/resume` and then `thread/read`. 3. The proxy writes the full message history in the `thread/read` response. runnerd rejects the frame and the resume fails. **Expected behavior** The resume succeeds. The size of the session history does not change the size of the `thread/read` response. **Root cause** `thread/read` in the OpenCode proxy added `transcript: await session.read?.()`. No consumer reads this field. The Rust provider reads only `thread.turns` (`latest_active_turn_id`). The Codex path already avoids message hydration for the same check ("Never hydrate message contents just to decide whether this thread has an active turn"). ## What Changed - `opencode-app-server-proxy.ts`: `thread/read` now returns `thread.id`, `thread.cwd` and `thread.turns` only. It does not call `session.read()`. - `test/fixtures/fake-opencode-server.mjs`: the fake OpenCode server can now return a large retained message history. The test enables this with a `fake-message-history-bytes` file in `XDG_DATA_HOME`. This follows the existing file-switch pattern of the fixture. Without the file, the response is `[]`, as before. - `src/cli/opencode-proxy-thread-read.test.ts`: a new regression test. It builds the real bundled proxy entrypoint, starts a session, stops the proxy, gives the persisted session a 5 MiB history, then starts a new proxy and sends `thread/resume` and `thread/read` (the runnerd restore sequence). It asserts that all stdout frames stay below 4 MiB and that `thread/read` still reports the thread id and turns. ## Verification - Red before the fix: `npx vitest run src/cli/opencode-proxy-thread-read.test.ts` failed with `expected 5243183 to be less than 4194304`. The 5,243,183-byte frame was the `thread/read` response. - Green after the fix: the same command passes. - `npx vitest run src/cli/opencode-app-server-proxy.test.ts src/cli/opencode-proxy-thread-read.test.ts`: 6 passed. - `npx vitest run src/cli/opencode-proxy src/backends/native-instruction-measurement.test.ts src/drivers/opencode`: 158 passed, 1 failed. The failure (`opencode-server-driver.test.ts` > "uses provider structure rather than prose length to select a post-tool answer") also fails on the unchanged base commit in this environment. This change does not touch that path. - `tsc -p tsconfig.json --noEmit` for `packages/paperclip-runner`: passed. - Safety checks: - Consumers: only the runnerd Rust provider (`read_thread`) calls `thread/read` on this proxy. It reads `thread.turns`. No TypeScript or Rust code reads `transcript`. - Active-turn detection does not change. The `turns` logic is the same code. - Resume now does less work: one `GET /session/{id}/message` request less. - Not run locally: the Rust tests and a full runnerd plus real OpenCode resume. This environment has no Rust toolchain. The Rust code does not change. CI runs the Rust suites. ## Risks - Low risk. The change removes one unused response field. - If an external tool read `transcript` from the proxy's `thread/read`, it does not get it now. I found no such consumer in this repository. - Tasks that already failed with `provider_frame_too_large` continue only after this proxy reaches the runner provider pack and the task runs again. ## Model Used - Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`). Used through Claude Code with tool use (shell, file edit, GitHub connector) and standard reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (related: #16401 kept OpenCode turns alive through oversized SSE events. It does not change `thread/read`.) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (none required: no documented contract includes `transcript`) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../src/cli/opencode-app-server-proxy.ts | 4 +- .../cli/opencode-proxy-thread-read.test.ts | 110 ++++++++++++++++++ .../test/fixtures/fake-opencode-server.mjs | 12 +- 3 files changed, 123 insertions(+), 3 deletions(-) create mode 100644 packages/paperclip-runner/src/cli/opencode-proxy-thread-read.test.ts diff --git a/packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts b/packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts index 67963ec4995..b1e13e59f30 100644 --- a/packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts +++ b/packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts @@ -411,6 +411,9 @@ async function handle(message: RpcMessage): Promise { break; case "thread/read": if (!session) throw new Error("OpenCode thread is not open"); + // runnerd reads only turn metadata after resume. Hydrating the retained + // OpenCode message history here grows with the session and can exceed + // runnerd's bounded stdout frame, which permanently blocks resume. result = { thread: { id: session.ids().driverSessionId, @@ -419,7 +422,6 @@ async function handle(message: RpcMessage): Promise { ? [{ id: activeTurnId, status: "inProgress" }] : [], }, - transcript: await session.read?.(), }; break; default: diff --git a/packages/paperclip-runner/src/cli/opencode-proxy-thread-read.test.ts b/packages/paperclip-runner/src/cli/opencode-proxy-thread-read.test.ts new file mode 100644 index 00000000000..fa0a9878e26 --- /dev/null +++ b/packages/paperclip-runner/src/cli/opencode-proxy-thread-read.test.ts @@ -0,0 +1,110 @@ +import { execFileSync, spawn } from "node:child_process"; +import { closeSync, openSync } from "node:fs"; +import { chmod, mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises"; +import { randomUUID } from "node:crypto"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { createInterface } from "node:readline"; + +import { expect, it } from "vitest"; + +// runnerd rejects any provider stdout line above CODEX_APP_SERVER_MAX_FRAME_BYTES. +const RUNNERD_MAX_FRAME_BYTES = 4 * 1024 * 1024; +const MODEL = "openrouter/deepseek/deepseek-v4-flash-0731"; + +type RpcFrame = { id?: string | number; method?: string; result?: unknown; error?: unknown }; + +async function startProxy(root: string, proxy: string, executable: string, runtime: string) { + const fd = process.platform === "linux" ? openSync(executable, "r") : undefined; + const child = spawn(process.execPath, [proxy, "--paperclip-trusted-opencode-executable", fd === undefined ? executable : "/proc/self/fd/3"], { + // Keep an enclosing Paperclip run's launch bindings out of this fixture. + env: { ...Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith("PAPERCLIP_"))), + PAPERCLIP_OPENCODE_RUNTIME_DIR: runtime, OPENROUTER_API_KEY: "fixture-key", + PAPERCLIP_RUN_ID: "resume-frame-run", PAPERCLIP_NORMALIZED_SESSION_ID: "resume-frame-session" }, + stdio: fd === undefined ? ["pipe", "pipe", "pipe"] : ["pipe", "pipe", "pipe", fd], + detached: process.platform !== "win32", + cwd: root, + }); + if (fd !== undefined) closeSync(fd); + const lineBytes: number[] = []; + const frames: RpcFrame[] = []; + const input = createInterface({ input: child.stdout! }); + input.on("line", line => { + lineBytes.push(Buffer.byteLength(line)); + frames.push(JSON.parse(line) as RpcFrame); + }); + child.stderr!.resume(); + let exited = false; + const exit = new Promise(done => child.once("exit", () => { exited = true; done(); })); + const send = (message: unknown) => child.stdin!.write(`${JSON.stringify(message)}\n`); + const response = async (id: number) => { + await expect.poll(() => frames.find(frame => frame.id === id), { timeout: 10_000 }).toBeDefined(); + const frame = frames.find(entry => entry.id === id)!; + expect(frame.error).toBeUndefined(); + return frame.result as Record; + }; + const stop = async () => { + child.stdin!.end(); + await Promise.race([exit, new Promise(done => setTimeout(done, 5_000))]); + if (!exited && child.pid) process.kill(process.platform === "win32" ? child.pid : -child.pid, "SIGKILL"); + await exit; + input.close(); + }; + return { send, response, stop, lineBytes }; +} + +it("resumes a session whose OpenCode message history exceeds runnerd's frame limit", async () => { + const root = await mkdtemp(join(tmpdir(), "opencode-proxy-thread-read-")); + const launchRoot = join(root, `.paperclip-verified-executable-${randomUUID().replaceAll("-", "")}`); + await mkdir(launchRoot, { mode: 0o700 }); + await chmod(launchRoot, 0o700); + const executable = join(launchRoot, "launch"); + const fixture = resolve("test/fixtures/fake-opencode-server.mjs"); + execFileSync("cc", ["-x", "c", "-o", executable, "-"], { input: `#include \n#include \nint main(int argc, char **argv) { char **args = calloc(argc + 2, sizeof(char *)); args[0] = ${JSON.stringify(process.execPath)}; args[1] = ${JSON.stringify(fixture)}; for (int i = 1; i < argc; i++) args[i + 1] = argv[i]; execv(args[0], args); return 127; }` }); + if (process.platform === "darwin") { + execFileSync("codesign", ["--force", "--sign", "-", executable]); + execFileSync("codesign", ["--verify", "--strict", executable]); + } + await chmod(executable, 0o500); + const proxy = join(root, "proxy.cjs"); + await writeFile(proxy, execFileSync(process.execPath, ["--input-type=module", "-e", `import { bundleVerifiedProviderEntrypoints } from "./scripts/build-verified-provider-entrypoints.mjs"; const entries = await bundleVerifiedProviderEntrypoints({ write: false }); process.stdout.write(entries.find(({ entrypoint }) => entrypoint.name === "opencode-app-server-proxy").verifiedResult.outputFiles[0].contents);`], { maxBuffer: 16 * 1024 * 1024 })); + const runtime = join(root, "runtime"); + const workspace = join(root, "workspace"); + await mkdir(workspace); + try { + const first = await startProxy(root, proxy, executable, runtime); + let threadId: string; + try { + first.send({ id: 1, method: "initialize", params: {} }); + await first.response(1); + first.send({ id: 2, method: "thread/start", params: { cwd: workspace, model: MODEL } }); + threadId = ((await first.response(2)).thread as { id: string }).id; + } finally { + await first.stop(); + } + + // The persisted OpenCode session has accumulated more retained message + // content than one runnerd stdout frame may carry. + const sessions = (await readdir(runtime, { withFileTypes: true })).filter(entry => entry.isDirectory()); + expect(sessions).toHaveLength(1); + await writeFile(join(runtime, sessions[0]!.name, "data", "fake-message-history-bytes"), String(RUNNERD_MAX_FRAME_BYTES + 1024 * 1024)); + + // Mirror runnerd's restore sequence: thread/resume, then thread/read to + // discover whether a provider turn is still active. + const resumed = await startProxy(root, proxy, executable, runtime); + try { + resumed.send({ id: 1, method: "initialize", params: {} }); + await resumed.response(1); + resumed.send({ id: 2, method: "thread/resume", params: { threadId, cwd: workspace, model: MODEL } }); + expect(((await resumed.response(2)).thread as { id: string }).id).toBe(threadId); + resumed.send({ id: 3, method: "thread/read", params: { threadId, includeTurns: true } }); + const read = await resumed.response(3); + expect(read.thread).toMatchObject({ id: threadId, turns: [] }); + expect(Math.max(...resumed.lineBytes)).toBeLessThan(RUNNERD_MAX_FRAME_BYTES); + } finally { + await resumed.stop(); + } + } finally { + await rm(root, { recursive: true, force: true }); + } +}, 60_000); diff --git a/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs b/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs index b08bfd226c8..3b03d4d52e2 100644 --- a/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs +++ b/packages/paperclip-runner/test/fixtures/fake-opencode-server.mjs @@ -313,8 +313,16 @@ const server = createServer(async (request, response) => { if ( request.method === "GET" && request.url === `/session/${session.id}/message` - ) - return json(response, 200, []); + ) { + // A long-lived session can hold many megabytes of retained message parts. + const historyBytes = Number( + await readFile(join(process.env.XDG_DATA_HOME, "fake-message-history-bytes"), "utf8").catch(() => "0"), + ); + return json(response, 200, historyBytes > 0 + ? [{ info: { id: "msg-history-1", sessionID: session.id, role: "assistant" }, + parts: [{ id: "part-history-1", type: "text", text: "x".repeat(historyBytes) }] }] + : []); + } if (request.method === "GET" && request.url === "/session/status") return json(response, 200, { [session.id]: { type: "idle" } }); if (request.method === "GET" && request.url?.startsWith("/question?"))