mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 08:28:48 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner runs OpenCode agents through runnerd and the
OpenCode app-server proxy
(`packages/paperclip-runner/src/cli/opencode-app-server-proxy.ts`).
> - When runnerd restores an OpenCode session, it sends `thread/resume`
and then `thread/read`. It uses `thread/read` only to find an active
provider turn.
> - The proxy answered `thread/read` with the full retained OpenCode
message history (`GET /session/{id}/message`) in one JSON-RPC line.
> - runnerd rejects a provider stdout line that is larger than 4 MiB
(`CODEX_APP_SERVER_MAX_FRAME_BYTES`). A long OpenCode session therefore
can never resume. Each restore fails with `harness stdout frame exceeded
4194304 bytes`, and the run stops with `provider_frame_too_large`.
> - This pull request makes the proxy return turn metadata only, the
same as the Codex metadata-only read.
> - The benefit is that long OpenCode sessions resume again, and resume
no longer reads the full message history.
## Linked Issues or Issue Description
**What happened?**
A Paperclip Runner OpenCode task on staging tried to resume its session
in a new sandbox. The restored OpenCode state was approximately 41.5 MB.
The `run.attach` command failed with `failed to resume OpenCode
provider: failed to start opencode provider: harness stdout frame
exceeded 4194304 bytes`. The server recorded
`native_session_recovery_failed` with error code
`provider_frame_too_large`, and it stopped automatic replay. Each new
attempt has the same failure, because the message history only grows.
**Steps to reproduce**
1. Run an OpenCode task until its session holds more than 4 MiB of
message parts.
2. Let the run continue in a new runner process, so that runnerd must
send `thread/resume` and then `thread/read`.
3. The proxy writes the full message history in the `thread/read`
response. runnerd rejects the frame and the resume fails.
**Expected behavior**
The resume succeeds. The size of the session history does not change the
size of the `thread/read` response.
**Root cause**
`thread/read` in the OpenCode proxy added `transcript: await
session.read?.()`. No consumer reads this field. The Rust provider reads
only `thread.turns` (`latest_active_turn_id`). The Codex path already
avoids message hydration for the same check ("Never hydrate message
contents just to decide whether this thread has an active turn").
## What Changed
- `opencode-app-server-proxy.ts`: `thread/read` now returns `thread.id`,
`thread.cwd` and `thread.turns` only. It does not call `session.read()`.
- `test/fixtures/fake-opencode-server.mjs`: the fake OpenCode server can
now return a large retained message history. The test enables this with
a `fake-message-history-bytes` file in `XDG_DATA_HOME`. This follows the
existing file-switch pattern of the fixture. Without the file, the
response is `[]`, as before.
- `src/cli/opencode-proxy-thread-read.test.ts`: a new regression test.
It builds the real bundled proxy entrypoint, starts a session, stops the
proxy, gives the persisted session a 5 MiB history, then starts a new
proxy and sends `thread/resume` and `thread/read` (the runnerd restore
sequence). It asserts that all stdout frames stay below 4 MiB and that
`thread/read` still reports the thread id and turns.
## Verification
- Red before the fix: `npx vitest run
src/cli/opencode-proxy-thread-read.test.ts` failed with `expected
5243183 to be less than 4194304`. The 5,243,183-byte frame was the
`thread/read` response.
- Green after the fix: the same command passes.
- `npx vitest run src/cli/opencode-app-server-proxy.test.ts
src/cli/opencode-proxy-thread-read.test.ts`: 6 passed.
- `npx vitest run src/cli/opencode-proxy
src/backends/native-instruction-measurement.test.ts
src/drivers/opencode`: 158 passed, 1 failed. The failure
(`opencode-server-driver.test.ts` > "uses provider structure rather than
prose length to select a post-tool answer") also fails on the unchanged
base commit in this environment. This change does not touch that path.
- `tsc -p tsconfig.json --noEmit` for `packages/paperclip-runner`:
passed.
- Safety checks:
- Consumers: only the runnerd Rust provider (`read_thread`) calls
`thread/read` on this proxy. It reads `thread.turns`. No TypeScript or
Rust code reads `transcript`.
- Active-turn detection does not change. The `turns` logic is the same
code.
- Resume now does less work: one `GET /session/{id}/message` request
less.
- Not run locally: the Rust tests and a full runnerd plus real OpenCode
resume. This environment has no Rust toolchain. The Rust code does not
change. CI runs the Rust suites.
## Risks
- Low risk. The change removes one unused response field.
- If an external tool read `transcript` from the proxy's `thread/read`,
it does not get it now. I found no such consumer in this repository.
- Tasks that already failed with `provider_frame_too_large` continue
only after this proxy reaches the runner provider pack and the task runs
again.
## Model Used
- Provider: Anthropic. Model: Claude Opus 5.5 (`claude-opus-5-5`). Used
through Claude Code with tool use (shell, file edit, GitHub connector)
and standard reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (related: #16401 kept OpenCode turns alive through oversized
SSE events. It does not change `thread/read`.)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (none
required: no documented contract includes `transcript`)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
111 lines
6.0 KiB
TypeScript
111 lines
6.0 KiB
TypeScript
import { execFileSync, spawn } from "node:child_process";
|
|
import { closeSync, openSync } from "node:fs";
|
|
import { chmod, mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
|
|
import { randomUUID } from "node:crypto";
|
|
import { tmpdir } from "node:os";
|
|
import { join, resolve } from "node:path";
|
|
import { createInterface } from "node:readline";
|
|
|
|
import { expect, it } from "vitest";
|
|
|
|
// runnerd rejects any provider stdout line above CODEX_APP_SERVER_MAX_FRAME_BYTES.
|
|
const RUNNERD_MAX_FRAME_BYTES = 4 * 1024 * 1024;
|
|
const MODEL = "openrouter/deepseek/deepseek-v4-flash-0731";
|
|
|
|
type RpcFrame = { id?: string | number; method?: string; result?: unknown; error?: unknown };
|
|
|
|
async function startProxy(root: string, proxy: string, executable: string, runtime: string) {
|
|
const fd = process.platform === "linux" ? openSync(executable, "r") : undefined;
|
|
const child = spawn(process.execPath, [proxy, "--paperclip-trusted-opencode-executable", fd === undefined ? executable : "/proc/self/fd/3"], {
|
|
// Keep an enclosing Paperclip run's launch bindings out of this fixture.
|
|
env: { ...Object.fromEntries(Object.entries(process.env).filter(([key]) => !key.startsWith("PAPERCLIP_"))),
|
|
PAPERCLIP_OPENCODE_RUNTIME_DIR: runtime, OPENROUTER_API_KEY: "fixture-key",
|
|
PAPERCLIP_RUN_ID: "resume-frame-run", PAPERCLIP_NORMALIZED_SESSION_ID: "resume-frame-session" },
|
|
stdio: fd === undefined ? ["pipe", "pipe", "pipe"] : ["pipe", "pipe", "pipe", fd],
|
|
detached: process.platform !== "win32",
|
|
cwd: root,
|
|
});
|
|
if (fd !== undefined) closeSync(fd);
|
|
const lineBytes: number[] = [];
|
|
const frames: RpcFrame[] = [];
|
|
const input = createInterface({ input: child.stdout! });
|
|
input.on("line", line => {
|
|
lineBytes.push(Buffer.byteLength(line));
|
|
frames.push(JSON.parse(line) as RpcFrame);
|
|
});
|
|
child.stderr!.resume();
|
|
let exited = false;
|
|
const exit = new Promise<void>(done => child.once("exit", () => { exited = true; done(); }));
|
|
const send = (message: unknown) => child.stdin!.write(`${JSON.stringify(message)}\n`);
|
|
const response = async (id: number) => {
|
|
await expect.poll(() => frames.find(frame => frame.id === id), { timeout: 10_000 }).toBeDefined();
|
|
const frame = frames.find(entry => entry.id === id)!;
|
|
expect(frame.error).toBeUndefined();
|
|
return frame.result as Record<string, unknown>;
|
|
};
|
|
const stop = async () => {
|
|
child.stdin!.end();
|
|
await Promise.race([exit, new Promise(done => setTimeout(done, 5_000))]);
|
|
if (!exited && child.pid) process.kill(process.platform === "win32" ? child.pid : -child.pid, "SIGKILL");
|
|
await exit;
|
|
input.close();
|
|
};
|
|
return { send, response, stop, lineBytes };
|
|
}
|
|
|
|
it("resumes a session whose OpenCode message history exceeds runnerd's frame limit", async () => {
|
|
const root = await mkdtemp(join(tmpdir(), "opencode-proxy-thread-read-"));
|
|
const launchRoot = join(root, `.paperclip-verified-executable-${randomUUID().replaceAll("-", "")}`);
|
|
await mkdir(launchRoot, { mode: 0o700 });
|
|
await chmod(launchRoot, 0o700);
|
|
const executable = join(launchRoot, "launch");
|
|
const fixture = resolve("test/fixtures/fake-opencode-server.mjs");
|
|
execFileSync("cc", ["-x", "c", "-o", executable, "-"], { input: `#include <unistd.h>\n#include <stdlib.h>\nint main(int argc, char **argv) { char **args = calloc(argc + 2, sizeof(char *)); args[0] = ${JSON.stringify(process.execPath)}; args[1] = ${JSON.stringify(fixture)}; for (int i = 1; i < argc; i++) args[i + 1] = argv[i]; execv(args[0], args); return 127; }` });
|
|
if (process.platform === "darwin") {
|
|
execFileSync("codesign", ["--force", "--sign", "-", executable]);
|
|
execFileSync("codesign", ["--verify", "--strict", executable]);
|
|
}
|
|
await chmod(executable, 0o500);
|
|
const proxy = join(root, "proxy.cjs");
|
|
await writeFile(proxy, execFileSync(process.execPath, ["--input-type=module", "-e", `import { bundleVerifiedProviderEntrypoints } from "./scripts/build-verified-provider-entrypoints.mjs"; const entries = await bundleVerifiedProviderEntrypoints({ write: false }); process.stdout.write(entries.find(({ entrypoint }) => entrypoint.name === "opencode-app-server-proxy").verifiedResult.outputFiles[0].contents);`], { maxBuffer: 16 * 1024 * 1024 }));
|
|
const runtime = join(root, "runtime");
|
|
const workspace = join(root, "workspace");
|
|
await mkdir(workspace);
|
|
try {
|
|
const first = await startProxy(root, proxy, executable, runtime);
|
|
let threadId: string;
|
|
try {
|
|
first.send({ id: 1, method: "initialize", params: {} });
|
|
await first.response(1);
|
|
first.send({ id: 2, method: "thread/start", params: { cwd: workspace, model: MODEL } });
|
|
threadId = ((await first.response(2)).thread as { id: string }).id;
|
|
} finally {
|
|
await first.stop();
|
|
}
|
|
|
|
// The persisted OpenCode session has accumulated more retained message
|
|
// content than one runnerd stdout frame may carry.
|
|
const sessions = (await readdir(runtime, { withFileTypes: true })).filter(entry => entry.isDirectory());
|
|
expect(sessions).toHaveLength(1);
|
|
await writeFile(join(runtime, sessions[0]!.name, "data", "fake-message-history-bytes"), String(RUNNERD_MAX_FRAME_BYTES + 1024 * 1024));
|
|
|
|
// Mirror runnerd's restore sequence: thread/resume, then thread/read to
|
|
// discover whether a provider turn is still active.
|
|
const resumed = await startProxy(root, proxy, executable, runtime);
|
|
try {
|
|
resumed.send({ id: 1, method: "initialize", params: {} });
|
|
await resumed.response(1);
|
|
resumed.send({ id: 2, method: "thread/resume", params: { threadId, cwd: workspace, model: MODEL } });
|
|
expect(((await resumed.response(2)).thread as { id: string }).id).toBe(threadId);
|
|
resumed.send({ id: 3, method: "thread/read", params: { threadId, includeTurns: true } });
|
|
const read = await resumed.response(3);
|
|
expect(read.thread).toMatchObject({ id: threadId, turns: [] });
|
|
expect(Math.max(...resumed.lineBytes)).toBeLessThan(RUNNERD_MAX_FRAME_BYTES);
|
|
} finally {
|
|
await resumed.stop();
|
|
}
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
}, 60_000);
|