Files
PaperClipAI/packages/adapter-utils/src/execution-target-stdin-race.test.ts
T
Nicky LeachandPaperclip ed8075b535 fix(adapter-utils): order stdin file writes in the sandbox process-session bridge (#11406)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters can run through a sandbox process-session bridge
> - The bridge writes streamed standard input to files before the remote
process reads them
> - Concurrent file writes can make a later chunk visible before an
earlier chunk
> - The remote process can then parse a tail fragment and wait forever
for the missing head
> - This pull request serializes host writes and makes an unexpected
file gap a loud error
> - The benefit is ordered input with a bounded failure path for sandbox
ACP sessions

## Linked Issues or Issue Description

No public GitHub issue exists for this change. The description below
follows `.github/ISSUE_TEMPLATE/bug_report.yml`.

**What happened?**

A sandbox ACP process-session bridge could stall after its handshake
when a run sent a large prompt. The host sent one un-awaited file write
for each standard input chunk. A small later chunk could finish before a
large earlier chunk. The remote poller then sent the tail bytes first.
The agent parser raised an error on the tail fragment, and the head
bytes stayed buffered without a newline.

**Expected behavior**

The bridge must expose standard input files in sequence. The remote
poller must report a clear error when an earlier file remains missing
beyond the retry budget.

**Steps to reproduce**

1. Start an ACP session through a sandbox process-session bridge.
2. Send a prompt that produces multiple standard input file chunks.
3. Delay finalization of an earlier chunk while a later chunk completes.
4. Observe that the remote parser can receive the later chunk first and
the session can stop without a clear error.

**Paperclip version or commit**

The change targets the current `master` branch at the submitted commit.

**Deployment mode**

The bug affects sandbox execution.

**Agent adapter(s) involved**

The failure affects the ACP process-session bridge.

**Database mode**

Not database-related.

**Additional context**

The fix keeps the existing per-file atomic write behavior. It adds
ordering at the host write boundary and a bounded ordering check in the
shared wrapper poll tail.

## What Changed

- Add a per-session promise chain for host standard input file writes.
- Keep a failed write from blocking later chain entries.
- Track the next expected sequence number in the shared wrapper poll
tail.
- Hold later files while an earlier file is missing within the existing
retry budget.
- Emit a loud error and advance after the retry budget expires.
- Add regression tests for host ordering, gap holding, and the loud
error path.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-stdin-race.test.ts` — 9
tests passed.
- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts` — 43 tests
passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — clean.
- With the source fix reverted, the 3 new tests fail and the 6 original
tests pass.
- CI must pass on the pull request before merge.

## Risks

Low risk.

- The host now serializes writes for each session, which can reduce
write parallelism.
- A failed write still emits one error and destroys the socket, as
before.
- The wrapper can emit a loud error after the existing retry budget when
a file gap persists.
- The change does not alter the atomic per-file write behavior.

## Model Used

OpenAI GPT-5, model ID `gpt-5`, with tool use and code execution. The
context window and internal reasoning details are not disclosed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-14 18:45:31 -07:00

624 lines
26 KiB
TypeScript

import { execFile as execFileCallback, spawn } from "node:child_process";
import { mkdir, mkdtemp, readFile, readdir, rename, rm, writeFile } from "node:fs/promises";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { afterEach, describe, expect, it } from "vitest";
import {
getProcessSessionRemoteSource,
startAdapterExecutionTargetProcessSessionBridge,
type AdapterSandboxExecutionTarget,
} from "./execution-target.js";
import { createCommandManagedSandboxCallbackBridgeQueueClient } from "./sandbox-callback-bridge.js";
import { runChildProcess, type RunProcessResult } from "./server-utils.js";
const execFile = promisify(execFileCallback);
// Regression coverage for the stdin file race (parent PAP-4037): the host sends
// each ACP message as a file in the sandbox stdin directory, and a poller in
// the sandbox reads the file and writes the data to the child. Two defects lost
// a message. The host write was not atomic, so the poller could read an empty
// or partial `.json` file. The poller deleted the file before it validated the
// content, so an empty read was lost and a partial read stopped the loop.
describe("stdin file race (parent PAP-4037)", () => {
const cleanupDirs: string[] = [];
afterEach(async () => {
while (cleanupDirs.length > 0) {
const dir = cleanupDirs.pop();
if (dir) await rm(dir, { recursive: true, force: true }).catch(() => undefined);
}
});
// ---- Poller wrapper harness -------------------------------------------
type DeliveredFrame = { seq: number; type: string; stream?: string; data?: string; message?: string };
// Run the real emitted poller wrapper as a node process. The streamed variant
// writes one JSON frame per line to its stdout, so the test reads the frames
// directly. The child command is `cat`, so every byte the poller writes to
// the child stdin comes back as a `data` frame.
async function startPollerWrapper(options?: { maxRetries?: number }) {
const sessionDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-stdin-poll-"));
cleanupDirs.push(sessionDir);
const stdinDir = path.join(sessionDir, "stdin");
await mkdir(stdinDir, { recursive: true });
const wrapperPath = path.join(sessionDir, "wrapper.mjs");
await writeFile(wrapperPath, getProcessSessionRemoteSource({ outputToStdout: true }), "utf8");
const config = { command: "cat", args: [] as string[], cwd: sessionDir, env: {} };
const commandPayload = Buffer.from(JSON.stringify(config), "utf8").toString("base64");
const env: Record<string, string> = {
...process.env,
PAPERCLIP_PROCESS_SESSION_DIR: sessionDir,
PAPERCLIP_PROCESS_SESSION_COMMAND_B64: commandPayload,
};
if (options?.maxRetries != null) {
env.PAPERCLIP_PROCESS_SESSION_STDIN_MAX_RETRIES = String(options.maxRetries);
}
const child = spawn(process.execPath, [wrapperPath], {
cwd: sessionDir,
env,
stdio: ["ignore", "pipe", "pipe"],
});
const frames: DeliveredFrame[] = [];
let stdoutBuffer = "";
child.stdout.on("data", (chunk: Buffer) => {
stdoutBuffer += chunk.toString("utf8");
const lines = stdoutBuffer.split("\n");
stdoutBuffer = lines.pop() || "";
for (const line of lines) {
if (!line.trim()) continue;
frames.push(JSON.parse(line) as DeliveredFrame);
}
});
const exited = new Promise<void>((resolve) => child.on("close", () => resolve()));
return {
sessionDir,
stdinDir,
frames,
// Write a complete stdin file with an atomic rename, so the test never
// creates its own partial-write race.
writeFileAtomic: async (name: string, content: string) => {
const finalPath = path.join(stdinDir, name);
const tempPath = `${finalPath}.writing`;
await writeFile(tempPath, content, "utf8");
await rename(tempPath, finalPath);
},
// Write a `.json` file directly, so a reader can observe it before the
// content lands. This simulates the non-atomic-write window.
writeFileRaw: async (name: string, content: string) => {
await writeFile(path.join(stdinDir, name), content, "utf8");
},
exited,
kill: () => child.kill("SIGKILL"),
};
}
function stdinMessage(text: string): string {
return `${JSON.stringify({ type: "stdin", data: Buffer.from(text, "utf8").toString("base64") })}\n`;
}
const stdinEndMessage = `${JSON.stringify({ type: "stdinEnd" })}\n`;
// Concatenate every stdout `data` frame and decode it back to text.
function collectDelivered(frames: DeliveredFrame[]): string {
return frames
.filter((frame) => frame.type === "data" && frame.stream === "stdout" && typeof frame.data === "string")
.map((frame) => Buffer.from(frame.data as string, "base64").toString("utf8"))
.join("");
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function waitFor(check: () => boolean, timeoutMs = 4_000): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (check()) return;
await delay(20);
}
throw new Error("Timed out waiting for condition.");
}
// ---- Poller tests -----------------------------------------------------
it("delivers a stdin file that appears empty first and then gets content", async () => {
const poller = await startPollerWrapper();
// The file appears empty first (the non-atomic-write window). The poller
// must keep it and retry, not delete it and lose the message.
await poller.writeFileRaw("000000000001.json", "");
await delay(200);
// The poller keeps the empty file for a later retry. A poller that deletes
// before it validates would drop the file here and lose the message.
const afterEmpty = await readdir(poller.stdinDir);
expect(afterEmpty).toContain("000000000001.json");
// The content lands in the same file. The poller must deliver it on a later
// cycle, because it kept the file across the empty read.
await poller.writeFileAtomic("000000000001.json", stdinMessage("late-payload"));
await waitFor(() => collectDelivered(poller.frames).includes("late-payload"));
await poller.writeFileAtomic("000000000002.json", stdinEndMessage);
await poller.exited;
expect(collectDelivered(poller.frames)).toBe("late-payload");
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
it("keeps polling after a malformed file and still delivers a later valid file", async () => {
const poller = await startPollerWrapper({ maxRetries: 3 });
// A malformed file sorts before the valid file. The poller keeps the send
// order: it holds the later file until it drops the malformed file after the
// retry limit, then it delivers the later valid file. So one bad file blocks
// the loop only until the retry limit, not forever.
await poller.writeFileRaw("000000000001.json", "{ this is not valid json");
await poller.writeFileAtomic("000000000002.json", stdinMessage("valid-after-bad"));
await waitFor(() => collectDelivered(poller.frames).includes("valid-after-bad"));
await poller.writeFileAtomic("000000000003.json", stdinEndMessage);
await poller.exited;
expect(collectDelivered(poller.frames)).toBe("valid-after-bad");
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
it("does not close the stream on a later stdinEnd while an earlier file awaits retry", async () => {
const poller = await startPollerWrapper();
// An earlier stdin file is momentarily unreadable (the non-atomic-write
// window). A later stdinEnd file is already complete. The poller must keep
// the send order: it must not read the stdinEnd ahead of the earlier file
// and close the stream. It must hold the stream open until the earlier file
// is readable.
await poller.writeFileRaw("000000000001.json", "");
await poller.writeFileAtomic("000000000002.json", stdinEndMessage);
// Give the poller time to scan. The stream stays open, so the child does not
// exit and no exit frame appears yet.
await delay(300);
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(false);
// The earlier file's content lands. The poller delivers it, then reads the
// stdinEnd and closes the stream.
await poller.writeFileAtomic("000000000001.json", stdinMessage("early-payload"));
await waitFor(() => collectDelivered(poller.frames).includes("early-payload"));
await poller.exited;
expect(collectDelivered(poller.frames)).toBe("early-payload");
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
it("drops a permanently malformed file after the retry limit and writes an error event", async () => {
const poller = await startPollerWrapper({ maxRetries: 3 });
// This file never becomes valid. After the retry limit the poller drops it
// and writes an error event, so the lost message fails loudly.
await poller.writeFileRaw("000000000001.json", "{ permanently broken");
await waitFor(() =>
poller.frames.some(
(frame) => frame.type === "error" && typeof frame.message === "string" && frame.message.includes("Dropped unreadable stdin file"),
),
);
// The loop still works after the drop: a later valid file is delivered.
await poller.writeFileAtomic("000000000002.json", stdinMessage("still-alive"));
await waitFor(() => collectDelivered(poller.frames).includes("still-alive"));
await poller.writeFileAtomic("000000000003.json", stdinEndMessage);
await poller.exited;
expect(collectDelivered(poller.frames)).toBe("still-alive");
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
it("holds a later stdin file until the missing earlier file appears", async () => {
const poller = await startPollerWrapper();
// File 2 is complete, but file 1 has not appeared yet (a host reordering).
// The poller must not deliver file 2 ahead of the missing file 1. It holds
// the send order and waits for the earlier file.
await poller.writeFileAtomic("000000000002.json", stdinMessage("second-payload"));
await delay(300);
// File 2 is still on disk and nothing was delivered: the poller holds it.
const afterHold = await readdir(poller.stdinDir);
expect(afterHold).toContain("000000000002.json");
expect(collectDelivered(poller.frames)).toBe("");
// File 1 arrives. The poller now delivers file 1 then file 2, in send order.
await poller.writeFileAtomic("000000000001.json", stdinMessage("first-payload"));
await waitFor(() => collectDelivered(poller.frames).includes("second-payload"));
expect(collectDelivered(poller.frames)).toBe("first-payloadsecond-payload");
await poller.writeFileAtomic("000000000003.json", stdinEndMessage);
await poller.exited;
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
it("fails loud and advances past a missing stdin file after the retry limit", async () => {
const poller = await startPollerWrapper({ maxRetries: 3 });
// File 1 never appears. File 2 is complete. After the retry limit the poller
// writes a loud error event and advances past the gap, then delivers file 2.
// So a permanent reordering fails loud, never silently.
await poller.writeFileAtomic("000000000002.json", stdinMessage("after-gap"));
await waitFor(() =>
poller.frames.some(
(frame) =>
frame.type === "error" &&
typeof frame.message === "string" &&
frame.message.includes("Advanced past missing stdin files"),
),
);
await waitFor(() => collectDelivered(poller.frames).includes("after-gap"));
await poller.writeFileAtomic("000000000003.json", stdinEndMessage);
await poller.exited;
expect(collectDelivered(poller.frames)).toBe("after-gap");
expect(poller.frames.some((frame) => frame.type === "exit")).toBe(true);
});
// ---- Host serialization test (drives the real bridge) -----------------
// A runner that runs each bridge shell script as a real child process, so the
// test drives the whole legacy-poll bridge: the socket handler, the command-
// managed `writeTextFile` script, the nohup wrapper, and the output poll.
function createLocalSandboxRunner(
onExecute?: (script: string) => Promise<void>,
) {
let counter = 0;
return {
execute: async (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
}): Promise<RunProcessResult> => {
counter += 1;
const script = input.args?.[1] ?? "";
if (onExecute) await onExecute(script);
const command =
input.command === "bash" ? "/bin/bash" : input.command === "sh" ? "/bin/sh" : input.command;
return runChildProcess(`stdin-order-run-${counter}`, command, input.args ?? [], {
cwd: input.cwd ?? process.cwd(),
env: input.env ?? {},
stdin: input.stdin,
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
graceSec: 5,
onLog: input.onLog ?? (async () => {}),
});
},
};
}
it("serializes host stdin writes so a slow earlier write still lands first", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-stdin-host-order-"));
cleanupDirs.push(rootDir);
// The child echoes every stdin byte to stdout, so the wrapper reports the
// exact bytes and order the child received on its stdin.
const childPath = path.join(rootDir, "echo-child.mjs");
await writeFile(childPath, "process.stdin.on('data', (c) => process.stdout.write(c));\n", "utf8");
// Record the send-order-relevant event: the completion of each stdin file's
// finalize (atomic rename). Delay the finalize of the FIRST file, so its
// write resolves slower than the second. Without serialization the second
// rename would land first; the per-session chain must keep the send order.
const finalizeOrder: string[] = [];
const runner = createLocalSandboxRunner(async (script) => {
const finalizeMatch = /base64 -d[\s\S]*mv '[^']*\.decoded' '([^']+\.json)'/.exec(script);
if (finalizeMatch) {
const remotePath = finalizeMatch[1];
if (remotePath.endsWith("000000000001.json")) await delay(300);
finalizeOrder.push(path.posix.basename(remotePath));
}
});
const target: AdapterSandboxExecutionTarget = {
kind: "remote",
transport: "sandbox",
providerKey: "local-test",
remoteCwd: rootDir,
timeoutMs: 30_000,
runner,
};
const bridge = await startAdapterExecutionTargetProcessSessionBridge({
runId: "run-stdin-host-order",
target,
runtimeRootDir: path.posix.join(rootDir, ".paperclip-runtime", "acpx"),
adapterKey: "acpx",
command: process.execPath,
args: [childPath],
cwd: rootDir,
env: {},
timeoutSec: 5,
onLog: async () => {},
});
expect(bridge).not.toBeNull();
let peer: net.Socket | null = null;
try {
const proxySource = await readFile(bridge!.agentCommand, "utf8");
const port = Number(/port: (\d+)/.exec(proxySource)?.[1] ?? Number.NaN);
const tokenLiteral = /const token = (".*?");/.exec(proxySource)?.[1];
expect(Number.isFinite(port)).toBe(true);
const token = JSON.parse(tokenLiteral as string) as string;
const peerSocket = net.createConnection({ host: "127.0.0.1", port });
peer = peerSocket;
peerSocket.setEncoding("utf8");
peerSocket.on("error", () => undefined);
const delivered: string[] = [];
let peerBuffer = "";
peerSocket.on("data", (chunk: string) => {
peerBuffer += chunk;
const lines = peerBuffer.split("\n");
peerBuffer = lines.pop() || "";
for (const line of lines) {
if (!line.trim()) continue;
const frame = JSON.parse(line) as { type?: string; stream?: string; data?: string };
if (frame.type === "data" && frame.stream === "stdout" && typeof frame.data === "string") {
delivered.push(Buffer.from(frame.data, "base64").toString("utf8"));
}
}
});
await new Promise<void>((resolve, reject) => {
peerSocket.once("connect", () => resolve());
peerSocket.once("error", reject);
});
// Send two stdin messages back to back. The first authenticates and writes
// file 1; the second writes file 2. Both are scheduled before file 1's
// delayed finalize resolves, so an un-chained handler would race them.
const head = `${JSON.stringify({ token, type: "stdin", data: Buffer.from("HEAD_ONE_", "utf8").toString("base64") })}\n`;
const tail = `${JSON.stringify({ token, type: "stdin", data: Buffer.from("TAIL_TWO", "utf8").toString("base64") })}\n`;
peerSocket.write(head);
peerSocket.write(tail);
// The two finalize renames complete in send order, not in the order the
// delayed and fast writes would otherwise finish.
await waitFor(() => finalizeOrder.length >= 2, 8_000);
expect(finalizeOrder.slice(0, 2)).toEqual(["000000000001.json", "000000000002.json"]);
// End to end: the child receives the two payloads intact and in send
// order, so the prompt is byte-identical on the child stdin.
await waitFor(() => delivered.join("").includes("TAIL_TWO"), 8_000);
expect(delivered.join("")).toBe("HEAD_ONE_TAIL_TWO");
} finally {
peer?.destroy();
await bridge?.stop();
}
});
it("holds stdinEnd on stop until an earlier pending stdin write lands first", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-stdin-stop-order-"));
cleanupDirs.push(rootDir);
const childPath = path.join(rootDir, "echo-child.mjs");
await writeFile(childPath, "process.stdin.on('data', (c) => process.stdout.write(c));\n", "utf8");
// Record each stdin file finalize (atomic rename). `finalizeStarted` marks
// the start; `finalizeOrder` marks the completion. Delay the FIRST chunk's
// finalize, so its write is still pending when `stop()` runs. `stop()` must
// chain the `stdinEnd` write after the pending chunk, so file 2 (stdinEnd)
// never finishes its rename before file 1.
const finalizeStarted: string[] = [];
const finalizeOrder: string[] = [];
const runner = createLocalSandboxRunner(async (script) => {
const finalizeMatch = /base64 -d[\s\S]*mv '[^']*\.decoded' '([^']+\.json)'/.exec(script);
if (finalizeMatch) {
const name = path.posix.basename(finalizeMatch[1]);
finalizeStarted.push(name);
if (name === "000000000001.json") await delay(300);
finalizeOrder.push(name);
}
});
const target: AdapterSandboxExecutionTarget = {
kind: "remote",
transport: "sandbox",
providerKey: "local-test",
remoteCwd: rootDir,
timeoutMs: 30_000,
runner,
};
const bridge = await startAdapterExecutionTargetProcessSessionBridge({
runId: "run-stdin-stop-order",
target,
runtimeRootDir: path.posix.join(rootDir, ".paperclip-runtime", "acpx"),
adapterKey: "acpx",
command: process.execPath,
args: [childPath],
cwd: rootDir,
env: {},
timeoutSec: 5,
onLog: async () => {},
});
expect(bridge).not.toBeNull();
let peer: net.Socket | null = null;
let stopped = false;
try {
const proxySource = await readFile(bridge!.agentCommand, "utf8");
const port = Number(/port: (\d+)/.exec(proxySource)?.[1] ?? Number.NaN);
const tokenLiteral = /const token = (".*?");/.exec(proxySource)?.[1];
expect(Number.isFinite(port)).toBe(true);
const token = JSON.parse(tokenLiteral as string) as string;
const peerSocket = net.createConnection({ host: "127.0.0.1", port });
peer = peerSocket;
peerSocket.setEncoding("utf8");
peerSocket.on("error", () => undefined);
peerSocket.on("data", () => undefined);
await new Promise<void>((resolve, reject) => {
peerSocket.once("connect", () => resolve());
peerSocket.once("error", reject);
});
// Send one stdin message. It authenticates and writes file 1, whose
// finalize the runner delays. Wait until that finalize has started, so the
// write is in flight when `stop()` runs.
const head = `${JSON.stringify({ token, type: "stdin", data: Buffer.from("HEAD_ONE_", "utf8").toString("base64") })}\n`;
peerSocket.write(head);
await waitFor(() => finalizeStarted.includes("000000000001.json"), 8_000);
// Stop the bridge while file 1's write is still pending. `stop()` awaits
// the chained `stdinEnd` write, so both finalizes are complete when it
// returns, in send order.
await bridge!.stop();
stopped = true;
expect(finalizeOrder).toEqual(["000000000001.json", "000000000002.json"]);
} finally {
peer?.destroy();
if (!stopped) await bridge?.stop();
}
});
// ---- Host atomic-write tests ------------------------------------------
// A runner that executes each bridge shell script on the local filesystem,
// so the test exercises the real command-managed `writeTextFile` script.
function createLocalShellRunner(scripts: string[]) {
return {
execute: async (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
}): Promise<RunProcessResult> => {
const args = input.args ?? [];
if ((input.command === "sh" || input.command === "bash") && args[0] === "-c" && typeof args[1] === "string") {
scripts.push(args[1]);
}
const command = input.command === "sh" ? "/bin/sh" : input.command === "bash" ? "/bin/bash" : input.command;
try {
const result = await execFile(command, args, {
cwd: input.cwd,
env: { ...process.env, ...input.env },
maxBuffer: 32 * 1024 * 1024,
});
return {
exitCode: 0,
signal: null,
timedOut: false,
stdout: result.stdout,
stderr: result.stderr,
pid: null,
startedAt: null,
};
} catch (error) {
const err = error as NodeJS.ErrnoException & { stdout?: string; stderr?: string; code?: string | number | null };
return {
exitCode: typeof err.code === "number" ? err.code : null,
signal: null,
timedOut: false,
stdout: err.stdout ?? "",
stderr: err.stderr ?? "",
pid: null,
startedAt: null,
};
}
},
};
}
it("finalizes the command-managed host write with an atomic rename onto the .json path", async () => {
const remoteRoot = await mkdtemp(path.join(os.tmpdir(), "paperclip-stdin-host-cmd-"));
cleanupDirs.push(remoteRoot);
const stdinDir = path.join(remoteRoot, "stdin");
await mkdir(stdinDir, { recursive: true });
const scripts: string[] = [];
const client = createCommandManagedSandboxCallbackBridgeQueueClient({
runner: createLocalShellRunner(scripts),
remoteCwd: remoteRoot,
timeoutMs: 30_000,
});
const jsonPath = path.join(stdinDir, "000000000001.json");
const body = `${JSON.stringify({ type: "stdin", data: Buffer.from("host-payload", "utf8").toString("base64") })}\n`;
await client.writeTextFile(jsonPath, body);
// The final file holds the complete body.
expect(await readFile(jsonPath, "utf8")).toBe(body);
// No temporary upload file remains next to the final file.
const entries = await readdir(stdinDir);
expect(entries).toEqual(["000000000001.json"]);
// The finalize script renames a non-`.json` temporary file onto the final
// path. It never redirects the decode output straight into the `.json`
// file, so a reader never sees an empty or partial `.json` file.
const finalizeScript = scripts.find((script) => script.includes("base64 -d"));
expect(finalizeScript).toBeDefined();
expect(finalizeScript).toContain(`mv `);
expect(finalizeScript).not.toContain(`> '${jsonPath}'`);
expect(finalizeScript).toContain(`> '${jsonPath}.paperclip-upload.decoded'`);
});
it("never exposes a partial .json file under a concurrent reader (command-managed host write)", async () => {
const remoteRoot = await mkdtemp(path.join(os.tmpdir(), "paperclip-stdin-host-race-"));
cleanupDirs.push(remoteRoot);
const stdinDir = path.join(remoteRoot, "stdin");
await mkdir(stdinDir, { recursive: true });
const client = createCommandManagedSandboxCallbackBridgeQueueClient({
runner: createLocalShellRunner([]),
remoteCwd: remoteRoot,
timeoutMs: 30_000,
});
const jsonPath = path.join(stdinDir, "000000000001.json");
// A large body needs many decode bytes, so the write window is wide.
const bigText = "x".repeat(64 * 1024);
const body = `${JSON.stringify({ type: "stdin", data: Buffer.from(bigText, "utf8").toString("base64") })}\n`;
let stop = false;
const readerErrors: string[] = [];
let observedComplete = 0;
const reader = (async () => {
while (!stop) {
const raw = await readFile(jsonPath, "utf8").catch(() => null);
if (raw) {
try {
JSON.parse(raw);
observedComplete += 1;
} catch (error) {
readerErrors.push(error instanceof Error ? error.message : String(error));
}
}
}
})();
for (let round = 0; round < 6; round += 1) {
await client.remove(jsonPath);
await client.writeTextFile(jsonPath, body);
}
stop = true;
await reader;
// Every read of the final file parsed as complete JSON. The reader never
// saw an empty or partial `.json` file.
expect(readerErrors).toEqual([]);
expect(observedComplete).toBeGreaterThan(0);
});
});