From 7a9177af0f5aa9cf925e9d40723944e39723cd7b Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Sun, 15 Nov 2026 10:13:19 -0600 Subject: [PATCH] test(runner): remove coreutils and pid-file timing assumptions from static checks (#21787) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip Runner package has a "static checks" CI job that runs `node --test` script tests > - Two of these tests failed on unrelated pull requests on 2026-11-15 and passed on re-run > - Each failure costs a re-run of the failed jobs and delays merges, and it teaches people to re-run instead of read > - One test assumed that a copied `/bin/sleep` acts as `sleep` under any name. The other read a pid file before the child finished writing it > - This pull request removes both assumptions, and one more instance of the second pattern in the same test family > - The benefit is that these tests no longer depend on the runner image or on process scheduling ## Linked Issues or Issue Description **What happened?** 1. `scripts/stage-runner-binary.test.mjs` failed in "ci / Verify Paperclip Runner (static checks)" on PR #21765, head `98197c87e9`, job 132850068174: - `atomically publishes while the previous executable is running`: `AssertionError: 1 !== null` at line 124 (`old.child.exitCode`), after 77 ms. - `a source rebuild selects fresh dev bytes ...`: `Expected "actual" not to be strictly deep-equal` (line 192). - That job ran on a GitHub-hosted `ubuntu-26.04` image (20261111.82.6). The same tests pass on the RunsOn `ubuntu24` image, for example job 132822472729. 2. `scripts/materialize-kimi-distribution.test.mjs` › `setup cancellation physically stops an in-flight owned child` failed in the same job on PR #21758, head `40e38fc28a`, job 132822472729: `AssertionError: The real child did not start` after 149 ms. **Expected behavior** The tests pass on every supported runner image and on a loaded runner. **Steps to reproduce** 1. stage-runner-binary: in a Linux container, replace `/bin/sleep` and `/bin/echo` with one multicall binary that picks the utility from `argv[0]` and exits 1 for an unknown name (this is how uutils coreutils works on Ubuntu 26.04). Run `node --test scripts/stage-runner-binary.test.mjs`. On `master` both tests fail with the CI messages, 20 of 20 runs. 2. kimi: make the child create the pid file, wait 300 ms, then write the pid. On `master` the test fails with `The real child did not start` in about 125 ms, the same as CI. **Root cause** 1. On Ubuntu 26.04, `/bin/sleep` and `/bin/echo` are the same uutils multicall binary. The test copies it to `paperclip-runnerd` and runs it. The binary sees `argv[0] = paperclip-runnerd`, finds no utility of that name, and exits 1. The fresh-build fixture (a copy of `echo`) also has the same bytes as the old artifact (a copy of `sleep`), so the "fresh bytes" assertion fails too. 2. The child wrote the pid with `writeFileSync`, which creates the file and then writes it. The test loop broke on the first successful read. An empty read gave `Number("") === 0`, so the loop stopped and the assertion failed. The loop also had a fixed budget of 100 polls of 10 ms for a Node child to start. **Production impact** None. Only tests change. The staging code works with any executable. The Kimi setup cancellation code is correct: the deterministic repro shows that the child is stopped once the test reads a real pid. ## What Changed - `stage-runner-binary.test.mjs`: the test helper passes `argv0` to `spawn`. The copied `sleep` runs with `argv0: "sleep"`, so a multicall binary still acts as `sleep`. The fresh-build fixture is now a copy of `/bin/sh` and runs `-c "exit 0"`, so its bytes always differ from the old `sleep` artifact. - `materialize-kimi-distribution.test.mjs`: the child writes `pid.tmp` and renames it to `pid`, so a read never sees a partial file. The test waits until the pid is valid or the command settles. The 3 s command timeout and 5 s test timeout are unchanged. - `provision-kimi-package.test.mjs`: the wait for the `download-ready` marker continues until the child exits, instead of a fixed 1 s budget. The existing 5 s kill timer still bounds it. - No retries, skips or timeout changes. ## Verification - stage-runner-binary, Linux container (`node:24-bookworm-slim`) with a multicall `sleep`/`echo` built from a 15-line Go program that dispatches on `argv[0]`: - `master`: 0/20 pass, same two assertions as CI. - this branch: 50/50 pass. - this branch with stock coreutils, and on macOS: all 9 tests pass. - kimi setup cancellation: - delayed-write repro: `master` fails, this branch passes. - loop on macOS, `master` test without extra load: 59/60 (one natural failure). This branch: 60/60. - loop with 4 extra busy-loop processes (load average about 13 before them): `master` 100/100, this branch 100/100. - `node --test scripts/stage-runner-binary.test.mjs scripts/materialize-kimi-distribution.test.mjs scripts/provision-kimi-package.test.mjs`: 19 pass, 1 skipped (public download test, opt-in). - The other CI flake from the same day, `runnerd-codex-transport-orphaned-tool-at-attach.test.ts`, is a Runner defect. It is fixed in a separate pull request. ## Risks - Low risk. Only tests change. - The rebuild test now uses `/bin/sh` as the fixture for the fresh build. This file exists on every supported Linux and macOS runner. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, GitHub CLI, Docker). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 --- .../scripts/materialize-kimi-distribution.test.mjs | 12 +++++++----- .../scripts/provision-kimi-package.test.mjs | 2 +- .../scripts/stage-runner-binary.test.mjs | 14 +++++++------- 3 files changed, 15 insertions(+), 13 deletions(-) diff --git a/packages/paperclip-runner/scripts/materialize-kimi-distribution.test.mjs b/packages/paperclip-runner/scripts/materialize-kimi-distribution.test.mjs index a3adb002918..0bd816430a9 100644 --- a/packages/paperclip-runner/scripts/materialize-kimi-distribution.test.mjs +++ b/packages/paperclip-runner/scripts/materialize-kimi-distribution.test.mjs @@ -39,12 +39,14 @@ test("distribution build admits network settings without provider keys or Node i test("setup cancellation physically stops an in-flight owned child", { timeout: 5_000 }, async t => { const { root } = await fixture(t), pidFile = join(root, "pid"); const controller = new AbortController(); - const result = runKimiBuildCommand(process.execPath, ["-e", "require('node:fs').writeFileSync(process.argv[1],String(process.pid));setInterval(()=>{},1000)", pidFile], { env: {}, timeout: 3_000 }, undefined, controller.signal).then(() => null, error => error); - let pid; + const result = runKimiBuildCommand(process.execPath, ["-e", "const f=require('node:fs');f.writeFileSync(process.argv[1]+'.tmp',String(process.pid));f.renameSync(process.argv[1]+'.tmp',process.argv[1]);setInterval(()=>{},1000)", pidFile], { env: {}, timeout: 3_000 }, undefined, controller.signal).then(() => null, error => error); + let pid, settled = false; + void result.then(() => { settled = true; }); try { - for (let attempt = 0; attempt < 100; attempt++) { - try { pid = Number(await readFile(pidFile, "utf8")); break; } - catch (error) { if (error.code !== "ENOENT") throw error; await delay(10); } + while (!settled && !(Number.isSafeInteger(pid) && pid > 0)) { + try { pid = Number(await readFile(pidFile, "utf8")); } + catch (error) { if (error.code !== "ENOENT") throw error; } + await delay(10); } assert.ok(Number.isSafeInteger(pid) && pid > 0, "The real child did not start"); controller.abort(); diff --git a/packages/paperclip-runner/scripts/provision-kimi-package.test.mjs b/packages/paperclip-runner/scripts/provision-kimi-package.test.mjs index acd9f190df9..e9ae78dfa81 100644 --- a/packages/paperclip-runner/scripts/provision-kimi-package.test.mjs +++ b/packages/paperclip-runner/scripts/provision-kimi-package.test.mjs @@ -54,7 +54,7 @@ test("installed Kimi setup is self-contained, rejects corrupt assets and cancels let stderr = ""; cancelled.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4096); }); const kill = setTimeout(() => cancelled.kill("SIGKILL"), 5_000); try { - for (let attempt = 0; attempt < 100; attempt++) { + while (cancelled.exitCode === null && cancelled.signalCode === null) { try { if (await readFile(ready, "utf8") === "ready") break; } catch (error) { if (error.code !== "ENOENT") throw error; } await delay(10); diff --git a/packages/paperclip-runner/scripts/stage-runner-binary.test.mjs b/packages/paperclip-runner/scripts/stage-runner-binary.test.mjs index e262d64a213..2127a9d1842 100644 --- a/packages/paperclip-runner/scripts/stage-runner-binary.test.mjs +++ b/packages/paperclip-runner/scripts/stage-runner-binary.test.mjs @@ -86,8 +86,8 @@ test("release metadata rejects incompatible native, harness and protocol contrac async function withOwnedChildren(root, action) { const owned = []; - const start = (command, args) => { - const child = spawn(command, args, { env: { PATH: "/usr/bin:/bin" }, stdio: "ignore" }); + const start = (command, args, argv0) => { + const child = spawn(command, args, { argv0, env: { PATH: "/usr/bin:/bin" }, stdio: "ignore" }); // close follows both exit and spawn error. A failed second spawn must not // interrupt retirement of the first still-running executable. const closed = new Promise(resolve => child.once("close", resolve)); @@ -115,7 +115,7 @@ test("atomically publishes while the previous executable is running", { skip: pr await withOwnedChildren(root, async start => { await stageRunnerBinary("/bin/sleep", destination); const previous = await stat(destination); - const old = start(destination, ["30"]); + const old = start(destination, ["30"], "sleep"); await old.spawned; await stageRunnerBinary("/bin/sleep", destination); const published = await stat(destination); @@ -124,7 +124,7 @@ test("atomically publishes while the previous executable is running", { skip: pr assert.equal(old.child.exitCode, null); assert.equal(old.child.signalCode, null); assert.deepEqual(await readdir(root), ["paperclip-runnerd"]); - const next = start(destination, ["0"]); + const next = start(destination, ["0"], "sleep"); await next.spawned; await next.closed; assert.equal(next.child.exitCode, 0); }); @@ -150,7 +150,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries await mkdir(path.dirname(source), { recursive: true }); await writeFile(source, "fn main() {}\n"); await mkdir(path.dirname(built), { recursive: true }); - await copyHostFixture("/bin/echo", built); + await copyHostFixture("/bin/sh", built); const previousBuild = path.join(root, "previous-build"); await copyHostFixture("/bin/sleep", previousBuild); await stageRunnerBinary(previousBuild, hostTarget); @@ -178,7 +178,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries const manifestPath = path.join(binaryRoot, "release-manifest.json"); const manifest = JSON.stringify({ schema: "paperclip.runner.release-binaries.v1", sourceRevision: "a".repeat(40), platforms }); await writeFile(manifestPath, manifest); - const old = start(hostTarget, ["30"]); + const old = start(hostTarget, ["30"], "sleep"); await old.spawned; const requirement = { repoRoot: root, nativeRunnerRequired: true }; assert.equal(paperclipRunnerBinaryNeedsBuild(requirement), true); @@ -198,7 +198,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries assert.equal(paperclipRunnerBinaryNeedsBuild(requirement), false); assert.equal(old.child.exitCode, null); assert.equal(old.child.signalCode, null); - const next = start(selected.binary, ["rebuilt"]); + const next = start(selected.binary, ["-c", "exit 0"]); await next.spawned; await next.closed; assert.equal(next.child.exitCode, 0); });