test(runner): remove coreutils and pid-file timing assumptions from static checks (#21787)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner package has a "static checks" CI job that runs
`node --test` script tests
> - Two of these tests failed on unrelated pull requests on 2026-11-15
and passed on re-run
> - Each failure costs a re-run of the failed jobs and delays merges,
and it teaches people to re-run instead of read
> - One test assumed that a copied `/bin/sleep` acts as `sleep` under
any name. The other read a pid file before the child finished writing it
> - This pull request removes both assumptions, and one more instance of
the second pattern in the same test family
> - The benefit is that these tests no longer depend on the runner image
or on process scheduling

## Linked Issues or Issue Description

**What happened?**

1. `scripts/stage-runner-binary.test.mjs` failed in "ci / Verify
Paperclip Runner (static checks)" on PR #21765, head `98197c87e9`, job
132850068174:
- `atomically publishes while the previous executable is running`:
`AssertionError: 1 !== null` at line 124 (`old.child.exitCode`), after
77 ms.
- `a source rebuild selects fresh dev bytes ...`: `Expected "actual" not
to be strictly deep-equal` (line 192).
- That job ran on a GitHub-hosted `ubuntu-26.04` image (20261111.82.6).
The same tests pass on the RunsOn `ubuntu24` image, for example job
132822472729.
2. `scripts/materialize-kimi-distribution.test.mjs` › `setup
cancellation physically stops an in-flight owned child` failed in the
same job on PR #21758, head `40e38fc28a`, job 132822472729:
`AssertionError: The real child did not start` after 149 ms.

**Expected behavior**
The tests pass on every supported runner image and on a loaded runner.

**Steps to reproduce**
1. stage-runner-binary: in a Linux container, replace `/bin/sleep` and
`/bin/echo` with one multicall binary that picks the utility from
`argv[0]` and exits 1 for an unknown name (this is how uutils coreutils
works on Ubuntu 26.04). Run `node --test
scripts/stage-runner-binary.test.mjs`. On `master` both tests fail with
the CI messages, 20 of 20 runs.
2. kimi: make the child create the pid file, wait 300 ms, then write the
pid. On `master` the test fails with `The real child did not start` in
about 125 ms, the same as CI.

**Root cause**
1. On Ubuntu 26.04, `/bin/sleep` and `/bin/echo` are the same uutils
multicall binary. The test copies it to `paperclip-runnerd` and runs it.
The binary sees `argv[0] = paperclip-runnerd`, finds no utility of that
name, and exits 1. The fresh-build fixture (a copy of `echo`) also has
the same bytes as the old artifact (a copy of `sleep`), so the "fresh
bytes" assertion fails too.
2. The child wrote the pid with `writeFileSync`, which creates the file
and then writes it. The test loop broke on the first successful read. An
empty read gave `Number("") === 0`, so the loop stopped and the
assertion failed. The loop also had a fixed budget of 100 polls of 10 ms
for a Node child to start.

**Production impact**
None. Only tests change. The staging code works with any executable. The
Kimi setup cancellation code is correct: the deterministic repro shows
that the child is stopped once the test reads a real pid.

## What Changed

- `stage-runner-binary.test.mjs`: the test helper passes `argv0` to
`spawn`. The copied `sleep` runs with `argv0: "sleep"`, so a multicall
binary still acts as `sleep`. The fresh-build fixture is now a copy of
`/bin/sh` and runs `-c "exit 0"`, so its bytes always differ from the
old `sleep` artifact.
- `materialize-kimi-distribution.test.mjs`: the child writes `pid.tmp`
and renames it to `pid`, so a read never sees a partial file. The test
waits until the pid is valid or the command settles. The 3 s command
timeout and 5 s test timeout are unchanged.
- `provision-kimi-package.test.mjs`: the wait for the `download-ready`
marker continues until the child exits, instead of a fixed 1 s budget.
The existing 5 s kill timer still bounds it.
- No retries, skips or timeout changes.

## Verification

- stage-runner-binary, Linux container (`node:24-bookworm-slim`) with a
multicall `sleep`/`echo` built from a 15-line Go program that dispatches
on `argv[0]`:
  - `master`: 0/20 pass, same two assertions as CI.
  - this branch: 50/50 pass.
  - this branch with stock coreutils, and on macOS: all 9 tests pass.
- kimi setup cancellation:
  - delayed-write repro: `master` fails, this branch passes.
- loop on macOS, `master` test without extra load: 59/60 (one natural
failure). This branch: 60/60.
- loop with 4 extra busy-loop processes (load average about 13 before
them): `master` 100/100, this branch 100/100.
- `node --test scripts/stage-runner-binary.test.mjs
scripts/materialize-kimi-distribution.test.mjs
scripts/provision-kimi-package.test.mjs`: 19 pass, 1 skipped (public
download test, opt-in).
- The other CI flake from the same day,
`runnerd-codex-transport-orphaned-tool-at-attach.test.ts`, is a Runner
defect. It is fixed in a separate pull request.

## Risks

- Low risk. Only tests change.
- The rebuild test now uses `/bin/sh` as the fixture for the fresh
build. This file exists on every supported Linux and macOS runner.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, GitHub CLI, Docker).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-15 10:13:19 -06:00
1 parent d14a3b88b1
commit 7a9177af0f
3 files changed
+15 -13

No files matched your search

@@ -39,12 +39,14 @@ test("distribution build admits network settings without provider keys or Node i
test("setup cancellation physically stops an in-flight owned child", { timeout: 5_000 }, async t => {
const { root } = await fixture(t), pidFile = join(root, "pid");
const controller = new AbortController();
const result = runKimiBuildCommand(process.execPath, ["-e", "require('node:fs').writeFileSync(process.argv[1],String(process.pid));setInterval(()=>{},1000)", pidFile], { env: {}, timeout: 3_000 }, undefined, controller.signal).then(() => null, error => error);
let pid;
const result = runKimiBuildCommand(process.execPath, ["-e", "const f=require('node:fs');f.writeFileSync(process.argv[1]+'.tmp',String(process.pid));f.renameSync(process.argv[1]+'.tmp',process.argv[1]);setInterval(()=>{},1000)", pidFile], { env: {}, timeout: 3_000 }, undefined, controller.signal).then(() => null, error => error);
let pid, settled = false;
void result.then(() => { settled = true; });
try {
for (let attempt = 0; attempt < 100; attempt++) {
try { pid = Number(await readFile(pidFile, "utf8")); break; }
catch (error) { if (error.code !== "ENOENT") throw error; await delay(10); }
while (!settled && !(Number.isSafeInteger(pid) && pid > 0)) {
try { pid = Number(await readFile(pidFile, "utf8")); }
catch (error) { if (error.code !== "ENOENT") throw error; }
await delay(10);
}
assert.ok(Number.isSafeInteger(pid) && pid > 0, "The real child did not start");
controller.abort();
@@ -54,7 +54,7 @@ test("installed Kimi setup is self-contained, rejects corrupt assets and cancels
let stderr = ""; cancelled.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4096); });
const kill = setTimeout(() => cancelled.kill("SIGKILL"), 5_000);
try {
for (let attempt = 0; attempt < 100; attempt++) {
while (cancelled.exitCode === null && cancelled.signalCode === null) {
try { if (await readFile(ready, "utf8") === "ready") break; }
catch (error) { if (error.code !== "ENOENT") throw error; }
await delay(10);
@@ -86,8 +86,8 @@ test("release metadata rejects incompatible native, harness and protocol contrac
async function withOwnedChildren(root, action) {
const owned = [];
const start = (command, args) => {
const child = spawn(command, args, { env: { PATH: "/usr/bin:/bin" }, stdio: "ignore" });
const start = (command, args, argv0) => {
const child = spawn(command, args, { argv0, env: { PATH: "/usr/bin:/bin" }, stdio: "ignore" });
// close follows both exit and spawn error. A failed second spawn must not
// interrupt retirement of the first still-running executable.
const closed = new Promise(resolve => child.once("close", resolve));
@@ -115,7 +115,7 @@ test("atomically publishes while the previous executable is running", { skip: pr
await withOwnedChildren(root, async start => {
await stageRunnerBinary("/bin/sleep", destination);
const previous = await stat(destination);
const old = start(destination, ["30"]);
const old = start(destination, ["30"], "sleep");
await old.spawned;
await stageRunnerBinary("/bin/sleep", destination);
const published = await stat(destination);
@@ -124,7 +124,7 @@ test("atomically publishes while the previous executable is running", { skip: pr
assert.equal(old.child.exitCode, null);
assert.equal(old.child.signalCode, null);
assert.deepEqual(await readdir(root), ["paperclip-runnerd"]);
const next = start(destination, ["0"]);
const next = start(destination, ["0"], "sleep");
await next.spawned; await next.closed;
assert.equal(next.child.exitCode, 0);
});
@@ -150,7 +150,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries
await mkdir(path.dirname(source), { recursive: true });
await writeFile(source, "fn main() {}\n");
await mkdir(path.dirname(built), { recursive: true });
await copyHostFixture("/bin/echo", built);
await copyHostFixture("/bin/sh", built);
const previousBuild = path.join(root, "previous-build");
await copyHostFixture("/bin/sleep", previousBuild);
await stageRunnerBinary(previousBuild, hostTarget);
@@ -178,7 +178,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries
const manifestPath = path.join(binaryRoot, "release-manifest.json");
const manifest = JSON.stringify({ schema: "paperclip.runner.release-binaries.v1", sourceRevision: "a".repeat(40), platforms });
await writeFile(manifestPath, manifest);
const old = start(hostTarget, ["30"]);
const old = start(hostTarget, ["30"], "sleep");
await old.spawned;
const requirement = { repoRoot: root, nativeRunnerRequired: true };
assert.equal(paperclipRunnerBinaryNeedsBuild(requirement), true);
@@ -198,7 +198,7 @@ test("a source rebuild selects fresh dev bytes while preserving release binaries
assert.equal(paperclipRunnerBinaryNeedsBuild(requirement), false);
assert.equal(old.child.exitCode, null);
assert.equal(old.child.signalCode, null);
const next = start(selected.binary, ["rebuilt"]);
const next = start(selected.binary, ["-c", "exit 0"]);
await next.spawned; await next.closed;
assert.equal(next.child.exitCode, 0);
});