Files
PaperClipAI/packages/paperclip-runner/scripts/provision-kimi-package.test.mjs
T
DottaandClaude Opus 5.5 7a9177af0f test(runner): remove coreutils and pid-file timing assumptions from static checks (#21787)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner package has a "static checks" CI job that runs
`node --test` script tests
> - Two of these tests failed on unrelated pull requests on 2026-11-15
and passed on re-run
> - Each failure costs a re-run of the failed jobs and delays merges,
and it teaches people to re-run instead of read
> - One test assumed that a copied `/bin/sleep` acts as `sleep` under
any name. The other read a pid file before the child finished writing it
> - This pull request removes both assumptions, and one more instance of
the second pattern in the same test family
> - The benefit is that these tests no longer depend on the runner image
or on process scheduling

## Linked Issues or Issue Description

**What happened?**

1. `scripts/stage-runner-binary.test.mjs` failed in "ci / Verify
Paperclip Runner (static checks)" on PR #21765, head `98197c87e9`, job
132850068174:
- `atomically publishes while the previous executable is running`:
`AssertionError: 1 !== null` at line 124 (`old.child.exitCode`), after
77 ms.
- `a source rebuild selects fresh dev bytes ...`: `Expected "actual" not
to be strictly deep-equal` (line 192).
- That job ran on a GitHub-hosted `ubuntu-26.04` image (20261111.82.6).
The same tests pass on the RunsOn `ubuntu24` image, for example job
132822472729.
2. `scripts/materialize-kimi-distribution.test.mjs` › `setup
cancellation physically stops an in-flight owned child` failed in the
same job on PR #21758, head `40e38fc28a`, job 132822472729:
`AssertionError: The real child did not start` after 149 ms.

**Expected behavior**
The tests pass on every supported runner image and on a loaded runner.

**Steps to reproduce**
1. stage-runner-binary: in a Linux container, replace `/bin/sleep` and
`/bin/echo` with one multicall binary that picks the utility from
`argv[0]` and exits 1 for an unknown name (this is how uutils coreutils
works on Ubuntu 26.04). Run `node --test
scripts/stage-runner-binary.test.mjs`. On `master` both tests fail with
the CI messages, 20 of 20 runs.
2. kimi: make the child create the pid file, wait 300 ms, then write the
pid. On `master` the test fails with `The real child did not start` in
about 125 ms, the same as CI.

**Root cause**
1. On Ubuntu 26.04, `/bin/sleep` and `/bin/echo` are the same uutils
multicall binary. The test copies it to `paperclip-runnerd` and runs it.
The binary sees `argv[0] = paperclip-runnerd`, finds no utility of that
name, and exits 1. The fresh-build fixture (a copy of `echo`) also has
the same bytes as the old artifact (a copy of `sleep`), so the "fresh
bytes" assertion fails too.
2. The child wrote the pid with `writeFileSync`, which creates the file
and then writes it. The test loop broke on the first successful read. An
empty read gave `Number("") === 0`, so the loop stopped and the
assertion failed. The loop also had a fixed budget of 100 polls of 10 ms
for a Node child to start.

**Production impact**
None. Only tests change. The staging code works with any executable. The
Kimi setup cancellation code is correct: the deterministic repro shows
that the child is stopped once the test reads a real pid.

## What Changed

- `stage-runner-binary.test.mjs`: the test helper passes `argv0` to
`spawn`. The copied `sleep` runs with `argv0: "sleep"`, so a multicall
binary still acts as `sleep`. The fresh-build fixture is now a copy of
`/bin/sh` and runs `-c "exit 0"`, so its bytes always differ from the
old `sleep` artifact.
- `materialize-kimi-distribution.test.mjs`: the child writes `pid.tmp`
and renames it to `pid`, so a read never sees a partial file. The test
waits until the pid is valid or the command settles. The 3 s command
timeout and 5 s test timeout are unchanged.
- `provision-kimi-package.test.mjs`: the wait for the `download-ready`
marker continues until the child exits, instead of a fixed 1 s budget.
The existing 5 s kill timer still bounds it.
- No retries, skips or timeout changes.

## Verification

- stage-runner-binary, Linux container (`node:24-bookworm-slim`) with a
multicall `sleep`/`echo` built from a 15-line Go program that dispatches
on `argv[0]`:
  - `master`: 0/20 pass, same two assertions as CI.
  - this branch: 50/50 pass.
  - this branch with stock coreutils, and on macOS: all 9 tests pass.
- kimi setup cancellation:
  - delayed-write repro: `master` fails, this branch passes.
- loop on macOS, `master` test without extra load: 59/60 (one natural
failure). This branch: 60/60.
- loop with 4 extra busy-loop processes (load average about 13 before
them): `master` 100/100, this branch 100/100.
- `node --test scripts/stage-runner-binary.test.mjs
scripts/materialize-kimi-distribution.test.mjs
scripts/provision-kimi-package.test.mjs`: 19 pass, 1 skipped (public
download test, opt-in).
- The other CI flake from the same day,
`runnerd-codex-transport-orphaned-tool-at-attach.test.ts`, is a Runner
defect. It is fixed in a separate pull request.

## Risks

- Low risk. Only tests change.
- The rebuild test now uses `/bin/sh` as the fixture for the fresh
build. This file exists on every supported Linux and macOS runner.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, GitHub CLI, Docker).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 10:13:19 -06:00

113 lines
9.4 KiB
JavaScript

import assert from "node:assert/strict";
import { execFile, spawn, spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { once } from "node:events";
import { mkdtemp, mkdir, readFile, readdir, realpath, rm, symlink, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import test from "node:test";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { bundleKimiProvisioner } from "./build-verified-provider-entrypoints.mjs";
import { KIMI_DISTRIBUTION_CLOSURE_SHA256 } from "../src/drivers/acpx/kimi-closure-pins.ts";
const run = promisify(execFile);
test("installed Kimi setup is self-contained, rejects corrupt assets and cancels pending downloads", { timeout: 10_000 }, async t => {
const root = await realpath(await mkdtemp(join(tmpdir(), "paperclip-kimi-public-layout-")));
t.after(() => rm(root, { recursive: true, force: true }));
const pkg = join(root, "package"), cli = join(pkg, "dist/vendor/paperclip-runner/cli");
await mkdir(cli, { recursive: true });
await writeFile(join(pkg, "package.json"), '{"name":"@paperclipai/server","type":"module"}');
await writeFile(join(pkg, "dist/index.js"), 'throw Error("server must not start");');
await writeFile(join(cli, "acpx-runtime-sidecar.cjs"), "// installed layout fixture");
const entry = join(cli, "provision-kimi.cjs");
const packageRoot = resolve(dirname(new URL(import.meta.url).pathname), "..");
const result = await bundleKimiProvisioner({ outputRoot: cli });
for (const output of Object.values(result.metafile.outputs)) for (const dependency of output.imports) assert.match(dependency.path, /^node:/);
assert.deepEqual((await readdir(join(cli, "kimi-provision-inputs"))).sort(), ["package-lock.json", "package.json"]);
for (const name of ["package-lock.json", "package.json"]) assert.deepEqual(await readFile(join(cli, "kimi-provision-inputs", name)), await readFile(join(packageRoot, "scripts/kimi-distribution", name)));
const setup = createRequire(import.meta.url)(entry);
const layout = await setup.provisionKimiPackageRoot(entry);
assert.equal(layout.root, pkg);
assert.equal(layout.assetRoot, join(pkg, "dist/vendor/paperclip-runner"));
const linked = join(cli, "linked.cjs"); await symlink(entry, linked);
await assert.rejects(setup.provisionKimiPackageRoot(linked), /installed entrypoint/);
assert.deepEqual(setup.provisionEnvironment({ PATH: "/usr/bin:/bin", HTTPS_PROXY: "http://127.0.0.1:9", KIMI_MODEL_API_KEY: "sensitive-canary", HOME: "/private/home", NODE_OPTIONS: "--require /foreign", NODE_PATH: "/foreign", NPM_TOKEN: "sensitive-canary" }), { PATH: "/usr/bin:/bin", LANG: "C.UTF-8", HTTPS_PROXY: "http://127.0.0.1:9" });
const target = `${process.platform}-${process.arch}`;
const parent = join(layout.assetRoot, "provider-assets/kimi");
await mkdir(join(parent, target, "runtime"), { recursive: true });
const deny = join(root, "deny.cjs");
await writeFile(deny, `process.nextTick(()=>{const assert=require('node:assert/strict');for(const key of ['KIMI_MODEL_API_KEY','NPM_TOKEN','HOME','NODE_OPTIONS','NODE_PATH'])assert.equal(process.env[key],undefined,key)});const fail=()=>{throw Error('UNEXPECTED_NETWORK_OR_CHILD')};globalThis.fetch=fail;for(const m of ['node:net','node:tls','node:http','node:https']){const x=require(m);for(const key of ['connect','createConnection','request','get'])if(key in x)x[key]=fail}const c=require('node:child_process');for(const key of ['spawn','execFile','exec'])c[key]=fail;`);
const child = spawnSync(process.execPath, ["--require", deny, entry], { env: { PATH: "/usr/bin:/bin", KIMI_MODEL_API_KEY: "sensitive-canary", NPM_TOKEN: "sensitive-canary", HOME: "/private/home", NODE_OPTIONS: "", NODE_PATH: "/foreign" }, encoding: "utf8", timeout: 10_000 });
assert.ifError(child.error); assert.equal(child.status, 1); assert.match(child.stderr, /Kimi setup failed/);
assert.doesNotMatch(child.stderr + child.stdout, /UNEXPECTED_NETWORK_OR_CHILD|sensitive-canary/);
assert.deepEqual(await readdir(parent), [target]);
assert.equal(JSON.parse(await readFile(join(pkg, "package.json"), "utf8")).name, "@paperclipai/server");
await rm(join(parent, target), { recursive: true });
const ready = join(root, "download-ready"), aborted = join(root, "download-aborted");
const preload = join(root, "pending-download.cjs");
await writeFile(preload, `globalThis.fetch=(_url,{signal})=>new Promise((_resolve,reject)=>{const pending=setInterval(()=>{},1000);require('node:fs').writeFileSync(${JSON.stringify(ready)},'ready');signal.addEventListener('abort',()=>{clearInterval(pending);require('node:fs').writeFileSync(${JSON.stringify(aborted)},'aborted');reject(signal.reason)},{once:true})});`);
const cancelled = spawn(process.execPath, ["--require", preload, entry], { env: { PATH: "/usr/bin:/bin" }, stdio: ["ignore", "ignore", "pipe"] });
const closed = once(cancelled, "close");
let stderr = ""; cancelled.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4096); });
const kill = setTimeout(() => cancelled.kill("SIGKILL"), 5_000);
try {
while (cancelled.exitCode === null && cancelled.signalCode === null) {
try { if (await readFile(ready, "utf8") === "ready") break; }
catch (error) { if (error.code !== "ENOENT") throw error; }
await delay(10);
}
assert.equal(await readFile(ready, "utf8"), "ready");
cancelled.kill("SIGTERM");
const [code, signal] = await closed;
assert.equal(code, 1); assert.equal(signal, null);
assert.match(stderr, /Kimi setup failed/);
assert.equal(await readFile(aborted, "utf8"), "aborted");
assert.deepEqual(await readdir(parent), []);
} finally { clearTimeout(kill); if (cancelled.exitCode === null && cancelled.signalCode === null) { cancelled.kill("SIGKILL"); await closed; } }
});
test("fresh archived installed provisioner downloads the exact closure, reuses it offline and rejects tampering", {
skip: process.env.PAPERCLIP_TEST_KIMI_PUBLIC_SETUP !== "1" && "Set PAPERCLIP_TEST_KIMI_PUBLIC_SETUP=1 for public registry downloads; no model calls or provider credentials",
timeout: 180_000,
}, async t => {
const root = await realpath(await mkdtemp(join(tmpdir(), "paperclip-kimi-public-setup-")));
t.after(() => rm(root, { recursive: true, force: true }));
const packed = join(root, "packed"), pkg = join(packed, "package"), cli = join(pkg, "dist/vendor/paperclip-runner/cli");
await mkdir(cli, { recursive: true });
await writeFile(join(pkg, "package.json"), '{"name":"@paperclipai/server","type":"module"}');
await writeFile(join(pkg, "dist/index.js"), 'throw Error("server must not start");');
await writeFile(join(cli, "acpx-runtime-sidecar.cjs"), "// installed layout fixture");
await bundleKimiProvisioner({ outputRoot: cli });
const archive = join(root, "setup.tgz"), installed = join(root, "installed");
await run("tar", ["-czf", archive, "-C", packed, "package"], { timeout: 10_000 });
await mkdir(installed);
await run("tar", ["-xzf", archive, "-C", installed], { timeout: 10_000 });
const entry = join(installed, "package/dist/vendor/paperclip-runner/cli/provision-kimi.cjs");
const api = createRequire(import.meta.url)(entry);
const environment = api.provisionEnvironment(process.env);
const fresh = await run(process.execPath, ["--use-env-proxy", entry], { env: { ...environment, KIMI_API_KEY: "sensitive-canary", NPM_TOKEN: "sensitive-canary", HOME: "/private/foreign-home" }, timeout: 120_000, maxBuffer: 1024 * 1024 });
assert.doesNotMatch(fresh.stdout + fresh.stderr, /sensitive-canary/);
const receipt = JSON.parse(fresh.stdout.trim());
assert.equal(receipt.status, "installed_verified");
const target = `${process.platform}-${process.arch}`;
const assets = join(installed, "package/dist/vendor/paperclip-runner/provider-assets/kimi", target);
const metadata = JSON.parse(await readFile(join(assets, "kimi-distribution.json"), "utf8"));
assert.equal(metadata.nativeClosureSha256, KIMI_DISTRIBUTION_CLOSURE_SHA256[target]);
const offline = { ...environment, HTTPS_PROXY: "http://127.0.0.1:9", HTTP_PROXY: "http://127.0.0.1:9", https_proxy: "http://127.0.0.1:9", http_proxy: "http://127.0.0.1:9", NO_PROXY: "", no_proxy: "" };
const reused = await run(process.execPath, ["--use-env-proxy", entry], { env: offline, timeout: 20_000, maxBuffer: 1024 * 1024 });
assert.equal(JSON.parse(reused.stdout.trim()).status, "verified_existing");
const worker = join(assets, "runtime/node_modules/@moonshot-ai/kimi-code/dist/search-worker.mjs");
const changed = Buffer.concat([await readFile(worker), Buffer.from("\n// integrity test\n")]);
await writeFile(worker, changed);
await assert.rejects(run(process.execPath, ["--use-env-proxy", entry], { env: offline, timeout: 20_000, maxBuffer: 1024 * 1024 }), error => {
assert.match(error.stderr, /Kimi setup failed/);
assert.doesNotMatch(error.stderr, /download|ECONNREFUSED/i);
return true;
});
assert.deepEqual(await readFile(worker), changed, "Invalid existing assets must not be silently replaced");
t.diagnostic(JSON.stringify({ target, closure: metadata.nativeClosureSha256, patch: metadata.patch.version, provisionerSha256: createHash("sha256").update(await readFile(entry)).digest("hex"), freshInstalled: true, offlineReuse: true, tamperRejected: true, modelPrompts: 0, scope: "self-contained public-layout setup component; full Paperclip tarball is a separate proof" }));
});