mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 20:29:18 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip Runner package has a "static checks" CI job that runs `node --test` script tests > - Two of these tests failed on unrelated pull requests on 2026-11-15 and passed on re-run > - Each failure costs a re-run of the failed jobs and delays merges, and it teaches people to re-run instead of read > - One test assumed that a copied `/bin/sleep` acts as `sleep` under any name. The other read a pid file before the child finished writing it > - This pull request removes both assumptions, and one more instance of the second pattern in the same test family > - The benefit is that these tests no longer depend on the runner image or on process scheduling ## Linked Issues or Issue Description **What happened?** 1. `scripts/stage-runner-binary.test.mjs` failed in "ci / Verify Paperclip Runner (static checks)" on PR #21765, head `98197c87e9`, job 132850068174: - `atomically publishes while the previous executable is running`: `AssertionError: 1 !== null` at line 124 (`old.child.exitCode`), after 77 ms. - `a source rebuild selects fresh dev bytes ...`: `Expected "actual" not to be strictly deep-equal` (line 192). - That job ran on a GitHub-hosted `ubuntu-26.04` image (20261111.82.6). The same tests pass on the RunsOn `ubuntu24` image, for example job 132822472729. 2. `scripts/materialize-kimi-distribution.test.mjs` › `setup cancellation physically stops an in-flight owned child` failed in the same job on PR #21758, head `40e38fc28a`, job 132822472729: `AssertionError: The real child did not start` after 149 ms. **Expected behavior** The tests pass on every supported runner image and on a loaded runner. **Steps to reproduce** 1. stage-runner-binary: in a Linux container, replace `/bin/sleep` and `/bin/echo` with one multicall binary that picks the utility from `argv[0]` and exits 1 for an unknown name (this is how uutils coreutils works on Ubuntu 26.04). Run `node --test scripts/stage-runner-binary.test.mjs`. On `master` both tests fail with the CI messages, 20 of 20 runs. 2. kimi: make the child create the pid file, wait 300 ms, then write the pid. On `master` the test fails with `The real child did not start` in about 125 ms, the same as CI. **Root cause** 1. On Ubuntu 26.04, `/bin/sleep` and `/bin/echo` are the same uutils multicall binary. The test copies it to `paperclip-runnerd` and runs it. The binary sees `argv[0] = paperclip-runnerd`, finds no utility of that name, and exits 1. The fresh-build fixture (a copy of `echo`) also has the same bytes as the old artifact (a copy of `sleep`), so the "fresh bytes" assertion fails too. 2. The child wrote the pid with `writeFileSync`, which creates the file and then writes it. The test loop broke on the first successful read. An empty read gave `Number("") === 0`, so the loop stopped and the assertion failed. The loop also had a fixed budget of 100 polls of 10 ms for a Node child to start. **Production impact** None. Only tests change. The staging code works with any executable. The Kimi setup cancellation code is correct: the deterministic repro shows that the child is stopped once the test reads a real pid. ## What Changed - `stage-runner-binary.test.mjs`: the test helper passes `argv0` to `spawn`. The copied `sleep` runs with `argv0: "sleep"`, so a multicall binary still acts as `sleep`. The fresh-build fixture is now a copy of `/bin/sh` and runs `-c "exit 0"`, so its bytes always differ from the old `sleep` artifact. - `materialize-kimi-distribution.test.mjs`: the child writes `pid.tmp` and renames it to `pid`, so a read never sees a partial file. The test waits until the pid is valid or the command settles. The 3 s command timeout and 5 s test timeout are unchanged. - `provision-kimi-package.test.mjs`: the wait for the `download-ready` marker continues until the child exits, instead of a fixed 1 s budget. The existing 5 s kill timer still bounds it. - No retries, skips or timeout changes. ## Verification - stage-runner-binary, Linux container (`node:24-bookworm-slim`) with a multicall `sleep`/`echo` built from a 15-line Go program that dispatches on `argv[0]`: - `master`: 0/20 pass, same two assertions as CI. - this branch: 50/50 pass. - this branch with stock coreutils, and on macOS: all 9 tests pass. - kimi setup cancellation: - delayed-write repro: `master` fails, this branch passes. - loop on macOS, `master` test without extra load: 59/60 (one natural failure). This branch: 60/60. - loop with 4 extra busy-loop processes (load average about 13 before them): `master` 100/100, this branch 100/100. - `node --test scripts/stage-runner-binary.test.mjs scripts/materialize-kimi-distribution.test.mjs scripts/provision-kimi-package.test.mjs`: 19 pass, 1 skipped (public download test, opt-in). - The other CI flake from the same day, `runnerd-codex-transport-orphaned-tool-at-attach.test.ts`, is a Runner defect. It is fixed in a separate pull request. ## Risks - Low risk. Only tests change. - The rebuild test now uses `/bin/sh` as the fixture for the fresh build. This file exists on every supported Linux and macOS runner. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, GitHub CLI, Docker). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
113 lines
9.4 KiB
JavaScript
113 lines
9.4 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { execFile, spawn, spawnSync } from "node:child_process";
|
|
import { createHash } from "node:crypto";
|
|
import { once } from "node:events";
|
|
import { mkdtemp, mkdir, readFile, readdir, realpath, rm, symlink, writeFile } from "node:fs/promises";
|
|
import { createRequire } from "node:module";
|
|
import { tmpdir } from "node:os";
|
|
import { dirname, join, resolve } from "node:path";
|
|
import test from "node:test";
|
|
import { setTimeout as delay } from "node:timers/promises";
|
|
import { promisify } from "node:util";
|
|
import { bundleKimiProvisioner } from "./build-verified-provider-entrypoints.mjs";
|
|
import { KIMI_DISTRIBUTION_CLOSURE_SHA256 } from "../src/drivers/acpx/kimi-closure-pins.ts";
|
|
|
|
const run = promisify(execFile);
|
|
|
|
test("installed Kimi setup is self-contained, rejects corrupt assets and cancels pending downloads", { timeout: 10_000 }, async t => {
|
|
const root = await realpath(await mkdtemp(join(tmpdir(), "paperclip-kimi-public-layout-")));
|
|
t.after(() => rm(root, { recursive: true, force: true }));
|
|
const pkg = join(root, "package"), cli = join(pkg, "dist/vendor/paperclip-runner/cli");
|
|
await mkdir(cli, { recursive: true });
|
|
await writeFile(join(pkg, "package.json"), '{"name":"@paperclipai/server","type":"module"}');
|
|
await writeFile(join(pkg, "dist/index.js"), 'throw Error("server must not start");');
|
|
await writeFile(join(cli, "acpx-runtime-sidecar.cjs"), "// installed layout fixture");
|
|
const entry = join(cli, "provision-kimi.cjs");
|
|
const packageRoot = resolve(dirname(new URL(import.meta.url).pathname), "..");
|
|
const result = await bundleKimiProvisioner({ outputRoot: cli });
|
|
for (const output of Object.values(result.metafile.outputs)) for (const dependency of output.imports) assert.match(dependency.path, /^node:/);
|
|
assert.deepEqual((await readdir(join(cli, "kimi-provision-inputs"))).sort(), ["package-lock.json", "package.json"]);
|
|
for (const name of ["package-lock.json", "package.json"]) assert.deepEqual(await readFile(join(cli, "kimi-provision-inputs", name)), await readFile(join(packageRoot, "scripts/kimi-distribution", name)));
|
|
const setup = createRequire(import.meta.url)(entry);
|
|
const layout = await setup.provisionKimiPackageRoot(entry);
|
|
assert.equal(layout.root, pkg);
|
|
assert.equal(layout.assetRoot, join(pkg, "dist/vendor/paperclip-runner"));
|
|
const linked = join(cli, "linked.cjs"); await symlink(entry, linked);
|
|
await assert.rejects(setup.provisionKimiPackageRoot(linked), /installed entrypoint/);
|
|
assert.deepEqual(setup.provisionEnvironment({ PATH: "/usr/bin:/bin", HTTPS_PROXY: "http://127.0.0.1:9", KIMI_MODEL_API_KEY: "sensitive-canary", HOME: "/private/home", NODE_OPTIONS: "--require /foreign", NODE_PATH: "/foreign", NPM_TOKEN: "sensitive-canary" }), { PATH: "/usr/bin:/bin", LANG: "C.UTF-8", HTTPS_PROXY: "http://127.0.0.1:9" });
|
|
const target = `${process.platform}-${process.arch}`;
|
|
const parent = join(layout.assetRoot, "provider-assets/kimi");
|
|
await mkdir(join(parent, target, "runtime"), { recursive: true });
|
|
const deny = join(root, "deny.cjs");
|
|
await writeFile(deny, `process.nextTick(()=>{const assert=require('node:assert/strict');for(const key of ['KIMI_MODEL_API_KEY','NPM_TOKEN','HOME','NODE_OPTIONS','NODE_PATH'])assert.equal(process.env[key],undefined,key)});const fail=()=>{throw Error('UNEXPECTED_NETWORK_OR_CHILD')};globalThis.fetch=fail;for(const m of ['node:net','node:tls','node:http','node:https']){const x=require(m);for(const key of ['connect','createConnection','request','get'])if(key in x)x[key]=fail}const c=require('node:child_process');for(const key of ['spawn','execFile','exec'])c[key]=fail;`);
|
|
const child = spawnSync(process.execPath, ["--require", deny, entry], { env: { PATH: "/usr/bin:/bin", KIMI_MODEL_API_KEY: "sensitive-canary", NPM_TOKEN: "sensitive-canary", HOME: "/private/home", NODE_OPTIONS: "", NODE_PATH: "/foreign" }, encoding: "utf8", timeout: 10_000 });
|
|
assert.ifError(child.error); assert.equal(child.status, 1); assert.match(child.stderr, /Kimi setup failed/);
|
|
assert.doesNotMatch(child.stderr + child.stdout, /UNEXPECTED_NETWORK_OR_CHILD|sensitive-canary/);
|
|
assert.deepEqual(await readdir(parent), [target]);
|
|
assert.equal(JSON.parse(await readFile(join(pkg, "package.json"), "utf8")).name, "@paperclipai/server");
|
|
await rm(join(parent, target), { recursive: true });
|
|
const ready = join(root, "download-ready"), aborted = join(root, "download-aborted");
|
|
const preload = join(root, "pending-download.cjs");
|
|
await writeFile(preload, `globalThis.fetch=(_url,{signal})=>new Promise((_resolve,reject)=>{const pending=setInterval(()=>{},1000);require('node:fs').writeFileSync(${JSON.stringify(ready)},'ready');signal.addEventListener('abort',()=>{clearInterval(pending);require('node:fs').writeFileSync(${JSON.stringify(aborted)},'aborted');reject(signal.reason)},{once:true})});`);
|
|
const cancelled = spawn(process.execPath, ["--require", preload, entry], { env: { PATH: "/usr/bin:/bin" }, stdio: ["ignore", "ignore", "pipe"] });
|
|
const closed = once(cancelled, "close");
|
|
let stderr = ""; cancelled.stderr.on("data", chunk => { stderr = (stderr + chunk).slice(-4096); });
|
|
const kill = setTimeout(() => cancelled.kill("SIGKILL"), 5_000);
|
|
try {
|
|
while (cancelled.exitCode === null && cancelled.signalCode === null) {
|
|
try { if (await readFile(ready, "utf8") === "ready") break; }
|
|
catch (error) { if (error.code !== "ENOENT") throw error; }
|
|
await delay(10);
|
|
}
|
|
assert.equal(await readFile(ready, "utf8"), "ready");
|
|
cancelled.kill("SIGTERM");
|
|
const [code, signal] = await closed;
|
|
assert.equal(code, 1); assert.equal(signal, null);
|
|
assert.match(stderr, /Kimi setup failed/);
|
|
assert.equal(await readFile(aborted, "utf8"), "aborted");
|
|
assert.deepEqual(await readdir(parent), []);
|
|
} finally { clearTimeout(kill); if (cancelled.exitCode === null && cancelled.signalCode === null) { cancelled.kill("SIGKILL"); await closed; } }
|
|
});
|
|
|
|
test("fresh archived installed provisioner downloads the exact closure, reuses it offline and rejects tampering", {
|
|
skip: process.env.PAPERCLIP_TEST_KIMI_PUBLIC_SETUP !== "1" && "Set PAPERCLIP_TEST_KIMI_PUBLIC_SETUP=1 for public registry downloads; no model calls or provider credentials",
|
|
timeout: 180_000,
|
|
}, async t => {
|
|
const root = await realpath(await mkdtemp(join(tmpdir(), "paperclip-kimi-public-setup-")));
|
|
t.after(() => rm(root, { recursive: true, force: true }));
|
|
const packed = join(root, "packed"), pkg = join(packed, "package"), cli = join(pkg, "dist/vendor/paperclip-runner/cli");
|
|
await mkdir(cli, { recursive: true });
|
|
await writeFile(join(pkg, "package.json"), '{"name":"@paperclipai/server","type":"module"}');
|
|
await writeFile(join(pkg, "dist/index.js"), 'throw Error("server must not start");');
|
|
await writeFile(join(cli, "acpx-runtime-sidecar.cjs"), "// installed layout fixture");
|
|
await bundleKimiProvisioner({ outputRoot: cli });
|
|
const archive = join(root, "setup.tgz"), installed = join(root, "installed");
|
|
await run("tar", ["-czf", archive, "-C", packed, "package"], { timeout: 10_000 });
|
|
await mkdir(installed);
|
|
await run("tar", ["-xzf", archive, "-C", installed], { timeout: 10_000 });
|
|
const entry = join(installed, "package/dist/vendor/paperclip-runner/cli/provision-kimi.cjs");
|
|
const api = createRequire(import.meta.url)(entry);
|
|
const environment = api.provisionEnvironment(process.env);
|
|
const fresh = await run(process.execPath, ["--use-env-proxy", entry], { env: { ...environment, KIMI_API_KEY: "sensitive-canary", NPM_TOKEN: "sensitive-canary", HOME: "/private/foreign-home" }, timeout: 120_000, maxBuffer: 1024 * 1024 });
|
|
assert.doesNotMatch(fresh.stdout + fresh.stderr, /sensitive-canary/);
|
|
const receipt = JSON.parse(fresh.stdout.trim());
|
|
assert.equal(receipt.status, "installed_verified");
|
|
const target = `${process.platform}-${process.arch}`;
|
|
const assets = join(installed, "package/dist/vendor/paperclip-runner/provider-assets/kimi", target);
|
|
const metadata = JSON.parse(await readFile(join(assets, "kimi-distribution.json"), "utf8"));
|
|
assert.equal(metadata.nativeClosureSha256, KIMI_DISTRIBUTION_CLOSURE_SHA256[target]);
|
|
const offline = { ...environment, HTTPS_PROXY: "http://127.0.0.1:9", HTTP_PROXY: "http://127.0.0.1:9", https_proxy: "http://127.0.0.1:9", http_proxy: "http://127.0.0.1:9", NO_PROXY: "", no_proxy: "" };
|
|
const reused = await run(process.execPath, ["--use-env-proxy", entry], { env: offline, timeout: 20_000, maxBuffer: 1024 * 1024 });
|
|
assert.equal(JSON.parse(reused.stdout.trim()).status, "verified_existing");
|
|
const worker = join(assets, "runtime/node_modules/@moonshot-ai/kimi-code/dist/search-worker.mjs");
|
|
const changed = Buffer.concat([await readFile(worker), Buffer.from("\n// integrity test\n")]);
|
|
await writeFile(worker, changed);
|
|
await assert.rejects(run(process.execPath, ["--use-env-proxy", entry], { env: offline, timeout: 20_000, maxBuffer: 1024 * 1024 }), error => {
|
|
assert.match(error.stderr, /Kimi setup failed/);
|
|
assert.doesNotMatch(error.stderr, /download|ECONNREFUSED/i);
|
|
return true;
|
|
});
|
|
assert.deepEqual(await readFile(worker), changed, "Invalid existing assets must not be silently replaced");
|
|
t.diagnostic(JSON.stringify({ target, closure: metadata.nativeClosureSha256, patch: metadata.patch.version, provisionerSha256: createHash("sha256").update(await readFile(entry)).digest("hex"), freshInstalled: true, offlineReuse: true, tamperRejected: true, modelPrompts: 0, scope: "self-contained public-layout setup component; full Paperclip tarball is a separate proof" }));
|
|
});
|