mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 17:33:51 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `Build` check. Its last step is the public npm install verifier, `scripts/verify-grok-npm-install.mjs`, which stages the eighteen public packages reachable from the server and the CLI, packs them, and installs them in a clean consumer. > - Before it stages anything, the verifier runs `scripts/build-standalone-public-packages.mjs`. That script installs and compiles the seven sandbox-provider plugins that live outside the pnpm workspace: one network `pnpm install --ignore-workspace` and one `tsc` per plugin. The line was copied from the build block of `release.sh` when the verifier was written in #14024. > - release.sh needs that build because it publishes those plugins. The verifier does not for anything it stages. Its package closure follows `workspace:` specs from the server and the CLI, and a `workspace:` spec can only name a workspace package, so none of the eighteen packages it packs is a standalone package and no tarball reads that build's output. > - One optional path at the end of the verifier does read it: qualification-package retention, which `docker-runner-check.yml` enables with `PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR` after running only `pnpm build`, packs the Daytona plugin from its `dist`. The PR `Build` job never sets that variable. > - The timing line from #15868 and #15888 puts the verifier's staging phase at 75.8s on the latest run, of which the staged sources, the bundled installs and the packs account for 46.5s. The rest, about 29s, is the work before staging, and the standalone build is almost all of it. `Build` at 433-530s is the slowest check in most PR runs once the server shards drop, so that time sits on the run's critical path. > - This pull request runs the standalone build only when retention is requested, keeps the `ui-dist` copy the server package needs, and times both on the timing line so what is left before staging is visible. > - The benefit is a `Build` job about 21-28s shorter, with no change to what the verifier installs or checks, no loss of coverage for the standalone build because Canary Dry Run still runs it through `release.sh`, and no change to the retention path that reads it. ## Linked Issues or Issue Description Replaces #15892, which GitHub closed automatically when the branch of #15888 (its base at the time) was deleted on merge. Same two commits, rebased onto `master` on top of #15888 (1ac48aff), #15868 (026bdcd0) and #15861 (85f4205d), which are all merged. #15892 reached Greptile 5/5 with every check green on the same content; the review history is there. **What existing behavior does this improve?** The `Build` PR check. Its public npm install verifier builds the seven standalone sandbox-provider plugins before it stages the eighteen packages it installs, and none of those packages uses that build. **Subsystem affected** CI and release scripts (`scripts/verify-grok-npm-install.mjs`, `scripts/build-standalone-public-packages.mjs`). **Current behavior** The verifier runs `node scripts/build-standalone-public-packages.mjs` and then `bash scripts/prepare-server-ui-dist.sh` before it stages any package, on every run. The first script removes each sandbox-provider plugin's `node_modules`, runs `pnpm install --ignore-workspace` against the registry, links the in-repo plugin SDK, and runs the plugin's `tsc` build, four plugins at a time. On the last `Build` run before this change ([38146108750](https://github.com/paperclipai/paperclip/actions/runs/38146108750)) `stage_packages_ms` was 75.8s and the staged sources, bundled installs and packs inside it took 0.9s, 15.7s and 29.9s, which leaves about 29s before staging. The verifier's package closure, computed in the same way as the script computes it, contains eighteen workspace packages and no sandbox-provider plugin. **Proposed behavior** Run the standalone build only when `PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR` is set, because that is the one path that reads it, and report it as `stage_standalone_build_ms` when it runs. Always run the `ui-dist` copy before staging. With `PAPERCLIP_RELEASE_REUSE_UI_DIST=1`, which the verifier already sets, that copies the `ui/dist` that `pnpm build` produced into `server/ui-dist`, which the server package publishes. Report that copy as `stage_ui_dist_ms` on the timing line. Leave `release.sh` unchanged, so the release flow and Canary Dry Run still build the standalone packages where they are published. **Reason and benefit** The verifier proves that the eighteen public packages install cleanly from the public registry. The standalone build adds seven network installs and compiles whose output is never packed. Removing it from the PR path shortens the staging phase by about 21-28s. Measured on this PR's first commit ([38150787810](https://github.com/paperclipai/paperclip/actions/runs/38150787810)): `stage_packages_ms` 54.2s against 75.8s, with `stage_ui_dist_ms` 21ms, and the time before staging, `stage_packages_ms` minus the four staging keys, down from about 29s to under 0.1s. **Breaking changes** None. The same eighteen packages are staged, packed, checked and installed from the same sources. The retention path builds exactly what it built before. `scripts/build-standalone-public-packages.mjs` exports two existing helpers for the new guard test and otherwise does not change. ## What Changed - `scripts/verify-grok-npm-install.mjs`: the `build-standalone-public-packages.mjs` call before staging runs only when `PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR` is set, in the same position as before, and reports `stage_standalone_build_ms`. The flag is read once into `retainQualificationPackages`, and the retention call at the end is gated by the same variable, so the build and the retention that reads it cannot diverge. The `prepare-server-ui-dist.sh` call stays, is timed, and reports `stage_ui_dist_ms`. The comment explains why release.sh and the retention path need the standalone build and the staged packages do not. The timing-line comment lists both new keys. - `scripts/build-standalone-public-packages.mjs`: `parseWorkspaceEntries` and `isWorkspacePackage` are exported. No behavior change. - `scripts/__tests__/grok-public-install-sandbox.test.mjs`: one new guard test. It asserts the verifier references the standalone build exactly once, inside the retention gate, timed under its own key, and that no second reading of the retention variable exists; that `retain-runner-qualification-packages.mjs` still packs the Daytona plugin from its `dist`, and that the Daytona plugin is a standalone package, so the gate cannot pass vacuously; recomputes the verifier's package closure from `release-package-map.mjs list` and the manifests through `workspace:` specs; checks that the public package map still carries packages outside the pnpm workspace; checks that every package in the closure is inside the workspace globs of `pnpm-workspace.yaml`; checks that the server package still publishes `ui-dist`; and checks the order: closure, gated standalone build, `ui-dist` copy, then staging, with the reuse flag set and the timing keys written. ### Review fix, second commit Greptile's P1 on the first commit was valid: `docker-runner-check.yml` runs `pnpm build` and then the verifier with `PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR` set, and `retainRunnerQualificationPackages` copies `packages/plugins/sandbox-providers/daytona/dist`. The Daytona plugin is excluded from the pnpm workspace, so `pnpm build` never produces that directory and the first commit would have failed that job with `ENOENT` after every install check had passed. The second commit restores the build on exactly that path: the verifier builds the standalone packages when, and only when, retention is requested, and the guard test pins the gate to the reason for it. ## Verification - `node --test scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs`: 37 tests pass. - The updated guard test fails against the first commit's verifier (no retention gate around the build) and passes with the second commit. The first commit's test failed against #15888's verifier the same way. - `node --check` passes on the verifier. - The verifier itself needs Docker and a network, which this workspace does not have. The first commit's `Build` job ([38150787810](https://github.com/paperclipai/paperclip/actions/runs/38150787810)) ran the branch's verifier and measured the saving: `stage_packages_ms` 54.2s (was 75.8s), `stage_ui_dist_ms` 21ms, `stage_sources_ms` 0.8s, `stage_bundled_ms` 21.0s, `stage_pack_ms` 32.4s, so under 0.1s remains before staging where about 29s was. The second commit does not change the PR path: the gate is false on every PR run, so that job's timing line should show the same shape with no `stage_standalone_build_ms` key. - The retention path was not run here: it needs Docker and the `docker-runner-check` workflow's inputs. The code on that path is the same `run(process.execPath, [...build-standalone-public-packages.mjs])` call the verifier made before this PR, in the same position. - `scripts/__tests__/release-dry-run-notes.test.mjs` exercises the standalone build inside `release.sh` and is unchanged; 11 of its 12 tests spawn `pnpm`, which is not installed in this workspace, and fail the same way on #15888's head. They pass in CI, where `pnpm` is installed. - The first commit's run failed only on `General tests (server (11/12))`: all 74 files and 1257 tests passed, then Vitest caught a teardown `EACCES` removing the per-worker `PAPERCLIP_HOME` under `native-session-resumption.test.ts`, the same error as the first attempt of #15888's run 38146108750. Neither PR touches server code; the failure is noted on the tracking issue. ## Risks - If a future change makes a package the verifier installs depend on a standalone package, the verifier would stage an unbuilt `dist`. That cannot happen through a `workspace:` spec, which is the only kind the closure follows, and the guard test fails if the closure ever contains a package outside the workspace globs. - If a future path in the verifier reads a standalone build without setting the retention variable, it would see no `dist`. The guard test pins the single reference to the gate; a new reader has to extend the gate and the test together. - The sandbox-provider plugins' builds are no longer exercised on every PR by the `Build` job. They are still built by `release.sh` in Canary Dry Run on every PR, by the verifier whenever retention is requested, and the `build-standalone-concurrency` and `release-dry-run-notes` tests still cover the script. - Low risk otherwise. The bundled installs, the packs, the consumer install, and every probe are unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.1 (Anthropic), model ID `claude-fable-5-1`, run as a Paperclip agent through the Claude Code local adapter with tool use (shell, file edits, GitHub CLI). Reasoning mode: extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (CI is running on the second commit) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (the P1 on the first commit is fixed in the second; re-review pending) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Bender (Fable) <bender-fable@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
344 lines
29 KiB
JavaScript
344 lines
29 KiB
JavaScript
import assert from 'node:assert/strict';
|
|
import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';
|
|
import { spawnSync } from 'node:child_process';
|
|
import { tmpdir } from 'node:os';
|
|
import { join } from 'node:path';
|
|
import test from 'node:test';
|
|
import { fileURLToPath } from 'node:url';
|
|
import { isWorkspacePackage, parseWorkspaceEntries } from '../build-standalone-public-packages.mjs';
|
|
import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, assertInstalledCodexPlatformProvenance, assertNoBundledCodexPayloads, grokConsumerDockerArgs, installedCodexProbeSource } from '../grok-public-install-sandbox.mjs';
|
|
|
|
const paths = { assets: '/private/staging/assets', consumer: '/private/staging/consumer', cache: '/private/staging/cache', uid: 1001, gid: 1001 };
|
|
const values = (args, flag) => args.flatMap((value, index) => value === flag ? [args[index + 1]] : []);
|
|
|
|
test('lifecycle execution has no network, host credentials, checkout, or elevated privileges', () => {
|
|
const args = grokConsumerDockerArgs({ ...paths, command: GROK_PUBLIC_INSTALL_LIFECYCLE });
|
|
assert.deepEqual(values(args, '--network'), ['none']);
|
|
assert.deepEqual(values(args, '--user'), ['1001:1001']);
|
|
assert.ok(args.includes('--read-only'));
|
|
assert.deepEqual(values(args, '--cap-drop'), ['ALL']);
|
|
assert.deepEqual(values(args, '--security-opt'), ['no-new-privileges']);
|
|
assert.deepEqual(values(args, '--mount'), [
|
|
'type=bind,src=/private/staging/assets,dst=/packages,readonly',
|
|
'type=bind,src=/private/staging/consumer,dst=/consumer',
|
|
'type=bind,src=/private/staging/cache,dst=/cache',
|
|
]);
|
|
assert.deepEqual(values(args, '--env'), ['HOME=/tmp', 'npm_config_cache=/cache', 'npm_config_nodedir=/usr/local', 'npm_config_audit=false', 'npm_config_fund=false', 'npm_config_ignore_scripts=false']);
|
|
assert.match(GROK_PUBLIC_INSTALL_IMAGE, /@sha256:[a-f0-9]{64}$/);
|
|
});
|
|
|
|
test('deferred lifecycle execution rebuilds the installed graph without dependency resolution', () => {
|
|
assert.deepEqual(GROK_PUBLIC_INSTALL_LIFECYCLE, ['npm', 'rebuild', '--offline', '--ignore-scripts=false', '--dangerously-allow-all-scripts']);
|
|
});
|
|
|
|
test('a root or malformed host identity cannot run lifecycle scripts', () => {
|
|
for (const uid of [0, -1, undefined, '1001']) {
|
|
assert.throws(() => grokConsumerDockerArgs({ ...paths, uid, command: ['npm', 'ci'] }), /unprivileged/);
|
|
}
|
|
});
|
|
|
|
test('only the scripts-disabled dependency download gets network access', () => {
|
|
const args = grokConsumerDockerArgs({ ...paths, download: true, command: ['npm', 'install', '--ignore-scripts'] });
|
|
assert.deepEqual(values(args, '--network'), ['bridge']);
|
|
assert.ok(values(args, '--env').includes('npm_config_ignore_scripts=true'));
|
|
});
|
|
|
|
test('Pi assembly gets bounded scratch capacity while preserving sandbox restrictions', () => {
|
|
const args = grokConsumerDockerArgs({ ...paths, download: true, temporarySizeMiB: 2048, command: ['node', '/consumer/node_modules/paperclipai/dist/index.js', 'runtime', 'setup', 'pi'] });
|
|
assert.deepEqual(values(args, '--tmpfs'), ['/tmp:rw,nosuid,nodev,noexec,size=2048m,mode=1777']);
|
|
assert.deepEqual(values(args, '--memory'), ['3g']);
|
|
assert.ok(args.includes('--read-only'));
|
|
assert.deepEqual(values(args, '--cap-drop'), ['ALL']);
|
|
assert.deepEqual(values(args, '--security-opt'), ['no-new-privileges']);
|
|
for (const temporarySizeMiB of [0, 255, 2049, Infinity, '2048']) {
|
|
assert.throws(() => grokConsumerDockerArgs({ ...paths, temporarySizeMiB, command: ['node'] }), /bounded/);
|
|
}
|
|
});
|
|
|
|
test('only an offline runtime probe can execute its verified scratch snapshot', () => {
|
|
const command = ['node', '/packages/pi-public-install-probe.mjs', '/consumer/node_modules/@paperclipai/server'];
|
|
const args = grokConsumerDockerArgs({ ...paths, command, temporarySizeMiB: 2048, temporaryExecutable: true });
|
|
assert.deepEqual(values(args, '--tmpfs'), ['/tmp:rw,nosuid,nodev,exec,size=2048m,mode=1777']);
|
|
assert.deepEqual(values(args, '--network'), ['none']);
|
|
assert.deepEqual(values(args, '--user'), ['1001:1001']);
|
|
assert.ok(args.includes('--read-only'));
|
|
assert.deepEqual(values(args, '--cap-drop'), ['ALL']);
|
|
assert.deepEqual(values(args, '--security-opt'), ['no-new-privileges']);
|
|
assert.throws(() => grokConsumerDockerArgs({ ...paths, command, download: true, temporaryExecutable: true }), /offline/);
|
|
assert.throws(() => grokConsumerDockerArgs({ ...paths, command, temporaryExecutable: 'true' }), /offline/);
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
assert.ok(source.includes("temporarySizeMiB: 2048, temporaryExecutable: true"));
|
|
});
|
|
|
|
test('the separately provisioned executable is exposed read-only to the offline probe', () => {
|
|
const prerequisite = '/private/staging/native/grok';
|
|
const args = grokConsumerDockerArgs({ ...paths, prerequisite, command: ['node', '/packages/probe.mjs', 'present'] });
|
|
assert.deepEqual(values(args, '--network'), ['none']);
|
|
assert.equal(values(args, '--mount').at(-1), `type=bind,src=${prerequisite},dst=/opt/paperclip/providers/grok/1.0.13/grok,readonly`);
|
|
});
|
|
|
|
test('verification never elevates PR-controlled provisioning or cleanup on the host', () => {
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
assert.doesNotMatch(source, /\bsudo\b/);
|
|
assert.ok(source.includes("const prerequisite = join(root, 'native/grok')"));
|
|
});
|
|
|
|
test('the image and Grok downloads start before staging and join without weakening the npm checks', () => {
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
const position = fragment => { const index = source.indexOf(fragment); assert.notEqual(index, -1, `verifier must contain ${fragment}`); return index; };
|
|
// Exactly the pinned consumer image is pulled, for the platform the probes run on.
|
|
const imagePull = position("prefetch('image', 'docker', ['pull', '--platform', 'linux/amd64', GROK_PUBLIC_INSTALL_IMAGE])");
|
|
// Grok is provisioned into the prefetch directory, never straight into `native/grok`.
|
|
const grokProvision = position("prefetch('grok', process.execPath, [join(repo, 'packages/paperclip-runner/scripts/provision-grok.mjs'), stagedGrok])");
|
|
assert.ok(source.includes("const stagedGrok = join(prefetched, 'grok')"));
|
|
assert.doesNotMatch(source, /provision-grok\.mjs'\), prerequisite\]/);
|
|
// Both downloads start before any package is staged and before the first sandbox runs.
|
|
const staging = position("visit('@paperclipai/server')");
|
|
const firstSandbox = position("isolated(['npm', 'install', '--ignore-scripts', '--omit=dev'");
|
|
assert.ok(imagePull < staging && grokProvision < staging);
|
|
assert.ok(position("await awaited('image', imageReady)") < firstSandbox);
|
|
// The binary moves to the canonical host path only after npm is proven not to
|
|
// have provisioned it, after the negative probe, and before the positive probe.
|
|
const npmCheck = position("'npm must not provision Grok'");
|
|
const negativeProbe = position("isolated(['node', '/packages/probe.mjs', 'missing'])");
|
|
const grokJoin = position("await awaited('grok', grokReady)");
|
|
const move = position('renameSync(stagedGrok, prerequisite)');
|
|
const positiveProbe = position("isolated(['node', '/packages/probe.mjs', 'present'], { prerequisite })");
|
|
assert.ok(npmCheck < negativeProbe && negativeProbe < grokJoin && grokJoin < move && move < positiveProbe);
|
|
// The children run through the helper covered by prefetch-child.test.mjs, and
|
|
// cleanup waits for them before removing the directory they write into.
|
|
assert.ok(source.includes("import { startPrefetch, stopPrefetches } from './prefetch-child.mjs'"));
|
|
assert.ok(position('await stopPrefetches(prefetches)') < position('rmSync(root, { recursive: true, force: true })'));
|
|
assert.equal(source.match(/rmSync\(root/g).length, 1);
|
|
});
|
|
|
|
test('the bundled-dependency installs run together while the other packages are copied, and pack in order', () => {
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
const position = fragment => { const index = source.indexOf(fragment); assert.notEqual(index, -1, `verifier must contain ${fragment}`); return index; };
|
|
// The installs are children of the script's own CLI entry through the helper
|
|
// covered by stage-bundled-packages.test.mjs, never in-process calls.
|
|
assert.doesNotMatch(source, /prepareBundledPackage/);
|
|
assert.ok(source.includes("import { bundledInstallTimings, startBundledInstalls } from './stage-bundled-packages.mjs'"));
|
|
// Only the bundled packages' sources are staged before the installs start;
|
|
// the installs are registered for cleanup at once, before anything else can
|
|
// fail; the other packages are copied while the installs run; the join comes
|
|
// before the packs start; and the packs run through the helper covered by
|
|
// pack-staged-packages.test.mjs, bundled packages first, each registered for
|
|
// cleanup as it starts, with the results put back into dependency order
|
|
// before every tarball is checked.
|
|
const bundledSources = position('for (const entry of entries.filter(entry => entry.bundled)) stageSource(entry)');
|
|
const start = position("startBundledInstalls(entries.filter(entry => entry.bundled), { logDir: prefetched, cwd: repo })");
|
|
const registered = position('prefetches.push(...installs.map(({ handle }) => handle))');
|
|
const otherSources = position('for (const entry of entries.filter(entry => !entry.bundled))');
|
|
const join = position("await awaited('bundled', Promise.all(installs.map(({ handle }) => handle.done)))");
|
|
assert.ok(source.includes("import { packConcurrency, packStagedPackages, packTimings } from './pack-staged-packages.mjs'"));
|
|
const packs = position("await packStagedPackages([...entries].sort((a, b) => Number(b.bundled) - Number(a.bundled)), { logDir: prefetched, env, concurrency, register: handle => prefetches.push(handle) })");
|
|
const reordered = position('packs.sort((a, b) => a.index - b.index)');
|
|
const checks = position("for (const tarball of tarballs) assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball])");
|
|
const copied = position('for (const tarball of tarballs) {');
|
|
assert.ok(bundledSources < start && start < registered && registered < otherSources && otherSources < join && join < packs && packs < reordered && reordered < checks && checks < copied);
|
|
assert.equal(source.match(/run\('npm', \['pack'/g).length, 1, 'only the lifecycle sentinel is packed in-process');
|
|
assert.ok(source.includes("destination: join(root, `pack-${index}`)"), 'each pack writes into its own directory');
|
|
// The timing line reports the split staging phase, each install, and the
|
|
// pack step with what the packs would have cost one after another.
|
|
for (const key of ['stage_ui_dist_ms', 'stage_sources_ms', 'stage_bundled_ms', 'stage_pack_ms', 'stage_pack_check_ms']) assert.ok(source.includes(`timing.${key} =`));
|
|
assert.ok(source.includes('...bundledInstallTimings(installs)'));
|
|
assert.ok(source.includes('Object.assign(timing, packTimings(packs, concurrency))'));
|
|
});
|
|
|
|
test('the verifier stages only workspace packages, so it skips the standalone build release.sh needs', () => {
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
const position = fragment => { const index = source.indexOf(fragment); assert.notEqual(index, -1, `verifier must contain ${fragment}`); return index; };
|
|
// The standalone public packages are the ones outside the pnpm workspace
|
|
// (the sandbox-provider plugins). release.sh builds them because it publishes
|
|
// them; the verifier packs only the closure of the server and the CLI under
|
|
// `workspace:` specs, and a `workspace:` spec can only name a workspace
|
|
// package, so no tarball here ever contains or depends on a standalone build.
|
|
// The one reader of that build is qualification-package retention, which
|
|
// packs the Daytona plugin from its `dist`, so the build runs exactly when
|
|
// retention is requested: the single reference sits inside that gate, and
|
|
// the same flag is what later decides whether retention runs at all.
|
|
const standaloneBuild = "run(process.execPath, [join(repo, 'scripts/build-standalone-public-packages.mjs')], repo)";
|
|
assert.equal(source.split('build-standalone-public-packages').length - 1, 1, 'the standalone build is referenced once, inside the retention gate');
|
|
const gate = position("const retainQualificationPackages = Boolean(process.env.PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR);\n if (retainQualificationPackages) {\n const standaloneStarted = performance.now();\n " + standaloneBuild);
|
|
assert.ok(source.includes('timing.stage_standalone_build_ms ='), 'the gated build is timed under its own key');
|
|
assert.ok(source.includes('if (retainQualificationPackages) {\n const retained = retainRunnerQualificationPackages({'), 'retention is gated by the same flag as the build');
|
|
assert.doesNotMatch(source, /if \(process\.env\.PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR\)/, 'no second reading of the retention flag can diverge from the build gate');
|
|
const retention = readFileSync(new URL('../retain-runner-qualification-packages.mjs', import.meta.url), 'utf8');
|
|
assert.ok(retention.includes("const pluginRelative = 'packages/plugins/sandbox-providers/daytona';") && retention.includes("cpSync(join(repo, pluginRelative, 'dist'), join(plugin, 'dist'), { recursive: true })"), 'retention still packs the Daytona plugin from its dist, which is why the gated build exists');
|
|
const repo = new URL('../../', import.meta.url);
|
|
const listing = spawnSync(process.execPath, [fileURLToPath(new URL('scripts/release-package-map.mjs', repo)), 'list'], { cwd: fileURLToPath(repo), encoding: 'utf8' });
|
|
assert.equal(listing.status, 0, listing.stderr);
|
|
const packages = new Map(listing.stdout.trim().split('\n').map(line => line.split('\t')).map(([dir, name]) => [name, { dir, manifest: JSON.parse(readFileSync(new URL(`${dir}/package.json`, repo), 'utf8')) }]));
|
|
const needed = new Set();
|
|
const visit = name => {
|
|
if (needed.has(name)) return;
|
|
const entry = packages.get(name); assert.ok(entry, `missing public workspace dependency ${name}`); needed.add(name);
|
|
for (const [dep, spec] of Object.entries({ ...entry.manifest.dependencies, ...entry.manifest.optionalDependencies })) if (spec.startsWith('workspace:')) visit(dep);
|
|
};
|
|
assert.ok(source.includes("visit('@paperclipai/server');\n visit('paperclipai');"));
|
|
visit('@paperclipai/server'); visit('paperclipai');
|
|
const workspace = parseWorkspaceEntries(readFileSync(new URL('pnpm-workspace.yaml', repo), 'utf8'));
|
|
const standalone = [...packages.values()].filter(({ dir }) => !isWorkspacePackage(dir, workspace)).map(({ dir }) => dir);
|
|
assert.ok(standalone.length > 0, 'the public package map must still carry standalone packages for this check to mean anything');
|
|
assert.ok(standalone.includes('packages/plugins/sandbox-providers/daytona'), 'the Daytona plugin retention packs is a standalone package, so pnpm build alone cannot produce its dist');
|
|
for (const name of needed) assert.ok(isWorkspacePackage(packages.get(name).dir, workspace), `${name} is staged by the verifier but is not a workspace package`);
|
|
// The server publishes `ui-dist`, so that copy stays, before any package is
|
|
// staged, reusing the UI that `pnpm build` produced, and it is timed. The
|
|
// gated standalone build keeps release.sh's order: after the closure is
|
|
// known and before the UI copy and any staged source.
|
|
assert.deepEqual(JSON.parse(readFileSync(new URL('server/package.json', repo), 'utf8')).files, ['dist', 'ui-dist', 'skills']);
|
|
assert.ok(source.includes("PAPERCLIP_RELEASE_REUSE_UI_DIST: '1'"));
|
|
const uiDist = position("run('bash', [join(repo, 'scripts/prepare-server-ui-dist.sh')], repo)");
|
|
assert.ok(position("visit('paperclipai')") < gate && gate < uiDist && uiDist < position('timing.stage_ui_dist_ms =') && uiDist < position('const stageSource = '));
|
|
});
|
|
|
|
test('the installed Codex probe exercises the public export and rejects incomplete or mismatched packages', () => {
|
|
const root = realpathSync(mkdtempSync(join(tmpdir(), 'installed-codex-probe-')));
|
|
try {
|
|
const consumer = join(root, 'consumer'); mkdirSync(consumer);
|
|
const target = `${process.platform}-${process.arch}`;
|
|
const platformPath = `node_modules/@openai/codex-${target}`;
|
|
const platform = join(consumer, platformPath); mkdirSync(platform, { recursive: true });
|
|
const nativeVendor = join(platform, 'vendor'); mkdirSync(nativeVendor);
|
|
const manifest = { name: '@openai/codex', version: `0.160.0-${target}`, os: [process.platform], cpu: [process.arch] };
|
|
writeFileSync(join(platform, 'package.json'), JSON.stringify(manifest));
|
|
const entry = { name: manifest.name, version: manifest.version,
|
|
resolved: `https://registry.npmjs.org/@openai/codex/-/codex-${manifest.version}.tgz`, integrity: `sha512-${Buffer.alloc(64, 1).toString('base64')}` };
|
|
const lock = { lockfileVersion: 3, packages: { [platformPath]: entry } };
|
|
writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock));
|
|
const server = join(consumer, 'node_modules/@paperclipai/server');
|
|
const vendor = join(server, 'dist/vendor/paperclip-runner'), drivers = join(vendor, 'drivers/acpx'); mkdirSync(drivers, { recursive: true });
|
|
writeFileSync(join(server, 'package.json'), JSON.stringify({ name: '@paperclipai/server', type: 'module' }));
|
|
const integrity = join(drivers, 'installation-integrity.js');
|
|
const security = join(vendor, 'drivers/codex/codex-security-config.js'); mkdirSync(join(vendor, 'drivers/codex'));
|
|
const securitySource = `export const codexExecutableReadOnlyRoots = () => [${JSON.stringify(nativeVendor)}];
|
|
export const createIsolatedCodexAppServerArgs = (source, roots) => roots.map(root => JSON.stringify(root) + '=\"read\"');`;
|
|
writeFileSync(security, securitySource);
|
|
const moduleSource = `import assert from 'node:assert/strict'; import { writeFileSync } from 'node:fs';
|
|
export function createAcpxPackageJsonResolver(root, manifest) { assert.equal(root, ${JSON.stringify(server)}); assert.equal(manifest, root + '/package.json'); return () => 'selected-authority'; }
|
|
export async function verifyQualifiedAcpxInstallation(profile, resolver) { assert.deepEqual(profile, { agent: 'codex', model: 'gpt-5.4' }); assert.equal(resolver(), 'selected-authority');
|
|
return { openCommand: async () => ({ close: async () => { writeFileSync(${JSON.stringify(join(server, 'lease-closed'))}, 'ok'); } }) }; }`;
|
|
writeFileSync(integrity, moduleSource);
|
|
writeFileSync(join(drivers, 'qualified-profiles.js'), "export const resolveQualifiedAcpxProfile = (agent, model) => ({ agent, model });");
|
|
const wrapper = join(consumer, 'node_modules/@openai/codex/bin'); mkdirSync(wrapper, { recursive: true });
|
|
writeFileSync(join(wrapper, '../package.json'), JSON.stringify({ name: '@openai/codex', version: '0.160.0' }));
|
|
const index = join(vendor, 'index.mjs'), command = join(wrapper, 'codex.js'), probe = join(root, 'probe.mjs');
|
|
const executable = version => writeFileSync(command, `#!/bin/sh\n[ "$1" = --version ] || exit 65\nprintf '%s\\n' 'codex-cli ${version}'\n`, { mode: 0o755 });
|
|
writeFileSync(probe, installedCodexProbeSource(index, consumer, '0.160.0'));
|
|
const run = () => spawnSync(process.execPath, [probe], { encoding: 'utf8', timeout: 10_000, env: { PATH: '/usr/bin:/bin', NODE_PATH: '' } });
|
|
writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(command)};`);
|
|
executable('0.160.0');
|
|
let result = run(); assert.equal(result.status, 0, result.stderr);
|
|
assert.equal(JSON.parse(result.stdout).pinnedCodexCommandVerified, true);
|
|
assert.equal(JSON.parse(result.stdout).codexPlatformPackageSource, 'official npm');
|
|
assert.equal(JSON.parse(result.stdout).codexConsumerPlatform, target);
|
|
assert.equal(JSON.parse(result.stdout).codexQualifiedInstallationVerified, true);
|
|
assert.equal(JSON.parse(result.stdout).codexCommandLeaseVerified, true);
|
|
assert.equal(JSON.parse(result.stdout).codexNativeSandboxResourcesVerified, true);
|
|
assert.equal(readFileSync(join(server, 'lease-closed'), 'utf8'), 'ok');
|
|
assert.equal(JSON.parse(result.stdout).providerCalls, 0);
|
|
const legacyPath = 'node_modules/legacy-codex-adapter/node_modules/@openai/codex-' + target;
|
|
const legacy = join(consumer, legacyPath); mkdirSync(legacy, { recursive: true });
|
|
const legacyManifest = { ...manifest, version: '0.156.1-' + target };
|
|
writeFileSync(join(legacy, 'package.json'), JSON.stringify(legacyManifest));
|
|
lock.packages[legacyPath] = { ...entry, version: legacyManifest.version,
|
|
resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-' + legacyManifest.version + '.tgz' };
|
|
writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock));
|
|
result = run(); assert.equal(result.status, 0, result.stderr);
|
|
assert.equal(JSON.parse(result.stdout).codexPlatformPackages.length, 2);
|
|
assert.equal(JSON.parse(result.stdout).codexSelectedPlatformPath, platformPath);
|
|
assert.equal(JSON.parse(result.stdout).codexSelectedPlatformVersion, '0.160.0-' + target);
|
|
const legacyCommand = join(legacy, 'codex');
|
|
writeFileSync(legacyCommand, '#!/bin/sh\nexit 0\n', { mode: 0o755 });
|
|
writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(legacyCommand)};`);
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /Selected Codex platform version must match the qualified pin/);
|
|
writeFileSync(index, `import { writeFileSync } from 'node:fs'; export const resolvePinnedCodexCommand = () => {
|
|
writeFileSync(${JSON.stringify(join(platform, 'package.json'))}, ${JSON.stringify(JSON.stringify({ ...manifest, altered: true }))});
|
|
return ${JSON.stringify(command)}; };`);
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /metadata must match its scanned provenance/);
|
|
writeFileSync(join(platform, 'package.json'), JSON.stringify(manifest));
|
|
writeFileSync(index, "export const resolvePinnedCodexCommand = () => { throw new Error('Bundled Codex runtime version mismatch'); };");
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /Bundled Codex runtime version mismatch/);
|
|
writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(command)};`);
|
|
writeFileSync(security, securitySource.replace(JSON.stringify(nativeVendor), '"/missing-vendor-root"'));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /vendor directory must be readable/);
|
|
writeFileSync(security, securitySource.replace(JSON.stringify(nativeVendor), `${JSON.stringify(nativeVendor)}, ${JSON.stringify(consumer)}`));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /must not grant enclosing npm/);
|
|
writeFileSync(security, securitySource.replace("'=\"read\"'", "'=\"write\"'"));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /retain the selected native executable resources/);
|
|
writeFileSync(security, securitySource);
|
|
const emptySlot = join(server, 'node_modules/@openai/codex-' + target);
|
|
mkdirSync(emptySlot, { recursive: true });
|
|
result = run(); assert.equal(result.status, 0, result.stderr);
|
|
assert.equal(JSON.parse(result.stdout).codexPlatformPackages.length, 2,
|
|
'An empty nested npm optional slot must not obscure the qualified hoisted host package');
|
|
writeFileSync(join(emptySlot, 'undeclared-native-payload'), 'native fixture; never executed');
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /empty regular directory; found.*undeclared-native-payload/);
|
|
rmSync(join(emptySlot, 'undeclared-native-payload'));
|
|
writeFileSync(join(emptySlot, 'package.json'), '{ malformed');
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /SyntaxError/);
|
|
rmSync(join(emptySlot, 'package.json'));
|
|
writeFileSync(integrity, moduleSource.replace('assert.deepEqual(profile,', "throw new Error('qualified executable digest mismatch'); assert.deepEqual(profile,"));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /qualified executable digest mismatch/);
|
|
writeFileSync(integrity, moduleSource.replace("writeFileSync(", "throw new Error('command lease close failed'); writeFileSync("));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /command lease close failed/);
|
|
writeFileSync(integrity, moduleSource);
|
|
lock.packages[platformPath].inBundle = true;
|
|
writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /downloaded, not bundled/);
|
|
delete lock.packages[platformPath].inBundle;
|
|
writeFileSync(join(consumer, 'package-lock.json'), JSON.stringify(lock));
|
|
const foreign = join(consumer, 'node_modules/@openai/codex-win32-arm64'); mkdirSync(foreign);
|
|
writeFileSync(join(foreign, 'package.json'), JSON.stringify({ name: '@openai/codex', version: '0.160.0-win32-arm64' }));
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /only the host/);
|
|
rmSync(foreign, { recursive: true });
|
|
executable('9.9.9'); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /qualified pin/);
|
|
rmSync(command); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /ENOENT/);
|
|
const outside = join(root, 'outside'); writeFileSync(outside, '#!/bin/sh\nexit 0\n', { mode: 0o755 });
|
|
writeFileSync(index, `export const resolvePinnedCodexCommand = () => ${JSON.stringify(outside)};`);
|
|
result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /inside the installed consumer/);
|
|
writeFileSync(index, 'export const unrelated = true;'); result = run(); assert.notEqual(result.status, 0); assert.match(result.stderr, /must export the pinned Codex resolver/);
|
|
} finally { rmSync(root, { recursive: true, force: true }); }
|
|
});
|
|
|
|
test('actual tar member checks allow Codex JavaScript but reject nested platform packages and native vendor payloads', () => {
|
|
const javascript = ['package/package.json', 'package/node_modules/@openai/codex/package.json',
|
|
'package/node_modules/@openai/codex/bin/codex.js', 'package/dist/vendor/paperclip-runner/index.js'];
|
|
assert.doesNotThrow(() => assertNoBundledCodexPayloads(javascript));
|
|
for (const payload of [
|
|
'package/node_modules/@openai/codex-linux-x64/package.json',
|
|
'package/node_modules/bridge/node_modules/@openai/codex-darwin-arm64/package.json',
|
|
'package/node_modules/@openai/codex-win32-x64/package.json',
|
|
'package/node_modules/@openai/codex/vendor/x86_64-unknown-linux-musl/codex/codex',
|
|
'package/dist/vendor/aarch64-apple-darwin/bin/codex',
|
|
'package/dist/vendor/x86_64-pc-windows-msvc/bin/codex.exe',
|
|
]) assert.throws(() => assertNoBundledCodexPayloads([...javascript, payload]), /must not bundle/);
|
|
assert.throws(() => assertNoBundledCodexPayloads(['package/../outside']), /inside the package/);
|
|
const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
|
|
assert.match(source, /assertNoBundledCodexPayloads\(run\('tar', \['-tzf', tarball\]\)/);
|
|
});
|
|
|
|
test('consumer provenance rejects bundled, foreign, inconsistent and non-official Codex payloads', () => {
|
|
const path = 'node_modules/@openai/codex-linux-x64';
|
|
const manifest = { name: '@openai/codex', version: '0.160.0-linux-x64', os: ['linux'], cpu: ['x64'] };
|
|
const entry = { version: manifest.version, resolved: `https://registry.npmjs.org/@openai/codex/-/codex-${manifest.version}.tgz`,
|
|
integrity: `sha512-${Buffer.alloc(64, 1).toString('base64')}` };
|
|
const check = (value = entry, installed = [{ path, manifest }]) => assertInstalledCodexPlatformProvenance({ packages: { [path]: value } }, installed, 'linux-x64');
|
|
assert.equal(check().codexConsumerHostOnly, true);
|
|
const legacyManifest = { ...manifest, version: '0.156.1-linux-x64' };
|
|
assert.equal(check({ ...entry, version: legacyManifest.version,
|
|
resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-0.156.1-linux-x64.tgz' }, [{ path, manifest: legacyManifest }]).codexConsumerHostOnly, true);
|
|
assert.throws(() => check(entry, [{ path, manifest: { ...manifest, version: '^0.160.0-linux-x64' } }]), /exact host version/);
|
|
for (const change of [
|
|
{ inBundle: true }, { link: true }, { version: '0.156.1-linux-x64' },
|
|
{ resolved: 'https://example.com/@openai/codex/-/codex-0.160.0-linux-x64.tgz' },
|
|
{ resolved: 'https://registry.npmjs.org/@openai/codex/-/codex-0.156.1-linux-x64.tgz' },
|
|
{ resolved: 'https://user@registry.npmjs.org/@openai/codex/-/codex-0.160.0-linux-x64.tgz' },
|
|
{ integrity: undefined }, { integrity: 'sha512-AA==' },
|
|
]) assert.throws(() => check({ ...entry, ...change }));
|
|
assert.throws(() => check(entry, []), /must install its host/);
|
|
assert.throws(() => check(entry, [{ path: 'node_modules/@openai/codex-darwin-arm64', manifest }]), /only the host/);
|
|
assert.throws(() => check(entry, [{ path, manifest: { ...manifest, name: 'untrusted' } }]), /identity/);
|
|
});
|