perf(ci): pack the verifier's staged packages concurrently (#15888)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every pull request runs the `Build` check. Its last step is the
public npm install verifier, `scripts/verify-grok-npm-install.mjs`,
which stages the eighteen public packages, packs them, and installs them
in a clean consumer.
> - The verifier packs the eighteen packages one after another with `npm
pack`. Each pack is a Node start-up plus gzip of that package's files.
The timing line added in #15868 shows this loop at 35.6-37.9s on four
`Build` runs, the largest part of the staging phase now that the bundled
installs overlap.
> - `Build` at 433-530s is the slowest check in most PR runs once the
server shards drop, so its staging phase sets the wall-clock time of the
run.
> - Each pack reads only its own `package-<index>` directory and writes
one tarball. The runner has four cores and the packs are CPU-bound.
There is no reason to run them one after the other.
> - This pull request runs the packs as concurrent children through the
prefetch helper, one per core, bundled packages first, each into its own
directory, and puts the results back into dependency order before every
tarball is checked.
> - The benefit is a shorter `Build` job, and a timing line that reports
the packs' serial cost next to their wall time, so the saving is
measured on this PR's own run instead of estimated.

## Linked Issues or Issue Description

Refs #15868, #15861. This branch is stacked on the branch of #15868,
which is stacked on #15861, because the same execution branch holds both
open PRs. The diff here is this PR's one commit. After #15868 merges,
this PR is retargeted to `master` and rebased.

**What existing behavior does this improve?**

The `Build` PR check. Its public npm install verifier packs the eighteen
public packages in one serial loop.

**Subsystem affected**

CI and release scripts (`scripts/verify-grok-npm-install.mjs`,
`scripts/prefetch-child.mjs`).

**Current behavior**

After the bundled installs are joined, the verifier runs `npm pack
--pack-destination <root>` for each of the eighteen packages in turn,
finds the new tarball by listing the shared root, and checks it with
`tar -tzf`. `stage_pack_ms` was 35.8s, 37.1s, 35.6s and 37.9s on the
four runs of #15868. The parent is idle apart from the one pack it waits
on, and three of the four cores are idle.

**Proposed behavior**

Start the packs as children through `startPrefetch`, at most one per
core (`os.availableParallelism()`), bundled packages first because they
make the largest tarballs. Each child packs into its own `pack-<index>`
directory, so the tarball a package produced is read from there instead
of guessed from a shared directory. Register each child for cleanup as
it starts. Put the results back into dependency order, then check every
tarball with `tar -tzf` as before, then copy them into the assets
directory as before. Report `stage_pack_check_ms`, `pack_concurrency`,
`pack_serial_ms` and `pack_longest_ms` on the timing line, and keep
`stage_pack_ms` as the whole step.

**Reason and benefit**

With four slots the step takes about the longest single pack or a
quarter of the serial sum, whichever is larger. The expected saving is
20-25s of the 36-38s, measured on this PR's own `Build` job by
`pack_serial_ms` against `stage_pack_ms`.

**Measured on this PR's first run
([38146108750](https://github.com/paperclipai/paperclip/actions/runs/38146108750),
`Build` 433s, verifier 214s):** `stage_pack_ms` 29.9s against 35.6-37.9s
before, so the step shed 6-8s, not the 20-25s estimated.
`pack_serial_ms` is 44.4s with `pack_concurrency` 4, and
`pack_longest_ms` is 28.3s: one pack (the server with its 513 MB bundled
closure, gzip level 9 inside npm) runs for almost the whole step, so the
step is bounded by that pack and not by the slot count. The seventeen
other packs fit inside it. The serial sum is 7-9s higher than the old
serial step, which is the four processes contending for the runner's
cores. `stage_packages_ms` 75.8s against 80.4-89.6s on #15868's runs;
`image_wait_ms` and `grok_wait_ms` stay 0.

**Breaking changes**

None. The same packages are packed with the same `npm pack` arguments,
the tarballs are checked the same way, and the consumer receives them in
the same order.

## What Changed

- `scripts/pack-staged-packages.mjs` (new): `packStagedPackages`
launches the packs in the order given with at most `concurrency`
children at once, through the prefetch helper, and resolves with each
entry's tarball path and wall time in the same order. Every child is
passed to `register` as it starts. The first failure rejects with the
tail of that child's log and nothing further is launched. A child that
leaves anything other than one tarball in its directory is a failure.
`packConcurrency` is one per core. `packTimings` gives the timing-line
keys.
- `scripts/verify-grok-npm-install.mjs`: each entry gets its `index` and
its own `pack-<index>` destination. The pack loop is replaced by one
`packStagedPackages` call, launched bundled-first and registered into
the `prefetches` list as each child starts, so an earlier failure stops
any pack still running before the scratch root is removed. The results
are sorted back into dependency order, every tarball is checked in that
order, and the copy into the assets directory is unchanged. Only the
lifecycle sentinel is still packed in-process.
- Timing line: `stage_pack_check_ms`, `pack_concurrency`,
`pack_serial_ms` and `pack_longest_ms` are added. `stage_pack_ms` still
covers the whole step, so the before and after numbers compare directly.
- `scripts/__tests__/pack-staged-packages.test.mjs` (new): five tests
with a local stand-in for npm and no network. The default concurrency
and the pack arguments; four packs with two slots run the two slow packs
together, start a third only once a slot is free, receive the exact `npm
pack` argument list, register every child, and come back in entry order
with their tarballs and timing keys although the fast packs finish
first; a failed pack rejects with its log tail, launches nothing more,
and leaves the running pack to be stopped; a pack that produces two
tarballs or none is a failure; and no entries resolve at once.
- `scripts/__tests__/grok-public-install-sandbox.test.mjs`: the guard
test on the verifier source now checks the import, the bundled-first
launch with cleanup registration, the reorder before the checks, the
checks before the copy, that only the sentinel is packed in-process,
that each pack has its own directory, and the new timing keys.
- `package.json`: the new test file is in `test:release-registry`.

## Verification

- `node --test scripts/__tests__/pack-staged-packages.test.mjs
scripts/__tests__/grok-public-install-sandbox.test.mjs
scripts/__tests__/stage-bundled-packages.test.mjs
scripts/__tests__/prefetch-child.test.mjs`: 35 tests pass (30 on
#15868's head plus the five new ones). The new file passes three
consecutive runs.
- `node --check` passes on both scripts.
- With the real npm 11.12.1 on Node 25.9: two small packages packed
through `packStagedPackages` with two slots produce one tarball each in
their own `pack-<index>` directory, with the expected `package/` listing
from `tar -tzf`.
- The verifier itself needs Docker and a network, which this workspace
does not have. This PR's own `Build` job runs the branch's verifier; its
timing line carries `stage_pack_ms`, `pack_serial_ms`, `pack_longest_ms`
and `pack_concurrency`, which is the measurement. Before, on #15868's
four runs: `stage_pack_ms` 35.8s / 37.1s / 35.6s / 37.9s.
- `docker-runner-check.yml` runs the same script after `pnpm build` and
needs no change.

## Risks

- Four `npm pack` processes run at once and share the host npm
configuration and cache. `npm pack --ignore-scripts` reads the package
directory and writes one tarball; it does not write to the cache beyond
its per-process log file. If a shared-state race appears, the failing
pack's log tail is in the error and the fix is to lower the concurrency.
- Memory and CPU: four Node processes instead of one on the four-core
`Build` runner, for about thirty seconds. The largest pack (the server
with its bundled dependencies) is the same process as before.
- The pack output moves from the CI log into per-pack log files in the
scratch root. A failure reports the last 2000 characters of the failing
pack's log; a successful pack's output is discarded with the scratch
root.
- The tarball checks (`tar -tzf`) now run after all packs instead of
after each pack. A bundled Codex payload is still rejected before any
tarball reaches the consumer; the only change is that the error arrives
a few seconds later.
- Low risk otherwise. The bundled installs, the consumer install, and
every probe are unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.1 (Anthropic), model ID `claude-fable-5-1`, run as a
Paperclip agent through the Claude Code local adapter with tool use
(shell, file edits, GitHub CLI). Reasoning mode: extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (run 38146108750: `server
(11/12)` failed on its first attempt on a teardown `EACCES` while
removing the per-worker `PAPERCLIP_HOME` after all 74 files and 1257
tests had passed, in a file this PR does not touch; the re-run of that
job passed and every check is green)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

package.json

@@ -61,7 +61,7 @@
     "smoke:posthog-live": "node scripts/smoke/posthog-live.mjs",
     "smoke:pipelines-tutorial": "./scripts/smoke/pipelines-tutorial-smoke.sh",
     "smoke:terminal-bench-loop-skill": "node scripts/smoke/terminal-bench-loop-skill-smoke.mjs",
-    "test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
+    "test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
     "storybook-visual:baseline": "node scripts/storybook-visual-baseline.mjs",
     "test:storybook-visual": "node scripts/storybook-visual-baseline.mjs download && node scripts/storybook-visual-baseline.mjs verify && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts",
     "test:storybook-visual:update": "node scripts/storybook-visual-baseline.mjs download && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts --update-snapshots && node scripts/storybook-visual-baseline.mjs pack",

scripts/__tests__/grok-public-install-sandbox.test.mjs

@@ -120,19 +120,28 @@ test('the bundled-dependency installs run together while the other packages are
   // Only the bundled packages' sources are staged before the installs start;
   // the installs are registered for cleanup at once, before anything else can
   // fail; the other packages are copied while the installs run; the join comes
-  // before the first pack; and the pack loop walks every package in order.
+  // before the packs start; and the packs run through the helper covered by
+  // pack-staged-packages.test.mjs, bundled packages first, each registered for
+  // cleanup as it starts, with the results put back into dependency order
+  // before every tarball is checked.
   const bundledSources = position('for (const entry of entries.filter(entry => entry.bundled)) stageSource(entry)');
   const start = position("startBundledInstalls(entries.filter(entry => entry.bundled), { logDir: prefetched, cwd: repo })");
   const registered = position('prefetches.push(...installs.map(({ handle }) => handle))');
   const otherSources = position('for (const entry of entries.filter(entry => !entry.bundled))');
   const join = position("await awaited('bundled', Promise.all(installs.map(({ handle }) => handle.done)))");
-  const packLoop = position('for (const { target } of entries)');
-  const firstPack = position("run('npm', ['pack', '--ignore-scripts', '--pack-destination', root], target)");
-  assert.ok(bundledSources < start && start < registered && registered < otherSources && otherSources < join && join < packLoop && packLoop < firstPack);
-  assert.equal(source.match(/run\('npm', \['pack'/g).length, 2, 'one package pack loop plus the lifecycle sentinel');
-  // The timing line reports the split staging phase and each install.
-  for (const key of ['stage_sources_ms', 'stage_bundled_ms', 'stage_pack_ms']) assert.ok(source.includes(`timing.${key} =`));
+  assert.ok(source.includes("import { packConcurrency, packStagedPackages, packTimings } from './pack-staged-packages.mjs'"));
+  const packs = position("await packStagedPackages([...entries].sort((a, b) => Number(b.bundled) - Number(a.bundled)), { logDir: prefetched, env, concurrency, register: handle => prefetches.push(handle) })");
+  const reordered = position('packs.sort((a, b) => a.index - b.index)');
+  const checks = position("for (const tarball of tarballs) assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball])");
+  const copied = position('for (const tarball of tarballs) {');
+  assert.ok(bundledSources < start && start < registered && registered < otherSources && otherSources < join && join < packs && packs < reordered && reordered < checks && checks < copied);
+  assert.equal(source.match(/run\('npm', \['pack'/g).length, 1, 'only the lifecycle sentinel is packed in-process');
+  assert.ok(source.includes("destination: join(root, `pack-${index}`)"), 'each pack writes into its own directory');
+  // The timing line reports the split staging phase, each install, and the
+  // pack step with what the packs would have cost one after another.
+  for (const key of ['stage_sources_ms', 'stage_bundled_ms', 'stage_pack_ms', 'stage_pack_check_ms']) assert.ok(source.includes(`timing.${key} =`));
   assert.ok(source.includes('...bundledInstallTimings(installs)'));
+  assert.ok(source.includes('Object.assign(timing, packTimings(packs, concurrency))'));
 });
 
 test('the installed Codex probe exercises the public export and rejects incomplete or mismatched packages', () => {

scripts/__tests__/pack-staged-packages.test.mjs

@@ -0,0 +1,107 @@
+import assert from 'node:assert/strict';
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import test from 'node:test';
+import { stopPrefetches } from '../prefetch-child.mjs';
+import { PACK_ARGS, packConcurrency, packStagedPackages, packTimings } from '../pack-staged-packages.mjs';
+
+// A small local script stands in for npm; no test here needs npm or a network.
+// It is run as `node <script> pack --ignore-scripts --pack-destination <dir>`,
+// the same argument list the real npm receives.
+const withScratch = async body => {
+  const root = mkdtempSync(join(tmpdir(), 'pack-staged-packages-'));
+  try { await body(root); } finally { rmSync(root, { recursive: true, force: true }); }
+};
+const entriesFor = (root, names) => names.map((name, index) => {
+  const target = join(root, `package-${index}`);
+  mkdirSync(target);
+  return { name, index, target, destination: join(root, `pack-${index}`) };
+});
+const fakeNpm = (root, body) => { const script = join(root, 'fake-npm.mjs'); writeFileSync(script, body); return [process.execPath, script]; };
+const settle = ms => new Promise(resolve => setTimeout(resolve, ms));
+
+test('the concurrency defaults to one child per core', () => {
+  assert.ok(Number.isInteger(packConcurrency()) && packConcurrency() >= 1);
+  assert.deepEqual(PACK_ARGS, ['pack', '--ignore-scripts', '--pack-destination']);
+});
+
+test('packs run together up to the concurrency, in the order given, and the tarballs come back in entry order', async () => withScratch(async root => {
+  // Each stand-in records when it started and finished, writes one tarball
+  // named after its package, and prints the tarball name as npm does. The
+  // first two packs are slow and the last two fast, so with two children at
+  // a time the slow pair runs together and the fast pair starts only once one
+  // of them has finished; the fast packs then finish out of entry order.
+  const command = fakeNpm(root, `
+    import { basename, join } from 'node:path';
+    import { writeFileSync } from 'node:fs';
+    const args = process.argv.slice(2);
+    const destination = args[args.length - 1];
+    const name = basename(process.cwd());
+    const started = Date.now();
+    const delay = name === 'package-0' || name === 'package-1' ? 400 : 50;
+    Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, delay);
+    writeFileSync(join(destination, name + '-1.0.0.tgz'), 'tarball');
+    writeFileSync(join(destination, '..', 'span-' + name), started + ' ' + Date.now());
+    console.log(args.join(' '));
+    console.log(name + '-1.0.0.tgz');
+  `);
+  const entries = entriesFor(root, ['@paperclipai/server', '@paperclipai/db', '@paperclipai/shared', 'paperclipai']);
+  const registered = [];
+  const packs = await packStagedPackages(entries, { logDir: root, concurrency: 2, register: handle => registered.push(handle), command });
+  assert.deepEqual(packs.map(({ name }) => name), entries.map(({ name }) => name));
+  assert.deepEqual(packs.map(({ index }) => index), [0, 1, 2, 3]);
+  for (const [index, pack] of packs.entries()) {
+    assert.equal(pack.tarball, join(root, `pack-${index}`, `package-${index}-1.0.0.tgz`));
+    assert.equal(readFileSync(pack.tarball, 'utf8'), 'tarball');
+    assert.ok(Number.isInteger(pack.durationMs) && pack.durationMs >= 0);
+    const log = readFileSync(join(root, `pack-${index}.log`), 'utf8');
+    assert.equal(log.split('\n')[0], `pack --ignore-scripts --pack-destination ${join(root, `pack-${index}`)}`);
+  }
+  assert.equal(registered.length, 4, 'every pack is registered for cleanup as it starts');
+  for (const handle of registered) assert.equal(handle.child.exitCode, 0);
+  const span = name => readFileSync(join(root, `span-${name}`), 'utf8').split(' ').map(Number);
+  const [slowA, slowB, fastA, fastB] = ['package-0', 'package-1', 'package-2', 'package-3'].map(span);
+  assert.ok(slowA[0] < slowB[1] && slowB[0] < slowA[1], 'the two slow packs ran at the same time');
+  const firstSlowEnd = Math.min(slowA[1], slowB[1]);
+  assert.ok(fastA[0] >= firstSlowEnd - 50 && fastB[0] >= firstSlowEnd - 50, 'a third pack starts only once a slot is free');
+  const timings = packTimings(packs, 2);
+  assert.equal(timings.pack_concurrency, 2);
+  assert.equal(timings.pack_serial_ms, packs.reduce((total, { durationMs }) => total + durationMs, 0));
+  assert.equal(timings.pack_longest_ms, Math.max(...packs.map(({ durationMs }) => durationMs)));
+}));
+
+test('a failed pack rejects with the tail of its log, launches nothing more, and leaves the running pack to be stopped', async () => withScratch(async root => {
+  const command = fakeNpm(root, `
+    import { basename } from 'node:path';
+    if (basename(process.cwd()) === 'package-1') { console.error('pack exploded'); process.exit(2); }
+    setTimeout(() => {}, 30_000);
+  `);
+  const entries = entriesFor(root, ['@paperclipai/server', '@paperclipai/db', '@paperclipai/shared', 'paperclipai']);
+  const registered = [];
+  await assert.rejects(packStagedPackages(entries, { logDir: root, concurrency: 2, register: handle => registered.push(handle), command }), /pack-1 prefetch failed \(exit 2\): pack exploded/);
+  await settle(100);
+  assert.equal(registered.length, 2, 'the two queued packs never start after the failure');
+  assert.equal(registered[0].child.exitCode, null, 'the other pack is still running when the failure surfaces');
+  assert.ok(!existsSync(join(root, 'pack-2')) && !existsSync(join(root, 'pack-3')));
+  await stopPrefetches(registered);
+  assert.notEqual(registered[0].child.signalCode ?? registered[0].child.exitCode, null);
+}));
+
+test('a pack that exits without producing exactly one tarball is a failure', async () => withScratch(async root => {
+  const command = fakeNpm(root, `
+    import { basename, join } from 'node:path';
+    import { writeFileSync } from 'node:fs';
+    const destination = process.argv[process.argv.length - 1];
+    const name = basename(process.cwd());
+    if (name === 'package-0') { writeFileSync(join(destination, 'a.tgz'), ''); writeFileSync(join(destination, 'b.tgz'), ''); }
+  `);
+  const entries = entriesFor(root, ['@paperclipai/server', '@paperclipai/db']);
+  await assert.rejects(packStagedPackages(entries, { logDir: root, concurrency: 1, command }), /@paperclipai\/server pack produced 2 tarballs/);
+  assert.deepEqual(readdirSync(root).filter(name => name.startsWith('pack-') && !name.endsWith('.log')), ['pack-0'], 'the second pack never started');
+  await assert.rejects(packStagedPackages([entries[1]], { logDir: root, concurrency: 1, command }), /@paperclipai\/db pack produced 0 tarballs/);
+}));
+
+test('no entries resolve at once', async () => withScratch(async root => {
+  assert.deepEqual(await packStagedPackages([], { logDir: root, concurrency: 2, command: [process.execPath, '-e', ''] }), []);
+}));

scripts/pack-staged-packages.mjs

@@ -0,0 +1,64 @@
+// The public-install verifier packs every staged public package with
+// `npm pack`. Each pack reads only its own `package-<index>` directory and
+// writes one tarball, and the work is a Node start-up plus CPU-bound gzip per
+// package, so the packs run as concurrent children through the prefetch
+// helper, at most one per core, instead of one after another. Each child
+// packs into its own destination directory, so the tarball a package produced
+// is read from there rather than guessed from a shared directory, and the
+// results come back in the order the entries were given.
+import { mkdirSync, readdirSync } from 'node:fs';
+import { availableParallelism } from 'node:os';
+import { join } from 'node:path';
+import { startPrefetch } from './prefetch-child.mjs';
+
+export const PACK_ARGS = ['pack', '--ignore-scripts', '--pack-destination'];
+
+export const packConcurrency = () => Math.max(1, availableParallelism());
+
+// Packs `entries` (`{ name, target, destination }`) with at most `concurrency`
+// children at once, launched in the order given, and resolves with each entry
+// extended by its `tarball` path and `durationMs`, in the same order. Every
+// child is passed to `register` as it starts, so the caller can stop the ones
+// still running when something fails. The first failure rejects with the tail
+// of that child's log, and nothing further is launched after it.
+export function packStagedPackages(entries, { logDir, env = process.env, concurrency = packConcurrency(), register = () => {}, command = ['npm'] }) {
+  const [cmd, ...commandArgs] = command;
+  const results = new Array(entries.length);
+  const queue = entries.map((entry, index) => ({ entry, index }));
+  let active = 0;
+  let failed = false;
+  return new Promise((resolve, reject) => {
+    const fail = error => { failed = true; reject(error); };
+    const launch = () => {
+      if (!failed && active === 0 && queue.length === 0) return resolve(results);
+      while (!failed && active < concurrency && queue.length > 0) {
+        const { entry, index } = queue.shift();
+        active += 1;
+        let handle;
+        try {
+          mkdirSync(entry.destination, { recursive: true });
+          handle = startPrefetch({ name: `pack-${index}`, cmd, args: [...commandArgs, ...PACK_ARGS, entry.destination], cwd: entry.target, env, logPath: join(logDir, `pack-${index}.log`) });
+          register(handle);
+        } catch (error) {
+          return fail(error);
+        }
+        handle.done.then(() => {
+          const packed = readdirSync(entry.destination).filter(file => file.endsWith('.tgz'));
+          if (packed.length !== 1) throw new Error(`${entry.name} pack produced ${packed.length} tarballs in ${entry.destination}`);
+          results[index] = { ...entry, tarball: join(entry.destination, packed[0]), durationMs: handle.durationMs };
+          active -= 1;
+          launch();
+        }).catch(fail);
+      }
+    };
+    launch();
+  });
+}
+
+// Timing-line keys: the concurrency used, the sum of the per-pack times (what
+// the packs would have cost one after another) and the longest single pack.
+export const packTimings = (packs, concurrency) => ({
+  pack_concurrency: concurrency,
+  pack_serial_ms: packs.reduce((total, { durationMs }) => total + durationMs, 0),
+  pack_longest_ms: packs.reduce((longest, { durationMs }) => Math.max(longest, durationMs), 0),
+});

scripts/verify-grok-npm-install.mjs

@@ -8,6 +8,7 @@ import { chmodSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, re
 import { tmpdir } from 'node:os';
 import { basename, dirname, join, resolve } from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { packConcurrency, packStagedPackages, packTimings } from './pack-staged-packages.mjs';
 import { materializePublishManifest } from './prepare-bundled-package.mjs';
 import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, assertNoBundledCodexPayloads, grokConsumerDockerArgs, installedCodexProbeSource } from './grok-public-install-sandbox.mjs';
 import { startPrefetch, stopPrefetches } from './prefetch-child.mjs';
@@ -73,12 +74,14 @@ try {
   // each need a host-side `npm install` of their bundled closure; those installs
   // write only their own `package-<index>` and read nothing another package
   // produces, so they start first and run together while the other packages
-  // are copied, instead of one after another inside one loop. Packing stays
-  // serial and in order, because each tarball is checked as it is produced.
+  // are copied, instead of one after another inside one loop. Packing is a
+  // Node start-up plus CPU-bound gzip per package that reads only the package's
+  // own directory, so the packs also run together, each into its own
+  // `pack-<index>`, and the tarballs are checked in order once all are made.
   const entries = [...needed].map((name, index) => {
     const { dir, manifest } = packages.get(name);
     const bundled = (manifest.bundleDependencies ?? manifest.bundledDependencies ?? []).length > 0;
-    return { name, dir, manifest, bundled, target: join(root, `package-${index}`), stagedSource: join(root, `source-${index}`) };
+    return { name, index, dir, manifest, bundled, target: join(root, `package-${index}`), stagedSource: join(root, `source-${index}`), destination: join(root, `pack-${index}`) };
   });
   const stageSource = ({ dir, manifest, stagedSource }) => {
     mkdirSync(stagedSource);
@@ -109,16 +112,20 @@ try {
   await awaited('bundled', Promise.all(installs.map(({ handle }) => handle.done)));
   timing.stage_bundled_ms = Math.round(performance.now() - installsStarted);
   const packStarted = performance.now();
-  const tarballs = [];
-  for (const { target } of entries) {
-    run('npm', ['pack', '--ignore-scripts', '--pack-destination', root], target);
-    const packed = readdirSync(root).filter(f => f.endsWith('.tgz') && !tarballs.includes(join(root, f)));
-    assert.equal(packed.length, 1);
-    const tarball = join(root, packed[0]);
-    assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball]).toString().trim().split('\n'));
-    tarballs.push(tarball);
-  }
+  // The bundled packages make the largest tarballs, so they are launched first
+  // and the short packs fill the remaining slots behind them. Each pack is
+  // registered for cleanup as it starts, so a failure stops the ones still
+  // running before the scratch root is removed. The results are put back into
+  // dependency order before the tarballs are checked and copied.
+  const concurrency = packConcurrency();
+  const packs = await packStagedPackages([...entries].sort((a, b) => Number(b.bundled) - Number(a.bundled)), { logDir: prefetched, env, concurrency, register: handle => prefetches.push(handle) });
+  packs.sort((a, b) => a.index - b.index);
+  const checksStarted = performance.now();
+  const tarballs = packs.map(({ tarball }) => tarball);
+  for (const tarball of tarballs) assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball]).toString().trim().split('\n'));
+  timing.stage_pack_check_ms = Math.round(performance.now() - checksStarted);
   timing.stage_pack_ms = Math.round(performance.now() - packStarted);
+  Object.assign(timing, packTimings(packs, concurrency));
   const assets = join(root, 'assets'); mkdirSync(assets, { mode: 0o755 });
   const consumer = join(root, 'consumer'); mkdirSync(consumer);
   const cache = join(root, 'cache'); mkdirSync(cache);