perf(ci): run the verifier's bundled-dependency installs concurrently (#15868)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every pull request runs the `Build` check. Its last step is the
public npm install verifier, `scripts/verify-grok-npm-install.mjs`,
which stages the eighteen public packages and installs them in a clean
consumer.
> - The verifier stages the packages in one serial loop. Three packages
bundle dependencies (`@paperclipai/server`,
`@paperclipai/adapter-utils`, `@paperclipai/db`). Each one runs a
host-side `npm install` of its bundled closure in-process, one after the
other, and nothing in the loop overlaps anything else. The staging phase
took 102-116s on recent `Build` jobs.
> - `Build` at 433-530s is the slowest check in most PR runs once the
server shards drop, so its staging phase sets the wall-clock time of the
run.
> - The three installs write only their own directories and read nothing
from each other. There is no reason to run them one after the other.
> - This pull request starts the three installs together as child
processes and copies the other fifteen packages while they run. Packing
stays serial and in order.
> - The benefit is a shorter `Build` job, and a timing line that reports
each install, so the saving is measured on this PR's own run instead of
estimated.

## Linked Issues or Issue Description

Refs #15861. This branch is stacked on the branch of #15861 because the
same execution branch holds that open PR. The diff here is this PR's
three commits. After #15861 merges, this PR is retargeted to `master`
and rebased.

**What existing behavior does this improve?**

The `Build` PR check. Its public npm install verifier stages the
eighteen public packages in one serial loop.

**Subsystem affected**

CI and release scripts (`scripts/verify-grok-npm-install.mjs`,
`scripts/prepare-bundled-package.mjs`, `scripts/prefetch-child.mjs`,
`scripts/prefetch-supervisor.mjs`).

**Current behavior**

The staging loop calls `prepareBundledPackage` in-process for the three
packages that bundle dependencies. Each call runs a host-side `npm
install` of that package's bundled closure, applies the repo patches,
and rewrites the manifest, before the loop moves to the next package.
The timing line reports the whole staging phase as one number, 102-116s
on recent runs. The three installs are the only network work in that
phase.

**Proposed behavior**

Stage the three bundled packages' sources first, start their installs at
the same time as children of the script's own CLI entry (the same path
`release.sh` uses), copy the other fifteen packages while the installs
run, join the installs, and then pack every package in dependency order
as before. Report `stage_sources_ms`, `stage_bundled_ms`,
`stage_pack_ms`, `bundled_wait_ms` and one `stage_bundled_<package>_ms`
per install on the timing line.

**Reason and benefit**

The staging phase shortens by the two shorter installs. Measured on this
PR's first run (below): the three installs sum to 25.9s and take 16.4s
of wall time together, so the overlap removes about 9.5s from the phase,
and the timing line now shows that packing (35.8s) and the
standalone-package build before the loop (about 27s) are the larger
remaining costs. The original estimate of 30-50s was too high because it
assumed the installs were about half of the phase; they are a quarter.

**Breaking changes**

None. The packed tarballs are produced by the same script, in the same
order, and are checked the same way.

## What Changed

- `scripts/stage-bundled-packages.mjs` (new): `startBundledInstalls`
starts one child per bundled package through the prefetch helper from
#15849, with the child's output in a log file. `bundledInstallTimings`
reads each child's wall time for the timing line. `bundledPackageSlug`
maps a package name to its log and timing key.
- `scripts/verify-grok-npm-install.mjs`: the staging loop is split. The
bundled packages' sources are staged first and their installs start
together. The other packages are copied while the installs run. The
install handles are pushed into the `prefetches` list at once, so an
earlier failure stops any install still running before the scratch root
is removed. The join comes before the first `npm pack`. The pack loop
walks every package in `needed` order and checks each tarball as before.
`prepareBundledPackage` is no longer imported.
- Timing line: `stage_sources_ms`, `stage_bundled_ms`, `stage_pack_ms`,
`bundled_wait_ms` and `stage_bundled_server_ms`,
`stage_bundled_adapter_utils_ms`, `stage_bundled_db_ms` are added.
`stage_packages_ms` and the download keys are unchanged, so the before
and after numbers compare directly.
- `scripts/__tests__/stage-bundled-packages.test.mjs` (new): five tests
with local Node children and no network. The slug mapping; the children
run the CLI entry `release.sh` runs and never an in-process call; two
installs run at the same time, each with its own source and target, and
their logs and timing keys are correct (each child exits only after the
other has started, so serial children would fail); a failed install
rejects with its own log tail while the other is still running and is
then stopped; and stopping the installs after one fails also stops the
stand-in npm process the other wrapper is blocked on.
- `scripts/__tests__/grok-public-install-sandbox.test.mjs`: one guard
test on the verifier source. The bundled sources are staged before the
installs start, the installs are registered for cleanup before the other
packages are copied, the join comes before the pack loop, the pack loop
is in order, there is still exactly one package pack loop, and the
timing keys are reported.
- `package.json`: the new test file is in `test:release-registry`.

### Review fix (second commit): stop a prefetch's whole process tree

Greptile pointed out that `stopPrefetches` signalled only the Node
wrapper of each install, while the wrapper is blocked in
`execFileSync('npm', ['install', ...])`. After one install failed, the
other wrapper died and its npm process kept writing into the scratch
root the verifier was about to remove.

- `scripts/prefetch-child.mjs`: on POSIX every prefetch child is started
`detached`, so it leads its own process group. `stopPrefetches` signals
the group (`process.kill(-pid, ...)`) and waits until no member of the
group exists, with `SIGKILL` after the grace period and a second grace
period before it gives up so cleanup cannot hang. On Windows the child
is signalled directly as before.
- `scripts/__tests__/prefetch-child.test.mjs`: two tests. A stopped
prefetch's grandchild is gone after `stopPrefetches` returns, and a
`SIGINT` to a separate parent process reaches the grandchild while the
parent still exits with `SIGINT`. Both are skipped on Windows.
- `scripts/__tests__/stage-bundled-packages.test.mjs`: one test, the
cancellation case Greptile asked for. The stand-in
`prepare-bundled-package` blocks in `execFileSync` on a stand-in npm
that announces its pid; after the other install fails, `stopPrefetches`
must leave that pid gone.

All three new tests fail against the previous helper (`grandchild <pid>
outlived stopPrefetches`, `outlived the parent's SIGINT`, `stand-in npm
<pid> outlived stopPrefetches`) and pass with the fix.

### Review fix (third commit): the parent runs no signal handler

Greptile pointed out that the second commit forwarded the terminal's
signals to the detached groups through a JavaScript handler in the
parent. The verifier spends most of its time blocked in `execFileSync`
(the standalone builds, every `npm pack`), where no handler can run, and
installing the handler also replaced the parent's immediate default
termination. So a Ctrl-C during a synchronous phase did nothing until
that command returned, and the detached installs and downloads kept
running meanwhile. Confirmed with the new test against the second
commit's helper: the parent survived the `SIGINT`, exited with code 1
once the synchronous command had died, and its handler never ran.

- `scripts/prefetch-supervisor.mjs` (new): every prefetch command now
runs under this small Node script as the leader of its own process
group. The supervisor holds a pipe from the parent on stdin (which the
parent writes to only to request a stop where it cannot signal the
group; see the fourth commit). The kernel closes it when the parent
exits for any reason, including a signal delivered while the parent is
blocked, or `SIGKILL`, and the supervisor then stops its group
(`SIGTERM`, `SIGKILL` after the grace period) and says so in the log. It
passes group-wide signals on to the command and stays alive until the
command exits, so its own exit reports the command's exit code or
signal. Run by hand with a terminal or `/dev/null` on stdin it watches
nothing.
- `scripts/prefetch-child.mjs`: `startPrefetch` spawns the supervisor
with the command, with stdin as the pipe. The forwarding handlers are
gone; the parent installs no signal handler, so its default termination
stays immediate. `stopPrefetches` is unchanged.
- `scripts/__tests__/prefetch-child.test.mjs`: the first test asserts
the parent has no `SIGINT`/`SIGTERM`/`SIGHUP` listener after a prefetch
starts. The "signal to the parent" test now runs for `SIGINT` and
`SIGKILL` and checks the supervisor's log line. One new test gives a
separate parent its own process group, blocks it in `execFileSync` on a
20s command, and sends `SIGINT` to that group the way a terminal sends
Ctrl-C: the parent must die of `SIGINT` within two seconds and the
prefetch's grandchild must be gone. Against the previous helper the
parent exits with code 1 instead, the grandchild outlives `SIGKILL`, and
the handler-count assertion fails.

### Review fix (fourth commit): a deliberate stop on Windows goes
through the pipe

Greptile pointed out that on Windows, where `startPrefetch` now returns
the supervisor, `stopPrefetches` called `child.kill('SIGTERM')` on it.
Windows has no process groups and `kill` there ends a process outright,
so the supervisor died without running its handler and its command kept
running after cleanup returned.

- `scripts/prefetch-child.mjs`: without process groups, the graceful
stop no longer signals the supervisor. `stopPrefetches` writes a stop
request to the pipe the supervisor already watches for the parent's exit
(`stdin.end('stop\n')`). Only the `SIGKILL` escalation targets the
supervisor itself, after it has had its grace period. POSIX keeps the
group signal, so `stopPrefetches` and the existing stop and failure
tests are unchanged there.
- `scripts/prefetch-supervisor.mjs`: data on stdin is a stop request and
EOF is the parent's exit; both end the group the same way (`SIGTERM`,
`SIGKILL` after the grace period) and the log line names which one it
was.
- `scripts/__tests__/prefetch-child.test.mjs`: one new test writes the
stop request to a running prefetch's pipe instead of calling
`stopPrefetches`. The supervisor must exit with `SIGTERM`, the command's
grandchild must be gone, and the log must say `stop requested` and not
`parent exited`. The previous supervisor fails it on the log line, since
it treated the request as the parent's exit. The Windows kill itself
cannot be reproduced on the Linux runners, so the Windows branch of
`signalTree` is verified by that supervisor test plus reading its two
lines. The stand-in that ignores `SIGTERM` is now skipped on Windows,
where `kill` cannot be ignored.

## Verification

- `node --test scripts/__tests__/stage-bundled-packages.test.mjs
scripts/__tests__/grok-public-install-sandbox.test.mjs
scripts/__tests__/prefetch-child.test.mjs`: 30 tests pass (was 19 before
this PR, 24 after the first commit, 27 after the second, 29 after the
third). The three files pass three consecutive runs.
- `node --check` passes on all four scripts.
- The verifier itself needs Docker and a network, which this workspace
does not have. This PR's own `Build` job runs the branch's verifier.
Measured on all four commits' runs
([38132868803](https://github.com/paperclipai/paperclip/actions/runs/38132868803):
`Build` 428s, `pnpm build` 112s, verifier step 214s;
[38136912842](https://github.com/paperclipai/paperclip/actions/runs/38136912842):
`Build` 447s;
[38139871529](https://github.com/paperclipai/paperclip/actions/runs/38139871529):
`Build` 432s;
[38142762538](https://github.com/paperclipai/paperclip/actions/runs/38142762538):
`Build` 457s, `pnpm build` 123s, verifier step 231s):

| Key | Run 1 | Run 2 | Run 3 | Run 4 | Before |
| --- | --- | --- | --- | --- | --- |
| `stage_packages_ms` | 80.4s | 84.9s | 81.5s | 89.6s | 102-116s on
master runs 38112764439, 38111191543, 38110932348, 38110914319,
38110897745, 38127611057; 84.5s on #15849's run on a runner as fast as
these |
| `stage_sources_ms` | 1.4s | 0.7s | 0.9s | 0.9s | (inside the number
above) |
| `stage_bundled_server_ms` / `adapter_utils` / `db` | 16.3s / 5.0s /
4.6s | 14.9s / 3.6s / 3.3s | 16.4s / 5.2s / 4.7s | 17.1s / 4.4s / 3.8s |
ran one after another, 21.8-25.9s together |
| `stage_bundled_ms` (wall, all three) | 16.4s | 15.0s | 16.5s | 17.2s |
21.8-25.9s |
| `bundled_wait_ms` | 15.3s | 14.7s | 16.1s | 16.7s | — |
| `stage_pack_ms` | 35.8s | 37.1s | 35.6s | 37.9s | (inside the number
above) |
| `consumer_install_ms` | 94.4s | 97.8s | 95.7s | 99.8s | 97-134s |
| `image_prefetch_ms` / `image_wait_ms` | 33.7s / 0ms | 31.8s / 0ms |
34.7s / 0ms | 33.7s / 0ms | unchanged |
| `grok_prefetch_ms` / `grok_wait_ms` | 2.1s / 0ms | 2.1s / 0ms | 2.0s /
0ms | 2.1s / 0ms | unchanged |

The overlap removed about 7-10s from the staging phase on each of the
four runs (21.8-25.9s of installs in 15.0-17.2s of wall time). The third
and fourth commits changed no timing: both waits stay at 0ms and the
staging phase sits inside the band of the first two runs. The rest of
the phase is the standalone-package build and the server UI dist prep
before the loop (about 27-32s) and the eighteen serial `npm pack` plus
`tar -tzf` checks (36-37s).

- `docker-runner-check.yml` runs the same script after `pnpm build` and
needs no change.

## Risks

- Three `npm install` processes share the host npm cache at the same
time. npm's cache (`cacache`) is designed for concurrent writers, and
the three installs have disjoint target directories. If a shared-cache
race appears, the failing install's log tail is in the error and the fix
is to serialize again. The first CI run showed none.
- The install output moves from the CI log into per-install log files in
the scratch root. A failure reports the last 2000 characters of the
failing install's log; a successful install's output is discarded with
the scratch root.
- Memory: three concurrent installs on the `Build` runner instead of
one. The closures are small (the server's eight bundled packages are the
largest), so this stays well under the runner's memory.
- Process groups: a detached child no longer receives the terminal's
signals on its own. The supervisor stops its group when the parent is
gone, which it learns from the kernel closing the stdin pipe, so this
holds for a Ctrl-C while the parent is blocked in synchronous work and
for `SIGKILL` alike. The parent keeps Node's default signal handling. If
the supervisor itself is killed with `SIGKILL` before its command, the
command is orphaned and finishes on its own (each install ends within
seconds; the image pull within a minute), the same as any orphaned
grandchild before this PR.
- One extra Node process per prefetch (five per verifier run). Each
costs about 40ms to start, inside the time the parent spends staging
anyway.
- Low risk otherwise. Packing, the tarball checks, the consumer install,
and every probe are unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.1 (Anthropic), model ID `claude-fable-5-1`, run as a
Paperclip agent through the Claude Code local adapter with tool use
(shell, file edits, GitHub CLI). Reasoning mode: extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green on all four commits
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(on the fourth commit Greptile reviewed 8 files and added no comments;
all three earlier threads are resolved)
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Bender (Fable) <bender-fable@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>

package.json

@@ -61,7 +61,7 @@
     "smoke:posthog-live": "node scripts/smoke/posthog-live.mjs",
     "smoke:pipelines-tutorial": "./scripts/smoke/pipelines-tutorial-smoke.sh",
     "smoke:terminal-bench-loop-skill": "node scripts/smoke/terminal-bench-loop-skill-smoke.mjs",
-    "test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
+    "test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
     "storybook-visual:baseline": "node scripts/storybook-visual-baseline.mjs",
     "test:storybook-visual": "node scripts/storybook-visual-baseline.mjs download && node scripts/storybook-visual-baseline.mjs verify && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts",
     "test:storybook-visual:update": "node scripts/storybook-visual-baseline.mjs download && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts --update-snapshots && node scripts/storybook-visual-baseline.mjs pack",

scripts/__tests__/grok-public-install-sandbox.test.mjs

@@ -110,6 +110,31 @@ test('the image and Grok downloads start before staging and join without weakeni
   assert.equal(source.match(/rmSync\(root/g).length, 1);
 });
 
+test('the bundled-dependency installs run together while the other packages are copied, and pack in order', () => {
+  const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8');
+  const position = fragment => { const index = source.indexOf(fragment); assert.notEqual(index, -1, `verifier must contain ${fragment}`); return index; };
+  // The installs are children of the script's own CLI entry through the helper
+  // covered by stage-bundled-packages.test.mjs, never in-process calls.
+  assert.doesNotMatch(source, /prepareBundledPackage/);
+  assert.ok(source.includes("import { bundledInstallTimings, startBundledInstalls } from './stage-bundled-packages.mjs'"));
+  // Only the bundled packages' sources are staged before the installs start;
+  // the installs are registered for cleanup at once, before anything else can
+  // fail; the other packages are copied while the installs run; the join comes
+  // before the first pack; and the pack loop walks every package in order.
+  const bundledSources = position('for (const entry of entries.filter(entry => entry.bundled)) stageSource(entry)');
+  const start = position("startBundledInstalls(entries.filter(entry => entry.bundled), { logDir: prefetched, cwd: repo })");
+  const registered = position('prefetches.push(...installs.map(({ handle }) => handle))');
+  const otherSources = position('for (const entry of entries.filter(entry => !entry.bundled))');
+  const join = position("await awaited('bundled', Promise.all(installs.map(({ handle }) => handle.done)))");
+  const packLoop = position('for (const { target } of entries)');
+  const firstPack = position("run('npm', ['pack', '--ignore-scripts', '--pack-destination', root], target)");
+  assert.ok(bundledSources < start && start < registered && registered < otherSources && otherSources < join && join < packLoop && packLoop < firstPack);
+  assert.equal(source.match(/run\('npm', \['pack'/g).length, 2, 'one package pack loop plus the lifecycle sentinel');
+  // The timing line reports the split staging phase and each install.
+  for (const key of ['stage_sources_ms', 'stage_bundled_ms', 'stage_pack_ms']) assert.ok(source.includes(`timing.${key} =`));
+  assert.ok(source.includes('...bundledInstallTimings(installs)'));
+});
+
 test('the installed Codex probe exercises the public export and rejects incomplete or mismatched packages', () => {
   const root = realpathSync(mkdtempSync(join(tmpdir(), 'installed-codex-probe-')));
   try {

scripts/__tests__/prefetch-child.test.mjs

@@ -1,5 +1,5 @@
 import assert from 'node:assert/strict';
-import { spawnSync } from 'node:child_process';
+import { spawn, spawnSync } from 'node:child_process';
 import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
@@ -25,6 +25,8 @@ const untilLogHas = async (path, text) => {
 
 test('a successful prefetch resolves, keeps its output, and is left alone by stop', async () => withScratch(async root => {
   const handle = start(root, 'ok', "setTimeout(() => process.stdout.write('fetched\\n'), 150)");
+  // The parent keeps its default, immediate termination: no handler of ours.
+  for (const signal of ['SIGINT', 'SIGTERM', 'SIGHUP']) assert.equal(process.listenerCount(signal), 0, `${signal} handler installed in the parent`);
   await handle.done;
   assert.equal(handle.child.exitCode, 0);
   assert.equal(readFileSync(join(root, 'ok.log'), 'utf8'), 'fetched\n');
@@ -66,7 +68,7 @@ test('stopping a running prefetch waits for it to exit so its directory can be r
   await assert.rejects(handle.done, /slow prefetch failed \(SIGTERM\): started/);
 }));
 
-test('a prefetch that ignores SIGTERM is killed after the grace period', async () => withScratch(async root => {
+test('a prefetch that ignores SIGTERM is killed after the grace period', { skip: process.platform === 'win32' && 'SIGTERM cannot be ignored on Windows' }, async () => withScratch(async root => {
   const handle = start(root, 'stubborn', "process.on('SIGTERM', () => {}); process.stdout.write('ignoring\\n'); setInterval(() => {}, 1000)");
   await untilLogHas(join(root, 'stubborn.log'), 'ignoring');
   const stopping = Date.now();
@@ -75,6 +77,97 @@ test('a prefetch that ignores SIGTERM is killed after the grace period', async (
   assert.equal(handle.child.signalCode, 'SIGKILL');
 }));
 
+// A process is gone once the kernel no longer knows it, or once it is a zombie
+// that only waits to be reaped: either way it can no longer write anywhere.
+const gone = pid => {
+  try { process.kill(pid, 0); } catch (error) { return error.code === 'ESRCH'; }
+  try { return /^\d+ \(.*\) Z /.test(readFileSync(`/proc/${pid}/stat`, 'utf8')); } catch { return false; }
+};
+const untilGone = async pid => {
+  for (let attempt = 0; attempt < 200; attempt++) {
+    if (gone(pid)) return true;
+    await new Promise(resolve => setTimeout(resolve, 25));
+  }
+  return false;
+};
+// The child starts a process of its own that announces its pid and then runs
+// on, the way the bundled-dependency installs block in `npm install`.
+const GRANDCHILD_SCRIPT = `
+  const { spawn } = require('node:child_process');
+  const grandchild = spawn(process.execPath, ['-e', "process.stdout.write('grandchild ' + process.pid + '\\\\n'); setInterval(() => {}, 1000)"], { stdio: 'inherit' });
+  grandchild.on('exit', () => process.exit(0));
+`;
+const grandchildPid = async log => { await untilLogHas(log, 'grandchild '); return Number(readFileSync(log, 'utf8').match(/grandchild (\d+)/)[1]); };
+
+test('stopping a prefetch also stops the processes it started', { skip: process.platform === 'win32' && 'process groups are POSIX only' }, async () => withScratch(async root => {
+  const handle = start(root, 'tree', GRANDCHILD_SCRIPT);
+  const pid = await grandchildPid(join(root, 'tree.log'));
+  assert.ok(!gone(pid), 'the grandchild runs before the stop');
+  await stopPrefetches([handle]);
+  assert.ok(await untilGone(pid), `grandchild ${pid} outlived stopPrefetches`);
+  assert.notEqual(handle.child.signalCode ?? handle.child.exitCode, null);
+}));
+
+test('a stop request written to the pipe ends the command and the processes it started, with no signal to the supervisor', { skip: process.platform === 'win32' && 'the grandchild check reads /proc' }, async () => withScratch(async root => {
+  // This is the stop path on a platform without process groups: there `kill`
+  // would end the supervisor outright and leave its command running, so
+  // `stopPrefetches` writes a request instead. The supervisor answers it
+  // the way it answers the parent's exit, and its own exit still reports the
+  // command's fate.
+  const handle = start(root, 'tree', GRANDCHILD_SCRIPT);
+  const pid = await grandchildPid(join(root, 'tree.log'));
+  handle.child.stdin.end('stop\n');
+  await assert.rejects(handle.done, /tree prefetch failed \(SIGTERM\)/);
+  assert.ok(await untilGone(pid), `grandchild ${pid} outlived the stop request`);
+  assert.match(readFileSync(join(root, 'tree.log'), 'utf8'), /stop requested; stopping/);
+  assert.doesNotMatch(readFileSync(join(root, 'tree.log'), 'utf8'), /parent exited/);
+  await stopPrefetches([handle]);
+}));
+
+// A separate parent process stands in for the verifier: it starts a prefetch
+// whose child starts a grandchild, then does `work` (a module-scope statement).
+const parentScript = (root, work) => `
+  import { execFileSync } from 'node:child_process';
+  import { startPrefetch } from ${JSON.stringify(new URL('../prefetch-child.mjs', import.meta.url).href)};
+  startPrefetch({ name: 'tree', cmd: process.execPath, args: ['-e', ${JSON.stringify(GRANDCHILD_SCRIPT)}], cwd: ${JSON.stringify(root)}, env: process.env, logPath: ${JSON.stringify(join(root, 'tree.log'))} });
+  ${work}
+`;
+
+for (const signal of ['SIGINT', 'SIGKILL']) {
+  test(`${signal} to the parent ends the processes a prefetch started, and the parent dies of it at once`, { skip: process.platform === 'win32' && 'process groups are POSIX only' }, async () => withScratch(async root => {
+    // The parent runs no handler, so it dies of the signal immediately, and the
+    // prefetch tree stops because the parent is gone, not because it was told:
+    // SIGKILL leaves the parent no chance to tell anyone anything.
+    const parent = spawn(process.execPath, ['--input-type=module', '-e', parentScript(root, 'setInterval(() => {}, 1000);')], { cwd: root, stdio: 'ignore' });
+    const exited = new Promise(resolve => parent.once('exit', (code, signal) => resolve({ code, signal })));
+    const pid = await grandchildPid(join(root, 'tree.log'));
+    parent.kill(signal);
+    assert.deepEqual(await exited, { code: null, signal });
+    assert.ok(await untilGone(pid), `grandchild ${pid} outlived the parent's ${signal}`);
+    assert.match(readFileSync(join(root, 'tree.log'), 'utf8'), /parent exited; stopping/);
+  }));
+}
+
+test('Ctrl-C while the parent is blocked in synchronous work stops it at once, and the prefetch tree with it', { skip: process.platform === 'win32' && 'process groups are POSIX only' }, async () => withScratch(async root => {
+  // The verifier runs `execFileSync` for the standalone builds while prefetches
+  // run, and a JavaScript signal handler cannot run until that returns. The
+  // parent gets its own group here so the signal can be sent the way a
+  // terminal sends Ctrl-C: to the foreground group, which is the parent and
+  // its synchronous command, and not the detached prefetch tree.
+  const parent = spawn(process.execPath, ['--input-type=module', '-e', parentScript(root, "execFileSync(process.execPath, ['-e', 'setTimeout(() => {}, 20000)'], { stdio: 'ignore' });")], { cwd: root, stdio: 'ignore', detached: true });
+  try {
+    const exited = new Promise(resolve => parent.once('exit', (code, signal) => resolve({ code, signal })));
+    const pid = await grandchildPid(join(root, 'tree.log'));
+    const signalled = Date.now();
+    process.kill(-parent.pid, 'SIGINT');
+    assert.deepEqual(await exited, { code: null, signal: 'SIGINT' });
+    assert.ok(Date.now() - signalled < 2000, `the parent took ${Date.now() - signalled}ms to die: it waited for the synchronous command`);
+    assert.ok(await untilGone(pid), `grandchild ${pid} outlived the parent's Ctrl-C`);
+  } finally {
+    try { process.kill(-parent.pid, 'SIGKILL'); } catch {}
+  }
+}));
+
 test('an unawaited failure never becomes an unhandled rejection', () => {
   const script = `
     import { startPrefetch } from ${JSON.stringify(new URL('../prefetch-child.mjs', import.meta.url).href)};

scripts/__tests__/stage-bundled-packages.test.mjs

@@ -0,0 +1,104 @@
+import assert from 'node:assert/strict';
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import test from 'node:test';
+import { stopPrefetches } from '../prefetch-child.mjs';
+import { PREPARE_BUNDLED_PACKAGE_SCRIPT, bundledInstallTimings, bundledPackageSlug, startBundledInstalls } from '../stage-bundled-packages.mjs';
+
+// A small local script stands in for `prepare-bundled-package.mjs`; no test
+// here needs npm or a network.
+const withScratch = async body => {
+  const root = mkdtempSync(join(tmpdir(), 'stage-bundled-packages-'));
+  try { await body(root); } finally { rmSync(root, { recursive: true, force: true }); }
+};
+const entriesFor = (root, names) => names.map((name, index) => ({ name, stagedSource: join(root, `source-${index}`), target: join(root, `package-${index}`) }));
+const fakeScript = (root, body) => { const script = join(root, 'fake-prepare.mjs'); writeFileSync(script, body); return script; };
+
+test('package names map to stable log and timing slugs', () => {
+  assert.equal(bundledPackageSlug('@paperclipai/server'), 'server');
+  assert.equal(bundledPackageSlug('@paperclipai/adapter-utils'), 'adapter_utils');
+  assert.equal(bundledPackageSlug('@paperclipai/db'), 'db');
+});
+
+test('the children run the CLI entry release.sh uses, never an in-process call', () => {
+  assert.ok(existsSync(PREPARE_BUNDLED_PACKAGE_SCRIPT));
+  const source = readFileSync(PREPARE_BUNDLED_PACKAGE_SCRIPT, 'utf8');
+  assert.ok(source.includes('if (process.argv[1] === fileURLToPath(import.meta.url))'));
+  assert.ok(source.includes('prepareBundledPackage(resolve(sourceDir), resolve(destinationDir))'));
+  const release = readFileSync(new URL('../release.sh', import.meta.url), 'utf8');
+  assert.match(release, /node "\$REPO_ROOT\/scripts\/prepare-bundled-package\.mjs" "\$REPO_ROOT\/\$pkg_dir" "\$publish_dir"/);
+});
+
+test('the installs run at the same time, each with its own source and target', async () => withScratch(async root => {
+  // Each child records its arguments, announces itself, and exits only once
+  // the other child has announced itself too. Serial children would deadlock
+  // here and the five-second guard would turn that into a failure.
+  const script = fakeScript(root, `
+    import { existsSync, writeFileSync } from 'node:fs';
+    import { basename, join } from 'node:path';
+    const [source, target] = process.argv.slice(2);
+    const root = join(target, '..');
+    writeFileSync(join(root, 'started-' + basename(target)), '');
+    console.log('install ' + source + ' -> ' + target);
+    const other = basename(target) === 'package-0' ? 'package-1' : 'package-0';
+    const deadline = Date.now() + 5000;
+    while (!existsSync(join(root, 'started-' + other))) {
+      if (Date.now() > deadline) { console.error('the other install never started'); process.exit(3); }
+      Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 20);
+    }
+    writeFileSync(join(target, 'package.json'), '{}');
+  `);
+  const entries = entriesFor(root, ['@paperclipai/server', '@paperclipai/db']);
+  for (const { target } of entries) mkdirSync(target);
+  const installs = startBundledInstalls(entries, { logDir: root, cwd: root, script });
+  assert.deepEqual(installs.map(({ slug }) => slug), ['server', 'db']);
+  await Promise.all(installs.map(({ handle }) => handle.done));
+  for (const [index, { stagedSource, target, slug, handle }] of installs.entries()) {
+    assert.equal(handle.child.exitCode, 0);
+    assert.equal(readFileSync(join(root, `bundled-${slug}.log`), 'utf8'), `install ${stagedSource} -> ${target}\n`);
+    assert.equal(target, join(root, `package-${index}`));
+    assert.ok(existsSync(join(target, 'package.json')));
+  }
+  const timings = bundledInstallTimings(installs);
+  assert.deepEqual(Object.keys(timings), ['stage_bundled_server_ms', 'stage_bundled_db_ms']);
+  for (const value of Object.values(timings)) assert.ok(Number.isInteger(value) && value >= 0);
+}));
+
+test('stopping the installs after one fails also stops the npm process the other started', { skip: process.platform === 'win32' && 'process groups are POSIX only' }, async () => withScratch(async root => {
+  // The real wrapper blocks in `execFileSync('npm', ['install', ...])`; here
+  // the stand-in npm announces its pid and runs on, and the wrapper blocks on
+  // it the same way. Stopping only the wrapper would leave it writing.
+  const script = fakeScript(root, `
+    import { execFileSync } from 'node:child_process';
+    const [, target] = process.argv.slice(2);
+    if (target.endsWith('package-1')) { console.error('bundled install exploded'); process.exit(2); }
+    execFileSync(process.execPath, ['-e', "process.stdout.write('npm ' + process.pid + '\\\\n'); setInterval(() => {}, 1000)"], { stdio: 'inherit' });
+  `);
+  const installs = startBundledInstalls(entriesFor(root, ['@paperclipai/server', '@paperclipai/db']), { logDir: root, cwd: root, script });
+  await assert.rejects(Promise.all(installs.map(({ handle }) => handle.done)), /bundled-db prefetch failed \(exit 2\)/);
+  const log = join(root, 'bundled-server.log');
+  const logHasPid = () => existsSync(log) && /npm \d+/.test(readFileSync(log, 'utf8'));
+  for (let attempt = 0; attempt < 200 && !logHasPid(); attempt++) await new Promise(resolve => setTimeout(resolve, 25));
+  const npmPid = Number(readFileSync(log, 'utf8').match(/npm (\d+)/)[1]);
+  const alive = pid => { try { process.kill(pid, 0); return !/^\d+ \(.*\) Z /.test(readFileSync(`/proc/${pid}/stat`, 'utf8')); } catch (error) { return error.code !== 'ESRCH'; } };
+  assert.ok(alive(npmPid), 'the stand-in npm runs before the stop');
+  await stopPrefetches(installs.map(({ handle }) => handle));
+  let stopped = false;
+  for (let attempt = 0; attempt < 200 && !(stopped = !alive(npmPid)); attempt++) await new Promise(resolve => setTimeout(resolve, 25));
+  assert.ok(stopped, `stand-in npm ${npmPid} outlived stopPrefetches`);
+  assert.notEqual(installs[0].handle.child.signalCode ?? installs[0].handle.child.exitCode, null);
+}));
+
+test('a failed install rejects with the tail of its own log and leaves the others to be stopped', async () => withScratch(async root => {
+  const script = fakeScript(root, `
+    const [, target] = process.argv.slice(2);
+    if (target.endsWith('package-1')) { console.error('bundled install exploded'); process.exit(2); }
+    setTimeout(() => {}, 30_000);
+  `);
+  const installs = startBundledInstalls(entriesFor(root, ['@paperclipai/server', '@paperclipai/adapter-utils']), { logDir: root, cwd: root, script });
+  await assert.rejects(Promise.all(installs.map(({ handle }) => handle.done)), /bundled-adapter_utils prefetch failed \(exit 2\): bundled install exploded/);
+  assert.equal(installs[0].handle.child.exitCode, null, 'the other install is still running when the failure surfaces');
+  await stopPrefetches(installs.map(({ handle }) => handle));
+  assert.notEqual(installs[0].handle.child.signalCode ?? installs[0].handle.child.exitCode, null);
+}));

scripts/prefetch-child.mjs

@@ -2,23 +2,72 @@
 // output in a log file, join it where the result is needed, and stop it before
 // the scratch directory it writes into is removed. No shell, no inherited
 // terminal, and a failure is reported where the caller awaits it.
+//
+// A child may start processes of its own: the bundled-dependency installs are
+// a Node wrapper blocked in `npm install`. Signalling only the wrapper would
+// leave npm writing into the scratch root while it is removed, so on POSIX
+// each command runs under `prefetch-supervisor.mjs` as the leader of its own
+// process group, and stopping signals and waits for the whole group. A group
+// of its own no longer receives the terminal's Ctrl-C, and the parent cannot
+// forward it: the verifier spends most of its time blocked in `execFileSync`,
+// where no JavaScript signal handler can run, and installing one would also
+// replace the parent's immediate default termination. Instead the supervisor
+// holds a pipe from the parent and stops its group when the pipe closes, which
+// the kernel does when the parent exits for any reason. The parent installs
+// no signal handlers.
+//
+// Windows has no process groups and `kill` there ends a process outright, so a
+// signalled supervisor could pass nothing on and its command would run on
+// after cleanup returned. There a deliberate stop goes through the same pipe
+// as a written request, which the supervisor answers by ending its command,
+// and only the SIGKILL escalation targets the supervisor itself.
 import { spawn } from 'node:child_process';
 import { closeSync, openSync, readFileSync, statSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
 
+export const PREFETCH_SUPERVISOR_SCRIPT = fileURLToPath(new URL('./prefetch-supervisor.mjs', import.meta.url));
+
+const GROUPS = process.platform !== 'win32';
 const running = child => child.pid !== undefined && child.exitCode === null && child.signalCode === null;
+const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
 
-// Returns { name, child, done, durationMs }. `done` resolves on exit code 0
+// Signals the child's whole process group (POSIX). Without groups the
-// and otherwise rejects with the tail of the log; the rejection is observed
+// supervisor is asked to stop its command through the pipe it watches, since
-// here so an unawaited failure never surfaces as an unhandled rejection.
+// `kill` would end the supervisor at once and leave the command running; only
+// SIGKILL is sent to the supervisor itself, after it has had its chance.
+const signalTree = (child, signal) => {
+  if (child.pid === undefined) return;
+  if (!GROUPS) {
+    if (!running(child)) return;
+    if (signal === 'SIGKILL') child.kill(signal);
+    else if (child.stdin?.writable) child.stdin.end('stop\n');
+    return;
+  }
+  try { process.kill(-child.pid, signal); } catch (error) { if (error.code !== 'ESRCH') throw error; }
+};
+// True while any member of the child's group exists. On a platform without
+// groups only the child itself is known.
+const treeAlive = child => {
+  if (child.pid === undefined) return false;
+  if (!GROUPS) return running(child);
+  try { process.kill(-child.pid, 0); return true; } catch (error) { return error.code !== 'ESRCH'; }
+};
+
+// Returns { name, child, done, durationMs }. `child` is the supervisor, whose
+// exit code or signal is the command's. `done` resolves on exit code 0 and
+// otherwise rejects with the tail of the log; the rejection is observed here
+// so an unawaited failure never surfaces as an unhandled rejection.
 export function startPrefetch({ name, cmd, args, cwd, env, logPath }) {
   const startedAt = Date.now();
   const log = openSync(logPath, 'w');
   let child;
   try {
-    child = spawn(cmd, args, { cwd, env, stdio: ['ignore', log, log] });
+    // stdin is the pipe the supervisor watches; nothing is ever written to it.
+    child = spawn(process.execPath, [PREFETCH_SUPERVISOR_SCRIPT, cmd, ...args], { cwd, env, stdio: ['pipe', log, log], detached: GROUPS });
   } finally {
     closeSync(log); // the child holds its own descriptor
   }
+  child.stdin?.on('error', () => {});
   const handle = { name, child, done: null, durationMs: null };
   let settled = false;
   handle.done = new Promise((settle, reject) => {
@@ -48,13 +97,22 @@ export function startPrefetch({ name, cmd, args, cwd, env, logPath }) {
   return handle;
 }
 
-// Stops every prefetch that is still running and resolves only once all of
+// Stops every prefetch that is still running, together with any process it
-// them have exited, so the caller can remove their working directory without
+// started, and resolves only once all of them have exited, so the caller can
-// racing a write. A child that ignores SIGTERM is killed after `graceMs`.
+// remove their working directory without racing a write. A tree that ignores
+// SIGTERM is killed after `graceMs`; one that still exists `graceMs` after
+// that is given up on so cleanup cannot hang.
 export async function stopPrefetches(prefetches, { graceMs = 5000 } = {}) {
   await Promise.all(prefetches.map(async ({ child, done }) => {
-    if (running(child)) child.kill('SIGTERM');
+    const started = Date.now();
-    const escalate = setTimeout(() => { if (running(child)) child.kill('SIGKILL'); }, graceMs);
+    let termed = false, killed = false;
-    try { await done; } catch {} finally { clearTimeout(escalate); }
+    while (treeAlive(child)) {
+      const elapsed = Date.now() - started;
+      if (!termed) { signalTree(child, 'SIGTERM'); termed = true; }
+      else if (!killed && elapsed >= graceMs) { signalTree(child, 'SIGKILL'); killed = true; }
+      else if (killed && elapsed >= 2 * graceMs) break;
+      await sleep(25);
+    }
+    try { await done; } catch {}
   }));
 }

scripts/prefetch-supervisor.mjs

@@ -0,0 +1,78 @@
+// Child side of prefetch-child.mjs: runs one prefetch command as the leader of
+// its own process group and stops the whole group when the parent that started
+// it is gone.
+//
+// The parent holds the write end of this process's stdin. The kernel closes
+// that end when the parent exits for any reason: a Ctrl-C delivered while the
+// parent is blocked in synchronous work, a plain `kill`, or SIGKILL. Reading
+// EOF here is therefore the only parent-death signal needed, and the parent
+// runs no signal handler of its own, so its default termination stays
+// immediate. The parent writes to the pipe for one reason only: to ask for a
+// deliberate stop where it cannot signal the group, which is Windows, where
+// `kill` ends a process outright and a signalled supervisor could not pass
+// anything on. Group-wide signals the parent sends through `stopPrefetches`
+// on POSIX reach this process too; it passes them on to the command and stays
+// alive until the command exits, so its own exit reports the command's
+// outcome (exit code, or the same signal).
+import { spawn } from 'node:child_process';
+import { fstatSync } from 'node:fs';
+import { constants } from 'node:os';
+
+const GROUPS = process.platform !== 'win32';
+const GRACE_MS = Number(process.env.PAPERCLIP_PREFETCH_GRACE_MS) || 5000;
+const [cmd, ...args] = process.argv.slice(2);
+if (!cmd) {
+  process.stderr.write('prefetch supervisor: usage: prefetch-supervisor.mjs <command> [args...]\n');
+  process.exit(64);
+}
+
+const child = spawn(cmd, args, { stdio: ['ignore', 'inherit', 'inherit'] });
+const signalChild = signal => { try { child.kill(signal); } catch {} };
+// On POSIX this process leads the group (the parent starts it detached), so
+// the group id is its own pid. Without groups only the command is known.
+const signalGroup = signal => {
+  if (!GROUPS) return signalChild(signal);
+  try { process.kill(-process.pid, signal); } catch (error) { if (error.code === 'ESRCH') signalChild(signal); }
+};
+
+// A signal to the group arrives here as well as at the command; passing it on
+// is harmless when the command already has it, and covers a signal sent to
+// this process alone. The parent owns the SIGKILL escalation in that case.
+for (const signal of ['SIGINT', 'SIGTERM', 'SIGHUP']) process.on(signal, () => signalChild(signal));
+
+let stopping = false;
+const stopGroup = reason => {
+  if (stopping) return;
+  stopping = true;
+  process.stderr.write(`prefetch supervisor: ${reason}; stopping ${cmd}\n`);
+  signalGroup('SIGTERM');
+  setTimeout(() => signalGroup('SIGKILL'), GRACE_MS).unref();
+};
+// Only a pipe can carry a stop request or report the parent's exit (Node
+// creates its stdio pipes as socket pairs). Run by hand with a terminal or
+// /dev/null on stdin, there is no parent to watch and nothing is read.
+let stdinIsPipe = false;
+try { const stdin = fstatSync(0); stdinIsPipe = stdin.isFIFO() || stdin.isSocket(); } catch {}
+if (stdinIsPipe) {
+  process.stdin.on('data', () => stopGroup('stop requested'));
+  const parentGone = () => stopGroup('parent exited');
+  process.stdin.on('end', parentGone);
+  process.stdin.on('close', parentGone);
+  process.stdin.on('error', parentGone);
+  process.stdin.resume();
+}
+
+child.on('error', error => {
+  process.stderr.write(`prefetch supervisor: ${error.message}\n`);
+  process.exit(127);
+});
+child.on('exit', (code, signal) => {
+  if (signal !== null) {
+    // Die of the same signal, so the parent sees the command's fate on this
+    // process. Dropping the listener restores the default disposition first.
+    process.removeAllListeners(signal);
+    try { process.kill(process.pid, signal); } catch {}
+    process.exit(128 + (constants.signals[signal] ?? 1));
+  }
+  process.exit(code ?? 1);
+});

scripts/stage-bundled-packages.mjs

@@ -0,0 +1,32 @@
+// The public-install verifier stages three packages that bundle dependencies
+// (`@paperclipai/server`, `@paperclipai/adapter-utils`, `@paperclipai/db`).
+// Each one is a host-side `npm install` of that package's bundled closure into
+// its own pack directory, and none reads what another produces, so they run as
+// concurrent children instead of one after another inside the staging loop.
+// Each child is the CLI entry of `prepare-bundled-package.mjs`, the same path
+// release.sh runs, with its output in a log file that a failure reports.
+import { join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { startPrefetch } from './prefetch-child.mjs';
+
+export const PREPARE_BUNDLED_PACKAGE_SCRIPT = fileURLToPath(new URL('./prepare-bundled-package.mjs', import.meta.url));
+
+// `@paperclipai/adapter-utils` -> `adapter_utils`, for log names and timing keys.
+export const bundledPackageSlug = name => name.replace(/^@paperclipai\//, '').replace(/[^a-z0-9]+/gi, '_');
+
+// Starts one child per entry (`{ name, stagedSource, target }`) at once and
+// returns the entries with their `slug` and prefetch `handle`. The caller joins
+// `handle.done` where the packed output is needed and registers the handles
+// for cleanup, so an earlier failure stops every install still running.
+export function startBundledInstalls(entries, { logDir, cwd, env = process.env, script = PREPARE_BUNDLED_PACKAGE_SCRIPT }) {
+  return entries.map(entry => {
+    const slug = bundledPackageSlug(entry.name);
+    const handle = startPrefetch({ name: `bundled-${slug}`, cmd: process.execPath, args: [script, entry.stagedSource, entry.target], cwd, env, logPath: join(logDir, `bundled-${slug}.log`) });
+    return { ...entry, slug, handle };
+  });
+}
+
+// Per-package wall time for the timing line. The helper measures from the
+// start to the child's last output; the parent is idle while it waits for the
+// installs, so the exit is observed as it happens.
+export const bundledInstallTimings = installs => Object.fromEntries(installs.map(({ slug, handle }) => [`stage_bundled_${slug}_ms`, handle.durationMs]));

scripts/verify-grok-npm-install.mjs

@@ -8,10 +8,11 @@ import { chmodSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, re
 import { tmpdir } from 'node:os';
 import { basename, dirname, join, resolve } from 'node:path';
 import { fileURLToPath } from 'node:url';
-import { materializePublishManifest, prepareBundledPackage } from './prepare-bundled-package.mjs';
+import { materializePublishManifest } from './prepare-bundled-package.mjs';
 import { GROK_PUBLIC_INSTALL_IMAGE, GROK_PUBLIC_INSTALL_LIFECYCLE, assertNoBundledCodexPayloads, grokConsumerDockerArgs, installedCodexProbeSource } from './grok-public-install-sandbox.mjs';
 import { startPrefetch, stopPrefetches } from './prefetch-child.mjs';
 import { retainRunnerQualificationPackages } from './retain-runner-qualification-packages.mjs';
+import { bundledInstallTimings, startBundledInstalls } from './stage-bundled-packages.mjs';
 const repo = resolve(dirname(fileURLToPath(import.meta.url)), '..');
 assert.equal(process.platform, 'linux', 'Run this verification on disposable EC2 Linux, not a developer host');
 const root = mkdtempSync(join(tmpdir(), 'paperclip-grok-public-install-'));
@@ -28,13 +29,18 @@ const releaseVersion = `0.0.0-grok-verify.${sourceRevision.slice(0, 12)}`;
 // `native/grok` path that the post-install assertion inspects.
 const prefetched = join(root, 'prefetch'); mkdirSync(prefetched);
 const stagedGrok = join(prefetched, 'grok');
+// Every background child, downloads and bundled-dependency installs alike, is
+// stopped in `finally`; `downloads` is the subset the timing line reports as
+// prefetches.
 const prefetches = [];
+const downloads = [];
 const timing = {};
 // A prefetch failure surfaces where its result is awaited, never as an
 // unhandled rejection that would mask an earlier staging error.
 const prefetch = (name, cmd, args) => {
   const handle = startPrefetch({ name, cmd, args, cwd: root, env, logPath: join(prefetched, `${name}.log`) });
   prefetches.push(handle);
+  downloads.push(handle);
   return handle.done;
 };
 let phaseStarted = performance.now();
@@ -61,11 +67,21 @@ try {
   // Source manifests remain untouched, including independently versioned SDKs.
   run(process.execPath, [join(repo, 'scripts/build-standalone-public-packages.mjs')], repo);
   run('bash', [join(repo, 'scripts/prepare-server-ui-dist.sh')], repo);
-  const tarballs = [];
+  // Each package's published files are staged into `source-<index>` and its
-  for (const [index, name] of [...needed].entries()) {
+  // pack input into `package-<index>`, with <index> in dependency order so the
+  // tarball list keeps matching `needed`. The packages that bundle dependencies
+  // each need a host-side `npm install` of their bundled closure; those installs
+  // write only their own `package-<index>` and read nothing another package
+  // produces, so they start first and run together while the other packages
+  // are copied, instead of one after another inside one loop. Packing stays
+  // serial and in order, because each tarball is checked as it is produced.
+  const entries = [...needed].map((name, index) => {
     const { dir, manifest } = packages.get(name);
-    const target = join(root, `package-${index}`); mkdirSync(target);
+    const bundled = (manifest.bundleDependencies ?? manifest.bundledDependencies ?? []).length > 0;
-    const stagedSource = join(root, `source-${index}`); mkdirSync(stagedSource);
+    return { name, dir, manifest, bundled, target: join(root, `package-${index}`), stagedSource: join(root, `source-${index}`) };
+  });
+  const stageSource = ({ dir, manifest, stagedSource }) => {
+    mkdirSync(stagedSource);
     for (const file of manifest.files ?? ['dist']) {
       // release.sh stages runtime skills into these public packages before pack.
       const releaseSkills = file === 'skills' && ['server', 'packages/adapters/claude-local', 'packages/adapters/codex-local'].includes(dir);
@@ -76,12 +92,25 @@ try {
     }
     const releaseManifest = { ...manifest, version: releaseVersion };
     writeFileSync(join(stagedSource, 'package.json'), JSON.stringify(releaseManifest));
-    if ((manifest.bundleDependencies ?? manifest.bundledDependencies ?? []).length) {
+    return releaseManifest;
-      prepareBundledPackage(stagedSource, target);
+  };
-    } else {
+  const sourcesStarted = performance.now();
-      cpSync(stagedSource, target, { recursive: true });
+  for (const entry of entries.filter(entry => entry.bundled)) stageSource(entry);
-      writeFileSync(join(target, 'package.json'), JSON.stringify(materializePublishManifest(releaseManifest)));
+  const installsStarted = performance.now();
-    }
+  const installs = startBundledInstalls(entries.filter(entry => entry.bundled), { logDir: prefetched, cwd: repo });
+  prefetches.push(...installs.map(({ handle }) => handle));
+  for (const entry of entries.filter(entry => !entry.bundled)) {
+    const releaseManifest = stageSource(entry);
+    mkdirSync(entry.target);
+    cpSync(entry.stagedSource, entry.target, { recursive: true });
+    writeFileSync(join(entry.target, 'package.json'), JSON.stringify(materializePublishManifest(releaseManifest)));
+  }
+  timing.stage_sources_ms = Math.round(performance.now() - sourcesStarted);
+  await awaited('bundled', Promise.all(installs.map(({ handle }) => handle.done)));
+  timing.stage_bundled_ms = Math.round(performance.now() - installsStarted);
+  const packStarted = performance.now();
+  const tarballs = [];
+  for (const { target } of entries) {
     run('npm', ['pack', '--ignore-scripts', '--pack-destination', root], target);
     const packed = readdirSync(root).filter(f => f.endsWith('.tgz') && !tarballs.includes(join(root, f)));
     assert.equal(packed.length, 1);
@@ -89,6 +118,7 @@ try {
     assertNoBundledCodexPayloads(run('tar', ['-tzf', tarball]).toString().trim().split('\n'));
     tarballs.push(tarball);
   }
+  timing.stage_pack_ms = Math.round(performance.now() - packStarted);
   const assets = join(root, 'assets'); mkdirSync(assets, { mode: 0o755 });
   const consumer = join(root, 'consumer'); mkdirSync(consumer);
   const cache = join(root, 'cache'); mkdirSync(cache);
@@ -184,8 +214,10 @@ try {
   // without re-running it. `*_prefetch_ms` runs from a download's start to its
   // last output, and `*_wait_ms` is how long the main flow waited for a
   // prefetch that had not finished yet; zero means the overlap hid it entirely.
-  const downloads = Object.fromEntries(prefetches.map(({ name, durationMs }) => [`${name}_prefetch_ms`, durationMs]));
+  // `stage_sources_ms`, `stage_bundled_ms` and `stage_pack_ms` split the
-  console.log(JSON.stringify({ schema: 'paperclip.public-npm-install.timing.v1', ...timing, ...downloads }));
+  // staging phase, and `stage_bundled_<package>_ms` is each concurrent install.
+  const downloaded = Object.fromEntries(downloads.map(({ name, durationMs }) => [`${name}_prefetch_ms`, durationMs]));
+  console.log(JSON.stringify({ schema: 'paperclip.public-npm-install.timing.v1', ...timing, ...downloaded, ...bundledInstallTimings(installs) }));
   if (process.env.PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR) {
     const retained = retainRunnerQualificationPackages({ repo, output: process.env.PAPERCLIP_RUNNER_QUALIFICATION_PACKAGES_DIR, sourceRevision, releaseVersion, env,
       publicArchives: [...needed].map((name, index) => ({ name, file: tarballs[index] })),