Files
PaperClipAI/packages/paperclip-runner/scripts/run-pr-vitest-lane.mjs
T
352153b5ed fix: run the cargo-building native-runner CI suite in the Rust-cached vitest lane (#13586)
## Thinking Path

Post-merge of #13557, the slowest check on the freshest fully-green PR
run
([35246999382](https://github.com/paperclipai/paperclip/actions/runs/35246999382))
was `ci / General tests (server (1/12))` at **339s**. The cause is one
suite:
`server/src/services/native-runtime/native-codex-runner.integration.test.ts`
runs 1 test in **277s of a 291s vitest step (95%)** because its
`beforeAll` cargo-builds the Runner release binaries, and the
general-server shards carry no Rust cache — every PR run cold-compiles
the full third-party crate graph. The other 19 suites in that shard
finish in under 70ms each.

The obvious fix (a dedicated Rust-cached matrix lane) requires editing
workflow files, which the available GitHub App credentials cannot push
(`workflows` permission). But the `Verify Paperclip Runner` lanes
**already restore the shared `release-runner-v1` Rust cache read-only**,
and their commands are `pnpm --filter @paperclipai/paperclip-runner
<package script>` — so the suite can move into a Rust-cached lane purely
through script changes.

## What Changed

- `scripts/run-vitest-stable.mjs`: new `general-server-native-runner`
group carrying exactly that suite. Under the PR workflow
(`GITHUB_WORKFLOW == "PR"`, inherited from `pr.yml` by the reusable
`pr-trusted.yml`) the without-chat server shards exclude it and
rebalance to ~211s of tests each. Every other caller — local runs,
`release-verify.yml` under the Release / Cloud readiness workflows —
keeps the suite in the shards, so a renamed or unknown workflow degrades
to today's slower-but-covered behavior instead of dropping coverage.
- `packages/paperclip-runner`: `test:typescript:vitest` now routes
through `scripts/run-pr-vitest-lane.mjs` — the identical
`ensure:eval-build-deps && build:rust && vitest run` chain (shard flags
passed through), plus the native-runner group on the **final PR shard
only** (`--shard=N/M` with `N == M`, i.e. today's `vitest 2/2`, the 122s
lane). With the restored cache the suite's cargo build becomes an
incremental rebuild.
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: guards pin the
whole contract — PR 12-shard coverage (shards + chat + native-runner =
full server group exactly), Release/local 10-shard runs keep the suite,
`pr.yml` is named `PR`, the vitest lanes partition with exactly one
final shard, the package-script wiring, and the wrapper's shard/workflow
gating via its `--dry-run` plan output.

No workflow files change. `.github/workflows/*` are untouched.

## Verification

- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs`: **36/36 pass**
locally on this branch (includes the new coverage, wiring, and
wrapper-gating guards). Both files run in CI's `Test general-server
shard partition` / `Test release verify workflow wiring` steps.
- Wrapper `--dry-run` plan matrix verified for all six shard/workflow
combinations plus malformed-shard rejection (pinned as a guard test).
- The executing proof is this PR's own CI: `ci / Verify Paperclip Runner
(vitest 2/2)` must go green while running the native-runner suite (its
log will show the `general-server-native-runner` group after the package
vitest shard), and the 12 `ci / General tests (server (x/12))` shards
must go green without it.

## Risks

- The exclusion keys on `GITHUB_WORKFLOW == "PR"`. Failure mode of a
rename is safe (suite falls back into the server shards, slower but
covered) and the guard test on `pr.yml`'s name makes it loud.
- `vitest 2/2` grows from ~122s to an expected ~210–260s — still well
under the ~306s `vitest 1/2` and ~326s e2e shards, and inside the
20-minute lane timeout. If the cache misses (key drift), the lane pays a
cold compile like the server shard does today; a miss is slow, never
wrong.
- Double-run/coverage-loss combinations are enumerated in the wrapper
header and pinned by tests: each caller runs the suite exactly once.

## Model Used

Claude (Bender agent, Paperclip) — Fable 5.

---
Expected savings once merged: the 339s `server (1/12)` check drops to
~265s-equivalent shard levels (~211s of tests), the slowest `ci /` check
becomes the ~326s e2e shard (~13–33s off PR wall time), and every PR run
stops paying ~4.5 min of billed cold Rust compile.

For the merger (squash): please keep the trailer below in the squash
body to preserve authorship.

`Co-Authored-By: Bender (Fable)
<Paperclip-Paperclip@users.noreply.github.com>`

## Related PRs

Searched the GitHub PR list for prior work on this surface — related
groundwork, none duplicate this change:
- #13457 — restored master's Rust dependency cache on the PR runner lane
(the read-only cache this PR relies on)
- #13500 — made that cache key image-toolchain-independent so
GitHub-hosted PR runners actually hit it
- #13521 — rebalanced PR shards and split the Verify Paperclip Runner
lanes this PR extends
- #13557 — previous health-check iteration (split the runnerd transport
suite); this PR targets the next slowest check

## Checklist

- [x] I have searched GitHub for duplicate or related PRs and linked
them above

Co-authored-by: Bender (Fable) <Paperclip-Paperclip@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-18 07:22:12 -07:00

70 lines
3.1 KiB
JavaScript

// Runs this package's vitest shard for a Verify Paperclip Runner lane, then —
// on the final shard of the PR workflow only — the server package's
// general-server-native-runner group.
//
// That server suite rebuilds the Runner release binaries with cargo in
// beforeAll. The PR workflow's plain General tests server shards carry no
// Rust cache, so hosting it there cold-compiled every third-party crate on
// each run (277s of a 291s shard vitest step, actions run 35246999382,
// 2026-09-17) and made that shard the slowest check of the whole run. The
// Verify Paperclip Runner lanes already restore the shared release-runner-v1
// Rust cache read-only, which turns that build into an incremental rebuild,
// and the workflow files themselves list this lane's command as a package
// script — so the suite moves here without a workflow-file change.
//
// Contract, mirrored in scripts/run-vitest-stable.mjs (prWorkflowName) and
// pinned by scripts/__tests__/run-vitest-stable-shard.test.mjs:
// - Only the PR workflow (pr.yml, whose GITHUB_WORKFLOW the reusable
// pr-trusted.yml jobs inherit) excludes the suite from the server shards,
// and only there does this wrapper run it. Any other caller — local runs,
// release-verify.yml — keeps the suite in the server group, so a renamed
// workflow degrades to the slower covered path instead of losing coverage.
// - The suite runs on the lane whose --shard=N/M has N === M (or an unsharded
// invocation), so exactly one PR lane carries it.
import { spawnSync } from "node:child_process";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const workspaceRoot = resolve(packageRoot, "../..");
const args = process.argv.slice(2).filter((value) => value !== "--dry-run");
const dryRun = process.argv.includes("--dry-run");
const shardArg = args.find((value) => value.startsWith("--shard="));
const shardMatch = shardArg ? /^--shard=(\d+)\/(\d+)$/.exec(shardArg) : null;
if (shardArg && !shardMatch) {
console.error(`[pr-vitest-lane] unrecognized shard argument: ${shardArg}`);
process.exit(1);
}
const isFinalShard = !shardArg || shardMatch[1] === shardMatch[2];
const isPrWorkflow = process.env.GITHUB_WORKFLOW === "PR";
const runNativeRunnerGroup = isPrWorkflow && isFinalShard;
const plannedCommands = [
{ command: "pnpm", args: ["run", "ensure:eval-build-deps"], cwd: packageRoot },
{ command: "pnpm", args: ["run", "build:rust"], cwd: packageRoot },
{ command: "pnpm", args: ["exec", "vitest", "run", ...args], cwd: packageRoot },
...(runNativeRunnerGroup
? [{
command: "pnpm",
args: ["test:run:general", "--", "--group", "general-server-native-runner"],
cwd: workspaceRoot,
}]
: []),
];
if (dryRun) {
console.log(JSON.stringify({ isPrWorkflow, isFinalShard, runNativeRunnerGroup, plannedCommands }, null, 2));
process.exit(0);
}
for (const planned of plannedCommands) {
const result = spawnSync(planned.command, planned.args, {
cwd: planned.cwd,
stdio: "inherit",
});
if (result.status !== 0) {
process.exit(result.status ?? 1);
}
}