mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
## Thinking Path Post-merge of #13557, the slowest check on the freshest fully-green PR run ([35246999382](https://github.com/paperclipai/paperclip/actions/runs/35246999382)) was `ci / General tests (server (1/12))` at **339s**. The cause is one suite: `server/src/services/native-runtime/native-codex-runner.integration.test.ts` runs 1 test in **277s of a 291s vitest step (95%)** because its `beforeAll` cargo-builds the Runner release binaries, and the general-server shards carry no Rust cache — every PR run cold-compiles the full third-party crate graph. The other 19 suites in that shard finish in under 70ms each. The obvious fix (a dedicated Rust-cached matrix lane) requires editing workflow files, which the available GitHub App credentials cannot push (`workflows` permission). But the `Verify Paperclip Runner` lanes **already restore the shared `release-runner-v1` Rust cache read-only**, and their commands are `pnpm --filter @paperclipai/paperclip-runner <package script>` — so the suite can move into a Rust-cached lane purely through script changes. ## What Changed - `scripts/run-vitest-stable.mjs`: new `general-server-native-runner` group carrying exactly that suite. Under the PR workflow (`GITHUB_WORKFLOW == "PR"`, inherited from `pr.yml` by the reusable `pr-trusted.yml`) the without-chat server shards exclude it and rebalance to ~211s of tests each. Every other caller — local runs, `release-verify.yml` under the Release / Cloud readiness workflows — keeps the suite in the shards, so a renamed or unknown workflow degrades to today's slower-but-covered behavior instead of dropping coverage. - `packages/paperclip-runner`: `test:typescript:vitest` now routes through `scripts/run-pr-vitest-lane.mjs` — the identical `ensure:eval-build-deps && build:rust && vitest run` chain (shard flags passed through), plus the native-runner group on the **final PR shard only** (`--shard=N/M` with `N == M`, i.e. today's `vitest 2/2`, the 122s lane). With the restored cache the suite's cargo build becomes an incremental rebuild. - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: guards pin the whole contract — PR 12-shard coverage (shards + chat + native-runner = full server group exactly), Release/local 10-shard runs keep the suite, `pr.yml` is named `PR`, the vitest lanes partition with exactly one final shard, the package-script wiring, and the wrapper's shard/workflow gating via its `--dry-run` plan output. No workflow files change. `.github/workflows/*` are untouched. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs`: **36/36 pass** locally on this branch (includes the new coverage, wiring, and wrapper-gating guards). Both files run in CI's `Test general-server shard partition` / `Test release verify workflow wiring` steps. - Wrapper `--dry-run` plan matrix verified for all six shard/workflow combinations plus malformed-shard rejection (pinned as a guard test). - The executing proof is this PR's own CI: `ci / Verify Paperclip Runner (vitest 2/2)` must go green while running the native-runner suite (its log will show the `general-server-native-runner` group after the package vitest shard), and the 12 `ci / General tests (server (x/12))` shards must go green without it. ## Risks - The exclusion keys on `GITHUB_WORKFLOW == "PR"`. Failure mode of a rename is safe (suite falls back into the server shards, slower but covered) and the guard test on `pr.yml`'s name makes it loud. - `vitest 2/2` grows from ~122s to an expected ~210–260s — still well under the ~306s `vitest 1/2` and ~326s e2e shards, and inside the 20-minute lane timeout. If the cache misses (key drift), the lane pays a cold compile like the server shard does today; a miss is slow, never wrong. - Double-run/coverage-loss combinations are enumerated in the wrapper header and pinned by tests: each caller runs the suite exactly once. ## Model Used Claude (Bender agent, Paperclip) — Fable 5. --- Expected savings once merged: the 339s `server (1/12)` check drops to ~265s-equivalent shard levels (~211s of tests), the slowest `ci /` check becomes the ~326s e2e shard (~13–33s off PR wall time), and every PR run stops paying ~4.5 min of billed cold Rust compile. For the merger (squash): please keep the trailer below in the squash body to preserve authorship. `Co-Authored-By: Bender (Fable) <Paperclip-Paperclip@users.noreply.github.com>` ## Related PRs Searched the GitHub PR list for prior work on this surface — related groundwork, none duplicate this change: - #13457 — restored master's Rust dependency cache on the PR runner lane (the read-only cache this PR relies on) - #13500 — made that cache key image-toolchain-independent so GitHub-hosted PR runners actually hit it - #13521 — rebalanced PR shards and split the Verify Paperclip Runner lanes this PR extends - #13557 — previous health-check iteration (split the runnerd transport suite); this PR targets the next slowest check ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above Co-authored-by: Bender (Fable) <Paperclip-Paperclip@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
70 lines
3.1 KiB
JavaScript
70 lines
3.1 KiB
JavaScript
// Runs this package's vitest shard for a Verify Paperclip Runner lane, then —
|
|
// on the final shard of the PR workflow only — the server package's
|
|
// general-server-native-runner group.
|
|
//
|
|
// That server suite rebuilds the Runner release binaries with cargo in
|
|
// beforeAll. The PR workflow's plain General tests server shards carry no
|
|
// Rust cache, so hosting it there cold-compiled every third-party crate on
|
|
// each run (277s of a 291s shard vitest step, actions run 35246999382,
|
|
// 2026-09-17) and made that shard the slowest check of the whole run. The
|
|
// Verify Paperclip Runner lanes already restore the shared release-runner-v1
|
|
// Rust cache read-only, which turns that build into an incremental rebuild,
|
|
// and the workflow files themselves list this lane's command as a package
|
|
// script — so the suite moves here without a workflow-file change.
|
|
//
|
|
// Contract, mirrored in scripts/run-vitest-stable.mjs (prWorkflowName) and
|
|
// pinned by scripts/__tests__/run-vitest-stable-shard.test.mjs:
|
|
// - Only the PR workflow (pr.yml, whose GITHUB_WORKFLOW the reusable
|
|
// pr-trusted.yml jobs inherit) excludes the suite from the server shards,
|
|
// and only there does this wrapper run it. Any other caller — local runs,
|
|
// release-verify.yml — keeps the suite in the server group, so a renamed
|
|
// workflow degrades to the slower covered path instead of losing coverage.
|
|
// - The suite runs on the lane whose --shard=N/M has N === M (or an unsharded
|
|
// invocation), so exactly one PR lane carries it.
|
|
import { spawnSync } from "node:child_process";
|
|
import { dirname, resolve } from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
|
|
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
|
const workspaceRoot = resolve(packageRoot, "../..");
|
|
const args = process.argv.slice(2).filter((value) => value !== "--dry-run");
|
|
const dryRun = process.argv.includes("--dry-run");
|
|
|
|
const shardArg = args.find((value) => value.startsWith("--shard="));
|
|
const shardMatch = shardArg ? /^--shard=(\d+)\/(\d+)$/.exec(shardArg) : null;
|
|
if (shardArg && !shardMatch) {
|
|
console.error(`[pr-vitest-lane] unrecognized shard argument: ${shardArg}`);
|
|
process.exit(1);
|
|
}
|
|
const isFinalShard = !shardArg || shardMatch[1] === shardMatch[2];
|
|
const isPrWorkflow = process.env.GITHUB_WORKFLOW === "PR";
|
|
const runNativeRunnerGroup = isPrWorkflow && isFinalShard;
|
|
|
|
const plannedCommands = [
|
|
{ command: "pnpm", args: ["run", "ensure:eval-build-deps"], cwd: packageRoot },
|
|
{ command: "pnpm", args: ["run", "build:rust"], cwd: packageRoot },
|
|
{ command: "pnpm", args: ["exec", "vitest", "run", ...args], cwd: packageRoot },
|
|
...(runNativeRunnerGroup
|
|
? [{
|
|
command: "pnpm",
|
|
args: ["test:run:general", "--", "--group", "general-server-native-runner"],
|
|
cwd: workspaceRoot,
|
|
}]
|
|
: []),
|
|
];
|
|
|
|
if (dryRun) {
|
|
console.log(JSON.stringify({ isPrWorkflow, isFinalShard, runNativeRunnerGroup, plannedCommands }, null, 2));
|
|
process.exit(0);
|
|
}
|
|
|
|
for (const planned of plannedCommands) {
|
|
const result = spawnSync(planned.command, planned.args, {
|
|
cwd: planned.cwd,
|
|
stdio: "inherit",
|
|
});
|
|
if (result.status !== 0) {
|
|
process.exit(result.status ?? 1);
|
|
}
|
|
}
|