Files
PaperClipAI/packages/paperclip-runner/scripts/run-pr-vitest-lane.mjs
T
Devin FoleyandPaperclip 4265cb3a2b fix(ci): cache native server integration builds in release verification (#15619)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Release verification must run the same server integration coverage
as PR verification.
> - Three server suites use real Rust Runner binaries.
> - PR checks already run them with the shared Rust dependency cache,
but release checks cold-build them inside ordinary server test shards.
> - A cold Dot Runner build can consume its entire setup deadline before
any test runs.
> - This pull request gives release verification a required cached
native integration lane and preserves every test.

## Linked Issues or Issue Description

**What happened?**

On master commit `1881894973a2b25838d8abed9bd8aeebc3af4441`, [Cloud
readiness run
37837810707](https://github.com/paperclipai/paperclip/actions/runs/37837810707)
failed in server shard 7. Dot Runner's `cargo build --release` exceeded
its 300-second child-process deadline. The other 1,640 tests in that
shard passed. The failed setup then tried to remove an undefined
temporary path and emitted a second error. Cargo output was captured as
an opaque buffer, which obscured build progress.

**What did you expect to happen?**

Build fixture binaries before tests in a lane with the existing Rust
dependency cache. Run all native integration tests and make their result
required for release readiness. A setup failure should keep its original
diagnostic.

**Steps to reproduce**

Run release verification on a clean runner. The existing
`general-server-without-chat` group retains the three Cargo-backed
suites outside the PR workflow. The Dot suite can time out during a cold
release build. Run the new workflow and partition tests against the
previous source to reproduce five routing and coverage failures
deterministically.

**Version / commit**

Observed at `1881894973a2b25838d8abed9bd8aeebc3af4441`; this change is
based on `ed6abbf158b`.

**Deployment mode**

GitHub Actions Release and Cloud readiness verification. No application
runtime or deployment action changes.

Related work: #15581 also edits Dot integration tests for onboarding
behavior. It does not repair release test routing. Searches found no
open PR for this failure.

## What Changed

- Add an explicit server test group that excludes the dedicated chat and
native suites. Preserve the existing PR and local test groups.
- Run all three native server suites in one required matrix lane with
the existing trusted Rust dependency cache.
- Build debug and release fixture binaries in a visible step with a
10-minute limit before tests start. Keep Cargo freshness checks and the
existing test deadlines.
- Stream Cargo diagnostics and clean up safely when Dot setup stops
before creating its temporary directory.
- Test complete, non-overlapping partitions for Release, Cloud
readiness, other callers, and existing PR/local groups. Check cache
restrictions, build order, and the source verification dependency.

## Verification

- Passed 44 workflow and partition tests: `node --test
scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs`.
- The same tests fail in five relevant cases against the previous
workflow and selector. They pass after this correction.
- An independent review repeated all 44 tests successfully.
- Passed `actionlint .github/workflows/release-verify.yml`, `git diff
--check`, and a secret scan.
- Passed all 381 workflow policy tests: `node --test
'.github/scripts/tests/*.test.mjs'`.
- Passed full `pnpm build` in a clean worktree.
- Built debug and release fixture binaries, then passed all 32 tests in
the three real native integration suites: `pnpm test:run:general --
--group general-server-native-runner` (49.5 seconds, no skips).
- Passed full `pnpm -r typecheck`.
- Injected a synthetic Cargo setup failure. The suite reports that
failure without the secondary undefined-path cleanup error.
- Exact-head CI completed: 53 successful checks, including all server
shards, both native Runner lanes, build, typecheck, browser tests, and
the canary dry run. Two optional Storybook checks were skipped.
- Started the duplicate local `pnpm test:run` aggregate and stopped it
after exact-head CI passed. No completed local aggregate result is
claimed. All test processes owned by that run exited.
- Greptile scored the final head
`ffe5ab5a28af2dfea9db1c1952c652f070bf52f8` at 5/5 with no review
threads.
- `Superagent Supply Chain Scan` is neutral, not passed: it only
supports exact dependency pin replacements and cannot verify structural
edits to `.github/workflows/release-verify.yml`. It reported no
annotations. Independent source review, Greptile, actionlint, and the
workflow policy tests cover this structural change.
- GitHub still requires a code-owner review for the workflow files.
There are no merge conflicts.

## Risks

- Adds one CI matrix job, which increases concurrent runner demand. The
existing cache writer and trust restrictions stay in place.
- Missing dependencies still compile from the lockfile. A build that
exceeds its step limit fails visibly; failed tests still block source
verification.
- Workflow execution uses the new lane after merge. PR tests validate
the workflow contract and run the existing native test lane.
- The supply chain scanner cannot analyze this workflow structure. Its
neutral result is an explicit coverage limit; it is not counted as a
successful scan.
- No runtime, schema, deployment, publication, credential, or
test-timeout changes.

## Model Used

OpenAI Codex, based on GPT-6, with repository inspection, code
execution, and independent agent review. The exact deployed model ID and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-08 14:33:10 -07:00

70 lines
3.1 KiB
JavaScript

// Runs this package's vitest shard for a Verify Paperclip Runner lane, then —
// on the final shard of the PR workflow only — the server package's
// general-server-native-runner group.
//
// That server suite rebuilds the Runner release binaries with cargo in
// beforeAll. The PR workflow's plain General tests server shards carry no
// Rust cache, so hosting it there cold-compiled every third-party crate on
// each run (277s of a 291s shard vitest step, actions run 35246999382,
// 2026-09-17) and made that shard the slowest check of the whole run. The
// Verify Paperclip Runner lanes already restore the shared release-runner-v1
// Rust cache read-only, which turns that build into an incremental rebuild,
// and the workflow files themselves list this lane's command as a package
// script — so the suite moves here without a workflow-file change.
//
// Contract, mirrored in scripts/run-vitest-stable.mjs (prWorkflowName) and
// pinned by scripts/__tests__/run-vitest-stable-shard.test.mjs:
// - Only the PR workflow (pr.yml, whose GITHUB_WORKFLOW the reusable
// pr-trusted.yml jobs inherit) excludes these suites from the existing
// without-chat group, and only there does this wrapper run them. Other
// callers keep that group complete. Release verification instead selects
// an explicit partition and its own required cached native lane.
// - The suite runs on the lane whose --shard=N/M has N === M (or an unsharded
// invocation), so exactly one PR lane carries it.
import { spawnSync } from "node:child_process";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const workspaceRoot = resolve(packageRoot, "../..");
const args = process.argv.slice(2).filter((value) => value !== "--dry-run");
const dryRun = process.argv.includes("--dry-run");
const shardArg = args.find((value) => value.startsWith("--shard="));
const shardMatch = shardArg ? /^--shard=(\d+)\/(\d+)$/.exec(shardArg) : null;
if (shardArg && !shardMatch) {
console.error(`[pr-vitest-lane] unrecognized shard argument: ${shardArg}`);
process.exit(1);
}
const isFinalShard = !shardArg || shardMatch[1] === shardMatch[2];
const isPrWorkflow = process.env.GITHUB_WORKFLOW === "PR";
const runNativeRunnerGroup = isPrWorkflow && isFinalShard;
const plannedCommands = [
{ command: "pnpm", args: ["run", "ensure:eval-build-deps"], cwd: packageRoot },
{ command: "pnpm", args: ["run", "build:rust"], cwd: packageRoot },
{ command: "pnpm", args: ["exec", "vitest", "run", ...args], cwd: packageRoot },
...(runNativeRunnerGroup
? [{
command: "pnpm",
args: ["test:run:general", "--", "--group", "general-server-native-runner"],
cwd: workspaceRoot,
}]
: []),
];
if (dryRun) {
console.log(JSON.stringify({ isPrWorkflow, isFinalShard, runNativeRunnerGroup, plannedCommands }, null, 2));
process.exit(0);
}
for (const planned of plannedCommands) {
const result = spawnSync(planned.command, planned.args, {
cwd: planned.cwd,
stdio: "inherit",
});
if (result.status !== 0) {
process.exit(result.status ?? 1);
}
}