mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 20:12:20 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the trusted PR workflow, and the `Typecheck + Release Registry` job is one of its required checks > - That job takes about 9.2 minutes on the 4 vCPU / 16 GB fleet runner (run 43344818387), against a target of 7 minutes or less for every job > - The job runs six steps one after another, and the first step type-checks the whole server program a second time, at about 13.5 GiB, which makes the runner swap > - Typecheck results are the check that authors most want back quickly, so this job is on the critical path of most pull requests > - This pull request removes the duplicate server type check and runs the independent checks in parallel through one small script, inside the same memory budget > - The benefit is about 39% less wall time for the job's checks on the same machine, with peak memory down from about 14 GiB (plus swap) to about 8.5 GiB ## Linked Issues or Issue Description **What existing behavior does this improve?** The `Typecheck + Release Registry` job in `.github/workflows/pr-trusted.yml`. It takes about 9.2 minutes. The target for every job in the workflow is 7 minutes or less. **Subsystem affected** CI (`.github/workflows/pr-trusted.yml`) and the typecheck scripts under `scripts/` and `server/scripts/`. **Current behavior** After about 75 s of setup, the job runs six steps in sequence on a 4 vCPU runner, about 470 s in all (run 43344818387): `typecheck:build-gaps` 121 s, test programs 85 s, server layers 128 s, test projects 38 s, CLI guidance test 5 s, release registry tests 95 s. Most of these steps use one thread. The `typecheck:build-gaps` step builds the server with a full type check of the whole server program. The server layer step then checks the same files again, and the `Build` job checks them a third time. That full program peaks at about 13.5 GiB, so the job peaks at about 14 GiB of 16 GiB and uses swap. **Proposed behavior** The job runs `node scripts/ci-typecheck-lane.mjs`. The script builds the server without type checks (`PAPERCLIP_TSC_EMIT_ONLY=1`, as the Docker image does) and still checks the runtime assets. It runs the steps that write shared files first, one at a time. Then it runs the remaining checks in parallel, one per CPU, with the server declaration layers first, because the test layers and test projects read them. The checks that are not on that path run at a lower CPU priority. **Reason and benefit** On the same runner, the job's checks take 61% of the time and about 60% of the memory (see Verification). The job should take about 6 minutes on the fleet instead of 9.2. **Breaking changes** None. The same files are type-checked with the same options. Local commands do not change. `pnpm run typecheck:build-gaps` still works as before. ## What Changed - `scripts/ci-typecheck-lane.mjs` (new): runs the job's checks. It writes shared outputs first, in sequence: links, plugin SDK, server build without type checks, the runtime asset check, the sandbox provider installs, and the Runner typecheck (it rebuilds the eval kernel, which other checks read). Then it runs these checks in parallel: the server declaration layers, each server test layer, each test project, the test programs, the other build-gap workspaces, the CLI guidance test and the release registry tests. A passing check prints its output in a collapsed log group. A failing check prints its full output between `=== FAILED` and `=== end of` lines, with an error annotation. The script ends with a table of each check's start, duration and result, and exits non-zero if any check fails. - `server/scripts/typecheck-layers.mjs`: new `--layer <name>` option that checks one layer against the configs and declarations of an earlier `--declarations-only` run, and an exported `listLayers`. - `scripts/typecheck-test-programs.mjs`: new `--prepare-only` and `--skip-prepare` options, so the installs happen before the parallel checks start. - `scripts/run-typecheck-build-gaps.mjs`: new `--list-packages` option, so the script still finds the build-gap workspaces and still fails when one has no `typecheck` script. - `.github/workflows/pr-trusted.yml`: the `typecheck_release_registry` job runs the script instead of the six steps. No other job changes. - Tests: new `scripts/__tests__/ci-typecheck-lane.test.mjs` (added to `test:release-registry`), new cases in `server/scripts/typecheck-layers.test.mjs` and `scripts/typecheck-test-programs.test.mjs`, and the workflow wiring assertions in `.github/scripts/tests/ci-plan.test.mjs` and `scripts/__tests__/runner-release-crate-cache.test.mjs`. - `tests/runner-e2e/source-sdk-public-inputs.json`: repinned, because `package.json` changed. - `doc/DEVELOPING.md`: new "Typecheck lane in CI" section, and the new options. ## Verification This PR's own CI ran the old job definition, because PR runs use the workflow from master. So I measured the commands on a stand-in machine: a temporary workflow (`.github/workflows/typecheck-lane-bench-temp.yml`, now deleted) on GitHub-hosted `ubuntu-latest` (4 vCPU Intel Xeon 6975P-C, 15.7 GiB, no secrets, `contents: read`). It ran in runs [43444356017](https://github.com/paperclipai/paperclip/actions/runs/43444356017), [43445393958](https://github.com/paperclipai/paperclip/actions/runs/43445393958), [43446786105](https://github.com/paperclipai/paperclip/actions/runs/43446786105), [43448707837](https://github.com/paperclipai/paperclip/actions/runs/43448707837) and [43450786493](https://github.com/paperclipai/paperclip/actions/runs/43450786493). Each step ran under `/usr/bin/time -v`, and a sampler read `/proc/meminfo` every 0.25 s. **Old steps, one at a time (3 runs, run 43445393958):** | Step | Wall s | CPU s | Peak RSS of one process | System memory peak | | --- | --- | --- | --- | --- | | `typecheck:build-gaps` | 128–155 | 376–400 | 13.2–13.6 GiB | 14.2–14.7 GiB | | ...of which the full server `tsc` | 40–51 | 146–164 | 12.8–13.8 GiB | 13.7–14.7 GiB, swap up to 738 MiB | | ...replaced by the emit-only server build | 6–10 | 20–22 | 2.9 GiB | 3.9 GiB | | Test programs | 67–87 | 147–170 | 1.9 GiB | 2.9 GiB | | Server layers | 102–127 | 150–172 | 4.0–4.2 GiB | 5.4–5.6 GiB | | Test projects | 31–37 | 49–59 | 2.1–2.2 GiB | 3.1–3.2 GiB | | CLI guidance test | 2–4 | 2–3 | 0.2 GiB | 1.6 GiB | | Release registry tests | 82–95 | 159–183 | 1.7 GiB | 3.0 GiB | CPU seconds are about twice the wall seconds for the `tsc` steps even on one checker thread. So each step uses about two of the four vCPUs. **Old sequence and new lane on the same runner (4 runs, run 43450786493).** Each job copied one installed checkout, then ran both, in alternating order: | Run | Old steps, wall s | New lane, wall s | Ratio | Old memory peak | New memory peak | | --- | --- | --- | --- | --- | --- | | 1 | 503 | 314 | 0.62 | 13.98 GiB | 8.33 GiB | | 2 | 647 | 388 | 0.60 | 14.04 GiB, swap 637 MiB | 8.13 GiB | | 3 | 627 | 385 | 0.61 | 13.98 GiB | 8.63 GiB | | 4 | 382 | 237 | 0.62 | 14.10 GiB, swap 1.1 GiB | 7.99 GiB | Hosted runners differ in speed by up to 1.7 times, so only the same-runner ratio is meaningful. Both arms include a cold compile of the Runner release crate (45–80 s) that the fleet restores from cache. The fleet ran the old steps in 470 s; at the measured ratio, the new lane should take about 290 s, and the job about 6 minutes with setup. The new lane peaks at about 55% of memory, with no swap. Local checks (on a loaded Mac, for correctness only): - `node --test scripts/__tests__/ci-typecheck-lane.test.mjs` — 8 pass. - `node --test .github/scripts/tests/*.test.mjs` — 807 pass. - `node --test scripts/__tests__/e2e-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs scripts/__tests__/run-typecheck-build-gaps.test.mjs` — pass. - `pnpm exec vitest run scripts/typecheck-layers.test.mjs` in `server/` — the new cases pass. ## Risks - This changes a required job that every PR depends on, and its own CI cannot prove it. Please watch the first `Typecheck + Release Registry` runs on master after merge: the wall time (expect about 6 minutes), the summary table at the end of the step, and any `terminated by` message. - A type error now shows in that check's `=== FAILED` block, not in a step of its own. Passing checks are folded, so the failure is the only expanded block. - When the server declaration layers fail, the test layers and test projects are reported as skipped. Before, the test layers still ran after a source error. Fix the source errors first; the next run checks the tests. - The Honeycomb CI time board and any step-name queries see one step, `Typecheck, CLI guidance safety and release registry tests`, instead of six. The script prints the duration of each check. - The release registry tests run beside the type checks at normal priority. Their slowest test took 46–78 s here, against a 180 s timeout, and 114 s on the slowest stand-in runner. - Low risk for memory: the new lane peaked at 8.0–8.6 GiB of 15.7 GiB, far below the old 14 GiB. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool use (shell, GitHub CLI, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
343 lines
21 KiB
JavaScript
343 lines
21 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { chmodSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
|
import { execFileSync, spawnSync } from "node:child_process";
|
|
import { tmpdir } from "node:os";
|
|
import path from "node:path";
|
|
import test from "node:test";
|
|
import { comparisonPaths, planTasks } from "../ci-plan.mjs";
|
|
import { mergeDiff } from "../ci-diff.mjs";
|
|
import { planLane } from "../../../scripts/ci-typecheck-lane.mjs";
|
|
|
|
const root = new URL("../../../", import.meta.url);
|
|
const config = JSON.parse(readFileSync(new URL("turbo.json", root), "utf8"));
|
|
const lanes = JSON.parse(readFileSync(new URL(".github/ci-task-matrix.json", root), "utf8"));
|
|
const plan = (paths, options = {}) => planTasks({ config, lanes, paths, ...options });
|
|
const selected = (result, job) => result.jobs[job].run;
|
|
const selectedTasks = (result) => new Set(Object.values(result.jobs).filter((job) => job.run)
|
|
.flatMap((job) => job.matrix.include.map((entry) => entry.task)));
|
|
|
|
test("trusted workflow fallbacks preserve every declared partition and command", () => {
|
|
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
|
|
for (const [job, entries] of Object.entries(lanes)) {
|
|
const block = workflow.split(`\n ${job}:\n`)[1].split(/\n [a-z0-9_]+:\n/)[0];
|
|
const normalized = block.replaceAll("(needs.test_selection.outputs.plan || needs.gate.outputs.plan)", "needs.gate.outputs.plan");
|
|
assert.ok(normalized.includes(`fromJSON(needs.gate.outputs.plan).jobs.${job}.run`), job);
|
|
if (!block.includes(" matrix:")) { assert.equal(entries.length, 1); continue; }
|
|
const fallback = block.match(/matrix:.*\|\| '(.*)'\) \}\}/)?.[1];
|
|
assert.ok(fallback, `${job}: missing full fallback`);
|
|
assert.deepEqual(JSON.parse(fallback), { include: entries }, job);
|
|
}
|
|
});
|
|
|
|
test("a UI component retains browser and build coverage without unrelated server/Rust jobs", () => {
|
|
const result = plan(["ui/src/components/Sidebar.tsx"]);
|
|
assert.equal(selected(result, "build"), true);
|
|
assert.equal(selected(result, "e2e_shards"), true);
|
|
assert.equal(selected(result, "verify_serialized_server"), false);
|
|
// The Runner support suite imports UI modules and checks that catalog-listed
|
|
// UI files stay tracked, so UI source changes are its inputs too.
|
|
assert.deepEqual([...selectedTasks(result)].filter((task) => /server|chat|runner/.test(task)), ["ci:runner-e2e-support"]);
|
|
assert.equal(result.jobs.general_tests.selected, 2);
|
|
});
|
|
|
|
test("source inspected across package boundaries selects its consumers", () => {
|
|
for (const file of ["ui/src/lib/issue-execution-policy.ts", "ui/src/index.css", "ui/src/pages/CompanyExport.tsx",
|
|
"ui/src/components/MarkdownBody.tsx"]) {
|
|
const tasks = selectedTasks(plan([file]));
|
|
assert.ok(tasks.has("ci:server"), file);
|
|
assert.ok(tasks.has("ci:runner-vitest"), file);
|
|
}
|
|
assert.ok(selectedTasks(plan(["ui/public/app-logos/example.svg"])).has("ci:workspaces-b"));
|
|
});
|
|
|
|
test("repository-wide guidance guards run uncached for every scanned UI file type", () => {
|
|
const source = readFileSync(new URL("server/src/__tests__/cli-invocation-safety.test.ts", root), "utf8");
|
|
const extensions = [...source.split("const SCAN_EXTENSIONS = new Set([")[1].split("]);")[0].matchAll(/"(\.[a-z]+)"/g)]
|
|
.map((match) => match[1]);
|
|
assert.ok(extensions.length > 0);
|
|
for (const extension of extensions) {
|
|
for (const folder of ["ui/src/components", "ui/src/pages", "ui/src/hooks", "ui/public"]) {
|
|
assert.ok(plan([`${folder}/guidance${extension}`]).jobs.typecheck_release_registry.run);
|
|
}
|
|
}
|
|
assert.equal(config.tasks["//#ci:typecheck"].cache, false);
|
|
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
|
|
const job = workflow.split("\n typecheck_release_registry:\n")[1].split("\n general_tests:\n")[0];
|
|
assert.match(job, /\n run: node scripts\/ci-typecheck-lane\.mjs\n/);
|
|
const guidance = planLane({ buildGapPackages: [], testLayers: [] }).tasks.find((task) => task.name === "CLI guidance safety");
|
|
assert.deepEqual(guidance.args, ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"]);
|
|
assert.ok(!job.includes("uses: ./.github/actions/run-ci-task"));
|
|
});
|
|
|
|
test("shared contracts, runtime prompts, protocol inputs and fixtures reach consumers", () => {
|
|
const shared = selectedTasks(plan(["packages/shared/src/types/issue.ts"]));
|
|
for (const task of ["ci:server", "ci:db", "ci:workspaces-a", "ci:workspaces-b", "ci:serialized", "ci:e2e"]) {
|
|
assert.ok(shared.has(task), task);
|
|
}
|
|
assert.ok(selectedTasks(plan(["server/src/onboarding-assets/default/AGENTS.md"])).has("ci:runner-vitest"));
|
|
assert.ok(selectedTasks(plan(["packages/paperclip-runner/protocol/schema.json"])).has("ci:runner-rust"));
|
|
for (const file of ["tests/runner-e2e/fixtures/prompt.md", "scripts/run-vitest-stable.mjs", "pnpm-lock.yaml"]) {
|
|
assert.ok(Object.values(plan([file]).jobs).every((job) => job.selected === job.total), file);
|
|
}
|
|
});
|
|
|
|
test("sandbox provider tests run only for the providers, the plugin SDK, and what the SDK imports", () => {
|
|
for (const file of ["packages/plugins/sandbox-providers/daytona/src/plugin.ts",
|
|
"packages/plugins/sandbox-providers/cloudflare/bridge-template/src/index.ts",
|
|
"packages/plugins/sdk/src/index.ts", "packages/shared/src/types/issue.ts"]) {
|
|
assert.equal(selected(plan([file]), "sandbox_provider_tests"), true, file);
|
|
}
|
|
for (const file of ["server/src/index.ts", "ui/src/components/Sidebar.tsx", "cli/src/index.ts",
|
|
"packages/paperclip-runner/src/index.ts", "packages/plugins/plugin-workspace-diff/src/worker.ts"]) {
|
|
assert.equal(selected(plan([file]), "sandbox_provider_tests"), false, file);
|
|
}
|
|
// The provider lockfiles, the shared install workspace and the runner script are global inputs.
|
|
for (const file of ["scripts/typecheck-sandbox-providers/pnpm-lock.yaml", "scripts/test-sandbox-providers.mjs",
|
|
"packages/plugins/sandbox-providers/daytona/package.json"]) {
|
|
assert.equal(selected(plan([file]), "sandbox_provider_tests"), true, file);
|
|
}
|
|
// The SDK depends on shared and nothing else in the workspace. A new dependency must join the inputs.
|
|
const sdk = JSON.parse(readFileSync(new URL("packages/plugins/sdk/package.json", root), "utf8"));
|
|
const workspace = Object.entries({ ...sdk.dependencies, ...sdk.peerDependencies })
|
|
.filter(([, version]) => String(version).startsWith("workspace:")).map(([name]) => name);
|
|
assert.deepEqual(workspace, ["@paperclipai/shared"]);
|
|
const shared = JSON.parse(readFileSync(new URL("packages/shared/package.json", root), "utf8"));
|
|
assert.deepEqual(Object.values({ ...shared.dependencies, ...shared.peerDependencies })
|
|
.filter((version) => String(version).startsWith("workspace:")), []);
|
|
});
|
|
|
|
test("renames out of source and deletions cannot masquerade as release notes", () => {
|
|
const paths = comparisonPaths({ files: [
|
|
{ filename: "releases/v2026.1029.0.md", previous_filename: "packages/shared/src/constants.ts", status: "renamed" },
|
|
{ filename: "server/src/deleted.ts", status: "removed" },
|
|
] });
|
|
assert.ok(selectedTasks(plan(paths)).has("ci:server"));
|
|
assert.ok(selectedTasks(plan(paths)).has("ci:db"));
|
|
});
|
|
|
|
test("unclassified paths and uncertain comparisons select full verification", () => {
|
|
for (const paths of [[], ["new-root/file.ts"], ["../escaped"]]) {
|
|
const result = plan(paths);
|
|
assert.ok(result.forceReason);
|
|
assert.ok(Object.values(result.jobs).every((job) => job.selected === job.total));
|
|
}
|
|
for (const files of [null, [], [{ status: "modified" }], [{ filename: "a", status: "renamed" }],
|
|
Array.from({ length: 300 }, () => ({ filename: "ui/src/test.ts" }))]) {
|
|
assert.throws(() => comparisonPaths({ files }));
|
|
}
|
|
});
|
|
|
|
test("shadow mode explains savings but runs every task; force overrides selection", () => {
|
|
const shadow = plan(["ui/src/components/Sidebar.tsx"], { mode: "shadow" });
|
|
assert.equal(shadow.jobs.verify_serialized_server.proposed, 0);
|
|
assert.equal(shadow.jobs.verify_serialized_server.selected, 9);
|
|
for (const options of [{ mode: "full" }, { forceReason: "Stack" }]) {
|
|
assert.ok(Object.values(plan(["ui/src/components/Sidebar.tsx"], options).jobs)
|
|
.every((job) => job.selected === job.total));
|
|
}
|
|
});
|
|
|
|
test("cache-affecting definition changes force execution across successive PR pushes", () => {
|
|
for (const file of ["turbo.json", "package.json", "packages/db/package.json", "scripts/ci/run-task.mjs",
|
|
".github/actions/run-ci-task/action.yml"]) {
|
|
for (const paths of [[file], [file, "packages/db/src/schema/issues.ts"]]) {
|
|
const result = plan(paths);
|
|
assert.match(result.forceReason, /Global CI input changed/);
|
|
assert.ok(Object.values(result.jobs).every((job) => job.selected === job.total));
|
|
}
|
|
}
|
|
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
|
|
assert.equal((workflow.match(/force:.*\.forceReason != null/g) ?? []).length, 3);
|
|
const action = readFileSync(new URL(".github/actions/run-ci-task/action.yml", root), "utf8");
|
|
assert.match(action, /CI_FORCE_TASKS: \$\{\{ inputs.force \}\}/);
|
|
assert.equal((action.match(/if:.*inputs.force != 'true'/g) ?? []).length, 2);
|
|
});
|
|
|
|
test("task dependencies propagate input changes and unsupported graph syntax fails closed", () => {
|
|
const fixture = { globalDependencies: [], tasks: {
|
|
"//#build": { inputs: ["lib/**"] }, "//#test": { inputs: ["app/**"], dependsOn: ["//#build"] },
|
|
} };
|
|
assert.equal(planTasks({ config: fixture, lanes: { tests: [{ task: "test" }] }, paths: ["lib/index.ts"] }).jobs.tests.run, true);
|
|
fixture.tasks["//#build"].dependsOn = ["//#test"];
|
|
assert.throws(() => planTasks({ config: fixture, lanes: { tests: [{ task: "test" }] }, paths: ["app/index.ts"] }), /Cyclic/);
|
|
assert.throws(() => plan(["ui/src/a.ts"], { mode: "typo" }), /Invalid CI mode/);
|
|
});
|
|
|
|
test("local planner reads a real Git rename and handles a missing revision conservatively", () => {
|
|
const directory = mkdtempSync(path.join(tmpdir(), "ci-plan-"));
|
|
const git = (...args) => execFileSync("git", args, { cwd: directory, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
|
|
try {
|
|
git("init", "-q"); git("config", "user.name", "CI Test"); git("config", "user.email", "ci@example.invalid");
|
|
mkdirSync(path.join(directory, "packages/shared/src"), { recursive: true });
|
|
writeFileSync(path.join(directory, "packages/shared/src/input.ts"), "original\n");
|
|
git("add", "."); git("commit", "-qm", "base");
|
|
const base = git("rev-parse", "HEAD").trim();
|
|
mkdirSync(path.join(directory, "releases"));
|
|
git("mv", "packages/shared/src/input.ts", "releases/v1.md"); git("commit", "-qm", "rename");
|
|
for (const revision of [base, "missing-revision"]) {
|
|
const result = spawnSync(process.execPath, [new URL("../ci-plan.mjs", import.meta.url).pathname, revision], {
|
|
cwd: directory, encoding: "utf8", env: { ...process.env, GITHUB_OUTPUT: "", GITHUB_STEP_SUMMARY: "",
|
|
CI_TASK_CONFIG: new URL("turbo.json", root).pathname, CI_TASK_MATRIX: new URL(".github/ci-task-matrix.json", root).pathname },
|
|
});
|
|
assert.equal(result.status, 0, result.stderr);
|
|
const resultPlan = JSON.parse(result.stdout);
|
|
assert.ok(selectedTasks(resultPlan).has("ci:server"));
|
|
if (revision !== base) assert.match(resultPlan.forceReason, /Diff unavailable/);
|
|
}
|
|
} finally { rmSync(directory, { recursive: true, force: true }); }
|
|
});
|
|
|
|
test("required aggregates reject omitted, failed and cancelled selected work", () => {
|
|
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
|
|
for (const id of ["verify", "e2e"]) {
|
|
const block = workflow.split(`\n ${id}:\n`)[1].split(/\n [a-z0-9_]+:\n/)[0];
|
|
const code = block.split(" node --input-type=module <<'NODE'\n")[1].split("\n NODE")[0]
|
|
.split("\n").map((line) => line.slice(10)).join("\n");
|
|
const names = /needs: \[([^\]]+)\]/.exec(block)[1].split(", ");
|
|
const resultPlan = plan(["ui/src/components/Sidebar.tsx"]);
|
|
const results = Object.fromEntries(names.map((name) => [name, { result:
|
|
name === "gate" || name === "policy" || resultPlan.jobs[name]?.run ? "success" : "skipped" }]));
|
|
const run = (env = {}) => spawnSync(process.execPath, ["--input-type=module", "-e", code], { encoding: "utf8",
|
|
env: { ...process.env, FULL_CI: "true", CI_PLAN: JSON.stringify(resultPlan), CI_RESULTS: JSON.stringify(results), ...env } });
|
|
assert.equal(run().status, 0);
|
|
for (const name of names) {
|
|
// Optional analysis can fail or be skipped; the unchanged task plan
|
|
// still has to pass. Selected-plan cases are exercised separately.
|
|
if (name === "test_selection") continue;
|
|
const previous = results[name].result;
|
|
for (const bad of previous === "success" ? ["failure", "cancelled", "skipped"] : ["success"]) {
|
|
results[name].result = bad;
|
|
assert.notEqual(run().status, 0, `${id}: ${name} must reject ${bad}`);
|
|
}
|
|
results[name].result = previous;
|
|
}
|
|
for (const result of ["failure", "cancelled", "skipped"]) {
|
|
results.gate.result = result;
|
|
const failedGate = run({ FULL_CI: "", CI_PLAN: "" });
|
|
assert.notEqual(failedGate.status, 0, `${id}: an absent routing output must not pass`);
|
|
assert.match(failedGate.stderr, /CI routing must succeed/);
|
|
}
|
|
assert.match(block, /^ {4}runs-on: \$\{\{ needs\.gate\.outputs\.runner \|\| 'ubuntu-latest' \}\}$/m,
|
|
`${id}: the always-run summary must have a runner when routing produces no output`);
|
|
}
|
|
});
|
|
|
|
test("the real routing step uses pinned trusted definitions and defaults to full on fetch failure", () => {
|
|
const workflow = readFileSync(new URL(".github/workflows/ci-routing.yml", root), "utf8");
|
|
const block = workflow.split(" - name: Plan affected CI tasks from trusted definitions\n")[1];
|
|
const shell = block.split(" run: |\n")[1].split("\n").map((line) => line.slice(10)).join("\n");
|
|
const directory = mkdtempSync(path.join(tmpdir(), "ci-routing-plan-"));
|
|
try {
|
|
const repository = path.join(directory, "source");
|
|
mkdirSync(repository);
|
|
const gitBin = execFileSync("which", ["git"], { encoding: "utf8" }).trim();
|
|
const git = (...args) => execFileSync(gitBin, args, { cwd: repository, encoding: "utf8" }).trim();
|
|
git("init", "-q", "-b", "main"); git("config", "user.name", "CI"); git("config", "user.email", "ci@example.invalid");
|
|
writeFileSync(path.join(repository, "base"), "base"); git("add", "."); git("commit", "-qm", "base");
|
|
git("checkout", "-qb", "pr");
|
|
mkdirSync(path.join(repository, "ui/src/components"), { recursive: true });
|
|
writeFileSync(path.join(repository, "ui/src/components/Sidebar.tsx"), "change");
|
|
git("add", "."); git("commit", "-qm", "PR");
|
|
const prHead = git("rev-parse", "HEAD");
|
|
git("checkout", "-q", "main");
|
|
writeFileSync(path.join(repository, "unrelated-master"), "already merged");
|
|
git("add", "."); git("commit", "-qm", "advance base");
|
|
const base = git("rev-parse", "HEAD");
|
|
git("merge", "--no-ff", "pr", "-m", "tested merge");
|
|
const merge = git("rev-parse", "HEAD");
|
|
writeFileSync(path.join(directory, "git"), `#!/usr/bin/env node
|
|
const {spawnSync}=require('node:child_process');
|
|
if(process.env.TEST_GIT_FAILURE==='true')process.exit(1);
|
|
const args=process.argv.slice(2).map(a=>a==='https://github.com/paperclipai/paperclip.git'?process.env.TEST_GIT_REPOSITORY:a);
|
|
const r=spawnSync(${JSON.stringify(gitBin)},args,{stdio:'inherit'});process.exit(r.status??1);
|
|
`);
|
|
chmodSync(path.join(directory, "git"), 0o755);
|
|
const gh = path.join(directory, "gh");
|
|
writeFileSync(gh, `#!/usr/bin/env node
|
|
const fs = require('node:fs'), path = require('node:path');
|
|
const endpoint = process.argv.find(arg => arg.startsWith('/repos/'));
|
|
if (endpoint?.endsWith('/git/ref/heads/master')) console.log('a'.repeat(40));
|
|
else if (endpoint?.includes('/contents/')) {
|
|
if (process.env.TEST_FETCH_FAILURE === 'true') process.exit(1);
|
|
const [file, query] = endpoint.split('/contents/')[1].split('?');
|
|
if (query !== 'ref=' + 'a'.repeat(40)) process.exit(2);
|
|
process.stdout.write(fs.readFileSync(path.join(process.env.TEST_TRUSTED_ROOT, file)));
|
|
}
|
|
else process.exit(3);
|
|
`);
|
|
chmodSync(gh, 0o755);
|
|
for (const overrides of [{}, { TEST_FETCH_FAILURE: "true" }, { TEST_GIT_FAILURE: "true" }, { PROFILE_CI: "true" }, { STACK_JSON: "{}" }, { FULL_CI: "false" }]) {
|
|
const output = path.join(directory, "output");
|
|
writeFileSync(output, "");
|
|
const result = spawnSync("bash", ["-c", shell], { encoding: "utf8", env: {
|
|
...process.env, PATH: `${directory}:${process.env.PATH}`, TEST_TRUSTED_ROOT: root.pathname,
|
|
RUNNER_TEMP: directory, FULL_CI: "true", PROFILE_CI: "false", FORCE_FULL_CI: "false",
|
|
CI_INCREMENTAL_MODE: "enabled", STACK_JSON: "null", EVENT_BASE_SHA: "b".repeat(40), EVENT_HEAD_SHA: prHead, EVENT_MERGE_SHA: merge,
|
|
TEST_GIT_REPOSITORY: repository,
|
|
GITHUB_OUTPUT: output, GITHUB_STEP_SUMMARY: "", ...overrides,
|
|
} });
|
|
assert.equal(result.status, 0, result.stderr);
|
|
const value = readFileSync(output, "utf8").trim();
|
|
if (overrides.TEST_FETCH_FAILURE || overrides.TEST_GIT_FAILURE || overrides.FULL_CI === "false") assert.equal(value, "plan=");
|
|
else {
|
|
assert.notEqual(value, "plan=", `${JSON.stringify(overrides)}: ${result.stderr}`);
|
|
const resultPlan = JSON.parse(value.slice("plan=".length));
|
|
assert.equal(resultPlan.jobs.general_tests.selected, overrides.PROFILE_CI || overrides.STACK_JSON ? 19 : 2);
|
|
assert.equal(resultPlan.diff.base, base);
|
|
assert.deepEqual(resultPlan.changed, ["ui/src/components/Sidebar.tsx"]);
|
|
}
|
|
}
|
|
} finally { rmSync(directory, { recursive: true, force: true }); }
|
|
});
|
|
|
|
|
|
test("audited prose does not widen source selection, but guidance still runs its uncached guard", () => {
|
|
const source = "ui/src/components/Sidebar.tsx";
|
|
const sourcePlan = plan([source]);
|
|
for (const doc of ["doc/DEVELOPING.md", "doc/DATABASE.md", "doc/plans/2026-10-30-example.md"]) {
|
|
const mixed = plan([source, doc]);
|
|
assert.equal(mixed.forceReason, null);
|
|
assert.deepEqual(mixed.jobs, sourcePlan.jobs);
|
|
assert.deepEqual(mixed.documentation, [doc]);
|
|
}
|
|
const guidance = plan(["doc/DEVELOPING.md"]);
|
|
assert.deepEqual([...selectedTasks(guidance)], ["ci:typecheck"]);
|
|
assert.equal(config.tasks["//#ci:typecheck"].cache, false);
|
|
assert.deepEqual([...selectedTasks(plan(["doc/plans/2026-10-30-example.md"]))], []);
|
|
for (const file of ["docs/api/issues.md", "doc/unknown.md", "doc/plans/tool.sh", "skills/paperclip/SKILL.md",
|
|
"server/src/onboarding-assets/default/AGENTS.md", "tests/fixtures/example.md"]) {
|
|
assert.ok(selectedTasks(plan([source, file])).has("ci:server"), file);
|
|
}
|
|
});
|
|
|
|
test("complete merge trees handle more than 300 files, base advancement, renames and deletions", (t) => {
|
|
const directory = mkdtempSync(path.join(tmpdir(), "ci-merge-diff-"));
|
|
t.after(() => rmSync(directory, { recursive: true, force: true }));
|
|
const git = (...args) => execFileSync("git", args, { cwd: directory, encoding: "utf8" }).trim();
|
|
const write = (file, value) => { mkdirSync(path.dirname(path.join(directory, file)), { recursive: true }); writeFileSync(path.join(directory, file), value); };
|
|
const commit = () => { git("add", "."); git("commit", "-qm", "fixture"); return git("rev-parse", "HEAD"); };
|
|
git("init", "-q", "-b", "main"); git("config", "user.name", "CI"); git("config", "user.email", "ci@example.invalid");
|
|
write("ui/src/deleted.ts", "old"); write("ui/src/renamed.ts", "rename"); const staleBase = commit();
|
|
git("checkout", "-qb", "pr");
|
|
for (let index = 0; index < 350; index++) write(`ui/src/component-${index}.ts`, "new");
|
|
git("rm", "ui/src/deleted.ts");
|
|
mkdirSync(path.join(directory, "doc/plans"), { recursive: true });
|
|
git("mv", "ui/src/renamed.ts", "doc/plans/2026-10-30-moved.md");
|
|
const prHead = commit();
|
|
git("checkout", "-q", "main"); write(".github/workflows/unrelated.yml", "already merged CI change"); const base = commit();
|
|
git("merge", "--no-ff", "pr", "-m", "tested merge"); const head = git("rev-parse", "HEAD");
|
|
const result = mergeDiff(head, prHead, directory);
|
|
assert.equal(result.diff.base, base); assert.notEqual(result.diff.base, staleBase);
|
|
assert.equal(result.changed.length, 353);
|
|
assert.ok(result.changed.includes("ui/src/deleted.ts")); assert.ok(result.changed.includes("ui/src/renamed.ts"));
|
|
assert.ok(!result.changed.includes(".github/workflows/unrelated.yml"));
|
|
assert.equal(plan(result.changed).forceReason, null);
|
|
const shallow = path.join(directory, "shallow");
|
|
git("clone", "--quiet", "--depth", "2", `file://${directory}`, shallow);
|
|
assert.equal(execFileSync("git", ["rev-parse", "--is-shallow-repository"], { cwd: shallow, encoding: "utf8" }).trim(), "true");
|
|
assert.deepEqual(mergeDiff(head, prHead, shallow), result);
|
|
assert.throws(() => mergeDiff(head, base, directory), /expected PR head/);
|
|
assert.throws(() => mergeDiff(prHead, prHead, directory), /merge commit/);
|
|
assert.throws(() => mergeDiff("f".repeat(40), prHead, directory));
|
|
});
|