mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
fix(ci): keep the typecheck job working on stacked PRs and stop its CLI guidance test timing out (#21451)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the required `Typecheck + Release Registry` job from the trusted workflow on `master` > - #21404 replaced that job's six sequential steps with one script, `scripts/ci-typecheck-lane.mjs`, that runs independent checks in parallel > - PR runs take the workflow from `master` but check out the PR's merge commit. A stacked PR's merge commit is built on its base branch, so it can lack a script that only `master` has > - The lane also ran the CLI guidance test beside the typechecks at a lower CPU priority, and one of its cases is slow enough to cross its 15 second timeout there > - This pull request makes the job fall back to the earlier commands when the script is absent, and runs the CLI guidance test before the parallel checks > - The benefit is that stacked PRs typecheck again and the job stops failing on a timeout that has nothing to do with the PR ## Linked Issues or Issue Description Refs #21404 **What happened?** In the first eight PR runs after #21404 merged (2026-11-13 18:06Z), the typecheck job failed three times for reasons unrelated to the PRs: - Twice on #20617, a PR stacked on `feat/ssh-agent-ai-accounts`: `Error: Cannot find module '/home/runner/_work/paperclip/paperclip/scripts/ci-typecheck-lane.mjs'` (jobs 132076887259 and 132079570039). The job failed 0 seconds into the step. - Once on run 43480469363 (job 132075678943): `CLI guidance safety failed with exit code 1`, from `Error: Test timed out in 15000ms.` in `allows only exact-allowlist pnpm paperclipai commands on every guidance surface`, which took 17059 ms. The same case passes in about 5 seconds when the test runs alone. **Expected behavior** The typecheck job passes or fails on the PR's own code, including for stacked PRs. **Actual behavior** Stacked PRs always fail the job. Other PRs fail it intermittently (1 of 6 runs that reached the lane). **Steps to reproduce** Open a PR whose base branch does not contain `scripts/ci-typecheck-lane.mjs`, or run the lane on a loaded 4 vCPU runner. ## What Changed - `.github/workflows/pr-trusted.yml`: the lane step runs only when `scripts/ci-typecheck-lane.mjs` exists in the checkout. A new step runs the six earlier commands, in their earlier order, when it does not. Only the `typecheck_release_registry` job changed. - `scripts/ci-typecheck-lane.mjs`: the CLI guidance test moved from the parallel checks to the end of the sequential preparation steps, at normal priority. - Tests: `scripts/__tests__/ci-typecheck-lane.test.mjs` and `.github/scripts/tests/ci-plan.test.mjs` assert the two conditions, that the earlier commands appear only in the fallback step, and that the CLI guidance test is a preparation step. - `doc/DEVELOPING.md`: the "Typecheck lane in CI" section describes both. ## Verification - `node --test scripts/__tests__/ci-typecheck-lane.test.mjs`: 8 pass. - `node --test '.github/scripts/tests/*.test.mjs'` (the `policy` job's command): 807 pass. - `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`: 11 pass. - Not exercised by this PR's own CI: PR runs use the workflow from `master`, so the new conditions first run after merge. The script change does run here, because the script is read from the PR's checkout. - Not measured: the lane's wall time with the CLI guidance test moved. The test takes 2 to 5 seconds alone, so the expected cost is that much. ## Risks - The fallback step repeats six commands in the workflow. It is needed only until open stacked PRs are based on a branch that has the script, and can be removed later. - `hashFiles` returns an empty string when no file matches, which is what the two conditions rely on. - If a future PR deletes or renames the script, the job silently uses the slower commands instead of failing. The wiring tests would fail in that PR. ## Model Used Claude Fable 5.2 (`claude-fable-5-2`), via Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
1 parent
aa4a06653c
commit
c6301c92ff
5 files changed
+48
-12
No files matched your search
@@ -64,7 +64,17 @@ test("repository-wide guidance guards run uncached for every scanned UI file typ
|
||||
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
|
||||
const job = workflow.split("\n typecheck_release_registry:\n")[1].split("\n general_tests:\n")[0];
|
||||
assert.match(job, /\n run: node scripts\/ci-typecheck-lane\.mjs\n/);
|
||||
const guidance = planLane({ buildGapPackages: [], testLayers: [] }).tasks.find((task) => task.name === "CLI guidance safety");
|
||||
assert.match(job, /\n if: \$\{\{ hashFiles\('scripts\/ci-typecheck-lane\.mjs'\) != '' \}\}\n run: node scripts\/ci-typecheck-lane\.mjs\n/);
|
||||
const fallback = job.split("\n if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}\n")[1].split("\n\n")[0];
|
||||
for (const command of [
|
||||
"pnpm run typecheck:build-gaps",
|
||||
"node scripts/typecheck-test-programs.mjs --low-memory",
|
||||
"pnpm --filter @paperclipai/server typecheck",
|
||||
"node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations",
|
||||
"server/src/__tests__/cli-invocation-safety.test.ts",
|
||||
"pnpm run test:release-registry",
|
||||
]) assert.ok(fallback.includes(command), command);
|
||||
const guidance = planLane({ buildGapPackages: [], testLayers: [] }).prepare.find((task) => task.name === "CLI guidance safety");
|
||||
assert.deepEqual(guidance.args, ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"]);
|
||||
assert.ok(!job.includes("uses: ./.github/actions/run-ci-task"));
|
||||
});
|
||||
|
||||
@@ -463,8 +463,21 @@ jobs:
|
||||
run: node scripts/runner-release-crate-cache.mjs mark-current
|
||||
|
||||
- name: Typecheck, CLI guidance safety and release registry tests
|
||||
if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') != '' }}
|
||||
run: node scripts/ci-typecheck-lane.mjs
|
||||
|
||||
- name: Typecheck a checkout that predates the typecheck lane script
|
||||
if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}
|
||||
env:
|
||||
PAPERCLIP_TYPECHECK_LOW_MEMORY: "1"
|
||||
run: |
|
||||
pnpm run typecheck:build-gaps
|
||||
node scripts/typecheck-test-programs.mjs --low-memory
|
||||
pnpm --filter @paperclipai/server typecheck
|
||||
node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations
|
||||
pnpm exec vitest run --project @paperclipai/server server/src/__tests__/cli-invocation-safety.test.ts
|
||||
pnpm run test:release-registry
|
||||
|
||||
- name: Upload CI profile timings
|
||||
continue-on-error: true
|
||||
if: always() && env.PAPERCLIP_CI_PROFILE == 'true'
|
||||
|
||||
+9
-3
@@ -1564,13 +1564,15 @@ It first runs the steps that write shared outputs, one at a time:
|
||||
(`typecheck-test-programs.mjs --prepare-only`).
|
||||
4. Typecheck the Runner. Its typecheck rebuilds the eval kernel, which other
|
||||
checks read, so it cannot run beside them.
|
||||
5. Run the CLI guidance test. It runs here, before the parallel checks, because
|
||||
one of its cases took 17 seconds beside them against a 15 second timeout.
|
||||
|
||||
Then it runs the remaining checks in parallel, as many as the machine has CPUs
|
||||
(default) or `--concurrency`. Each `tsc` uses one thread. The server
|
||||
declaration layers start first, because the test layers and the test projects
|
||||
read them. The test programs, the other build-gap workspaces and the CLI
|
||||
guidance test run at a lower CPU priority, so they use only the CPU that the
|
||||
server layers and the release registry tests leave free.
|
||||
read them. The test programs and the other build-gap workspaces run at a lower
|
||||
CPU priority, so they use only the CPU that the server layers and the release
|
||||
registry tests leave free.
|
||||
|
||||
A passing check prints its output in a collapsed log group. A failing check
|
||||
prints its output in full, between `=== FAILED ...` and `=== end of ...`
|
||||
@@ -1581,6 +1583,10 @@ nothing else runs.
|
||||
|
||||
To add a check to the job, add it to `planLane` in the script.
|
||||
|
||||
PR runs take the workflow from `master` but check out the PR's merge commit. A
|
||||
stacked PR, whose base is another branch, can have a merge commit without this
|
||||
script. The job then runs the six earlier commands one after another instead.
|
||||
|
||||
#### Typecheck test files outside the build
|
||||
|
||||
Most package tsconfigs exclude tests, and Vitest strips types, so a type error in
|
||||
|
||||
@@ -25,6 +25,9 @@ test("the shared outputs are written before the parallel checks start", () => {
|
||||
assert.ok(prepare.some((task) => task.name === `typecheck ${name}`), name);
|
||||
assert.ok(!tasks.some((task) => task.name === `typecheck ${name}`), name);
|
||||
}
|
||||
const guidance = prepare.find((task) => task.name === "CLI guidance safety");
|
||||
assert.ok(guidance.args.includes("server/src/__tests__/cli-invocation-safety.test.ts"));
|
||||
assert.equal(guidance.background, undefined);
|
||||
const programs = tasks.find((task) => task.name === "typecheck test programs");
|
||||
assert.ok(programs.args.includes("--skip-prepare"));
|
||||
for (const task of tasks) {
|
||||
@@ -48,7 +51,7 @@ test("every check of the sequential job is planned, and only declaration readers
|
||||
}
|
||||
for (const name of ["paperclipai", "@paperclipai/ui"]) assert.ok(byName.has(`typecheck ${name}`), name);
|
||||
assert.deepEqual(byName.get("release registry tests").args, ["run", "test:release-registry"]);
|
||||
assert.ok(byName.get("CLI guidance safety").args.includes("server/src/__tests__/cli-invocation-safety.test.ts"));
|
||||
assert.ok(!byName.has("CLI guidance safety"));
|
||||
const waiting = tasks.filter((task) => task.needs);
|
||||
assert.equal(waiting.length, 2 + PROJECTS.length);
|
||||
assert.deepEqual(
|
||||
@@ -146,7 +149,11 @@ test("the PR typecheck job runs the lane after the Runner release crate restore"
|
||||
const job = workflow.split("\n typecheck_release_registry:\n")[1].split("\n general_tests:\n")[0];
|
||||
const lane = job.indexOf("run: node scripts/ci-typecheck-lane.mjs");
|
||||
assert.ok(lane > job.indexOf("- name: Mark the restored Runner release crate current"));
|
||||
const missingScript = "if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}";
|
||||
const [beforeFallback, fallback] = job.split(missingScript);
|
||||
assert.ok(beforeFallback.includes("if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') != '' }}\n run: node scripts/ci-typecheck-lane.mjs"));
|
||||
for (const command of ["typecheck:build-gaps", "typecheck-test-programs.mjs", "typecheck-test-projects.mjs", "test:release-registry"]) {
|
||||
assert.ok(!job.includes(command), command);
|
||||
assert.ok(!beforeFallback.includes(command), command);
|
||||
assert.ok(fallback.includes(command), command);
|
||||
}
|
||||
});
|
||||
@@ -11,6 +11,7 @@ import { PROJECTS } from "./typecheck-test-projects.mjs";
|
||||
export const SERIAL_BUILD_GAPS = ["@paperclipai/paperclip-runner"];
|
||||
export const BACKGROUND_PRIORITY = 10;
|
||||
const SERVER_DECLARATIONS = "server declarations";
|
||||
export const CLI_GUIDANCE_SAFETY = "CLI guidance safety";
|
||||
const lowMemory = { [LOW_MEMORY_VARIABLE]: "1" };
|
||||
|
||||
export function planLane({ buildGapPackages, testLayers, projects = PROJECTS }) {
|
||||
@@ -37,6 +38,11 @@ export function planLane({ buildGapPackages, testLayers, projects = PROJECTS })
|
||||
},
|
||||
{ name: "install test program dependencies", command: "node", args: ["scripts/typecheck-test-programs.mjs", "--prepare-only"] },
|
||||
...buildGapPackages.filter((name) => SERIAL_BUILD_GAPS.includes(name)).map((name) => typecheckPackage(name, false)),
|
||||
{
|
||||
name: CLI_GUIDANCE_SAFETY,
|
||||
command: "pnpm",
|
||||
args: ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"],
|
||||
},
|
||||
];
|
||||
const needsDeclarations = [SERVER_DECLARATIONS];
|
||||
const tasks = [
|
||||
@@ -67,12 +73,6 @@ export function planLane({ buildGapPackages, testLayers, projects = PROJECTS })
|
||||
needs: needsDeclarations,
|
||||
})),
|
||||
...buildGapPackages.filter((name) => !SERIAL_BUILD_GAPS.includes(name)).map((name) => typecheckPackage(name, true)),
|
||||
{
|
||||
name: "CLI guidance safety",
|
||||
command: "pnpm",
|
||||
args: ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"],
|
||||
background: true,
|
||||
},
|
||||
];
|
||||
return { prepare, tasks };
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user