fix(ci): keep the typecheck job working on stacked PRs and stop its CLI guidance test timing out (#21451)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every pull request runs the required `Typecheck + Release Registry`
job from the trusted workflow on `master`
> - #21404 replaced that job's six sequential steps with one script,
`scripts/ci-typecheck-lane.mjs`, that runs independent checks in
parallel
> - PR runs take the workflow from `master` but check out the PR's merge
commit. A stacked PR's merge commit is built on its base branch, so it
can lack a script that only `master` has
> - The lane also ran the CLI guidance test beside the typechecks at a
lower CPU priority, and one of its cases is slow enough to cross its 15
second timeout there
> - This pull request makes the job fall back to the earlier commands
when the script is absent, and runs the CLI guidance test before the
parallel checks
> - The benefit is that stacked PRs typecheck again and the job stops
failing on a timeout that has nothing to do with the PR

## Linked Issues or Issue Description

Refs #21404

**What happened?**
In the first eight PR runs after #21404 merged (2026-11-13 18:06Z), the
typecheck job failed three times for reasons unrelated to the PRs:

- Twice on #20617, a PR stacked on `feat/ssh-agent-ai-accounts`: `Error:
Cannot find module
'/home/runner/_work/paperclip/paperclip/scripts/ci-typecheck-lane.mjs'`
(jobs 132076887259 and 132079570039). The job failed 0 seconds into the
step.
- Once on run 43480469363 (job 132075678943): `CLI guidance safety
failed with exit code 1`, from `Error: Test timed out in 15000ms.` in
`allows only exact-allowlist pnpm paperclipai commands on every guidance
surface`, which took 17059 ms. The same case passes in about 5 seconds
when the test runs alone.

**Expected behavior**
The typecheck job passes or fails on the PR's own code, including for
stacked PRs.

**Actual behavior**
Stacked PRs always fail the job. Other PRs fail it intermittently (1 of
6 runs that reached the lane).

**Steps to reproduce**
Open a PR whose base branch does not contain
`scripts/ci-typecheck-lane.mjs`, or run the lane on a loaded 4 vCPU
runner.

## What Changed

- `.github/workflows/pr-trusted.yml`: the lane step runs only when
`scripts/ci-typecheck-lane.mjs` exists in the checkout. A new step runs
the six earlier commands, in their earlier order, when it does not. Only
the `typecheck_release_registry` job changed.
- `scripts/ci-typecheck-lane.mjs`: the CLI guidance test moved from the
parallel checks to the end of the sequential preparation steps, at
normal priority.
- Tests: `scripts/__tests__/ci-typecheck-lane.test.mjs` and
`.github/scripts/tests/ci-plan.test.mjs` assert the two conditions, that
the earlier commands appear only in the fallback step, and that the CLI
guidance test is a preparation step.
- `doc/DEVELOPING.md`: the "Typecheck lane in CI" section describes
both.

## Verification

- `node --test scripts/__tests__/ci-typecheck-lane.test.mjs`: 8 pass.
- `node --test '.github/scripts/tests/*.test.mjs'` (the `policy` job's
command): 807 pass.
- `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`:
11 pass.
- Not exercised by this PR's own CI: PR runs use the workflow from
`master`, so the new conditions first run after merge. The script change
does run here, because the script is read from the PR's checkout.
- Not measured: the lane's wall time with the CLI guidance test moved.
The test takes 2 to 5 seconds alone, so the expected cost is that much.

## Risks

- The fallback step repeats six commands in the workflow. It is needed
only until open stacked PRs are based on a branch that has the script,
and can be removed later.
- `hashFiles` returns an empty string when no file matches, which is
what the two conditions rely on.
- If a future PR deletes or renames the script, the job silently uses
the slower commands instead of failing. The wiring tests would fail in
that PR.

## Model Used

Claude Fable 5.2 (`claude-fable-5-2`), via Claude Code, with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Fable 5.2 authored and GitHub committed 2026-11-13 13:24:52 -06:00
1 parent aa4a06653c
commit c6301c92ff
5 files changed
+48 -12

No files matched your search

+11 -1
View File
@@ -64,7 +64,17 @@ test("repository-wide guidance guards run uncached for every scanned UI file typ
const workflow = readFileSync(new URL(".github/workflows/pr-trusted.yml", root), "utf8");
const job = workflow.split("\n typecheck_release_registry:\n")[1].split("\n general_tests:\n")[0];
assert.match(job, /\n run: node scripts\/ci-typecheck-lane\.mjs\n/);
const guidance = planLane({ buildGapPackages: [], testLayers: [] }).tasks.find((task) => task.name === "CLI guidance safety");
assert.match(job, /\n if: \$\{\{ hashFiles\('scripts\/ci-typecheck-lane\.mjs'\) != '' \}\}\n run: node scripts\/ci-typecheck-lane\.mjs\n/);
const fallback = job.split("\n if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}\n")[1].split("\n\n")[0];
for (const command of [
"pnpm run typecheck:build-gaps",
"node scripts/typecheck-test-programs.mjs --low-memory",
"pnpm --filter @paperclipai/server typecheck",
"node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations",
"server/src/__tests__/cli-invocation-safety.test.ts",
"pnpm run test:release-registry",
]) assert.ok(fallback.includes(command), command);
const guidance = planLane({ buildGapPackages: [], testLayers: [] }).prepare.find((task) => task.name === "CLI guidance safety");
assert.deepEqual(guidance.args, ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"]);
assert.ok(!job.includes("uses: ./.github/actions/run-ci-task"));
});
+13
View File
@@ -463,8 +463,21 @@ jobs:
run: node scripts/runner-release-crate-cache.mjs mark-current
- name: Typecheck, CLI guidance safety and release registry tests
if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') != '' }}
run: node scripts/ci-typecheck-lane.mjs
- name: Typecheck a checkout that predates the typecheck lane script
if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}
env:
PAPERCLIP_TYPECHECK_LOW_MEMORY: "1"
run: |
pnpm run typecheck:build-gaps
node scripts/typecheck-test-programs.mjs --low-memory
pnpm --filter @paperclipai/server typecheck
node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations
pnpm exec vitest run --project @paperclipai/server server/src/__tests__/cli-invocation-safety.test.ts
pnpm run test:release-registry
- name: Upload CI profile timings
continue-on-error: true
if: always() && env.PAPERCLIP_CI_PROFILE == 'true'
+9 -3
View File
@@ -1564,13 +1564,15 @@ It first runs the steps that write shared outputs, one at a time:
(`typecheck-test-programs.mjs --prepare-only`).
4. Typecheck the Runner. Its typecheck rebuilds the eval kernel, which other
checks read, so it cannot run beside them.
5. Run the CLI guidance test. It runs here, before the parallel checks, because
one of its cases took 17 seconds beside them against a 15 second timeout.
Then it runs the remaining checks in parallel, as many as the machine has CPUs
(default) or `--concurrency`. Each `tsc` uses one thread. The server
declaration layers start first, because the test layers and the test projects
read them. The test programs, the other build-gap workspaces and the CLI
guidance test run at a lower CPU priority, so they use only the CPU that the
server layers and the release registry tests leave free.
read them. The test programs and the other build-gap workspaces run at a lower
CPU priority, so they use only the CPU that the server layers and the release
registry tests leave free.
A passing check prints its output in a collapsed log group. A failing check
prints its output in full, between `=== FAILED ...` and `=== end of ...`
@@ -1581,6 +1583,10 @@ nothing else runs.
To add a check to the job, add it to `planLane` in the script.
PR runs take the workflow from `master` but check out the PR's merge commit. A
stacked PR, whose base is another branch, can have a merge commit without this
script. The job then runs the six earlier commands one after another instead.
#### Typecheck test files outside the build
Most package tsconfigs exclude tests, and Vitest strips types, so a type error in
+9 -2
View File
@@ -25,6 +25,9 @@ test("the shared outputs are written before the parallel checks start", () => {
assert.ok(prepare.some((task) => task.name === `typecheck ${name}`), name);
assert.ok(!tasks.some((task) => task.name === `typecheck ${name}`), name);
}
const guidance = prepare.find((task) => task.name === "CLI guidance safety");
assert.ok(guidance.args.includes("server/src/__tests__/cli-invocation-safety.test.ts"));
assert.equal(guidance.background, undefined);
const programs = tasks.find((task) => task.name === "typecheck test programs");
assert.ok(programs.args.includes("--skip-prepare"));
for (const task of tasks) {
@@ -48,7 +51,7 @@ test("every check of the sequential job is planned, and only declaration readers
}
for (const name of ["paperclipai", "@paperclipai/ui"]) assert.ok(byName.has(`typecheck ${name}`), name);
assert.deepEqual(byName.get("release registry tests").args, ["run", "test:release-registry"]);
assert.ok(byName.get("CLI guidance safety").args.includes("server/src/__tests__/cli-invocation-safety.test.ts"));
assert.ok(!byName.has("CLI guidance safety"));
const waiting = tasks.filter((task) => task.needs);
assert.equal(waiting.length, 2 + PROJECTS.length);
assert.deepEqual(
@@ -146,7 +149,11 @@ test("the PR typecheck job runs the lane after the Runner release crate restore"
const job = workflow.split("\n typecheck_release_registry:\n")[1].split("\n general_tests:\n")[0];
const lane = job.indexOf("run: node scripts/ci-typecheck-lane.mjs");
assert.ok(lane > job.indexOf("- name: Mark the restored Runner release crate current"));
const missingScript = "if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') == '' }}";
const [beforeFallback, fallback] = job.split(missingScript);
assert.ok(beforeFallback.includes("if: ${{ hashFiles('scripts/ci-typecheck-lane.mjs') != '' }}\n run: node scripts/ci-typecheck-lane.mjs"));
for (const command of ["typecheck:build-gaps", "typecheck-test-programs.mjs", "typecheck-test-projects.mjs", "test:release-registry"]) {
assert.ok(!job.includes(command), command);
assert.ok(!beforeFallback.includes(command), command);
assert.ok(fallback.includes(command), command);
}
});
+6 -6
View File
@@ -11,6 +11,7 @@ import { PROJECTS } from "./typecheck-test-projects.mjs";
export const SERIAL_BUILD_GAPS = ["@paperclipai/paperclip-runner"];
export const BACKGROUND_PRIORITY = 10;
const SERVER_DECLARATIONS = "server declarations";
export const CLI_GUIDANCE_SAFETY = "CLI guidance safety";
const lowMemory = { [LOW_MEMORY_VARIABLE]: "1" };
export function planLane({ buildGapPackages, testLayers, projects = PROJECTS }) {
@@ -37,6 +38,11 @@ export function planLane({ buildGapPackages, testLayers, projects = PROJECTS })
},
{ name: "install test program dependencies", command: "node", args: ["scripts/typecheck-test-programs.mjs", "--prepare-only"] },
...buildGapPackages.filter((name) => SERIAL_BUILD_GAPS.includes(name)).map((name) => typecheckPackage(name, false)),
{
name: CLI_GUIDANCE_SAFETY,
command: "pnpm",
args: ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"],
},
];
const needsDeclarations = [SERVER_DECLARATIONS];
const tasks = [
@@ -67,12 +73,6 @@ export function planLane({ buildGapPackages, testLayers, projects = PROJECTS })
needs: needsDeclarations,
})),
...buildGapPackages.filter((name) => !SERIAL_BUILD_GAPS.includes(name)).map((name) => typecheckPackage(name, true)),
{
name: "CLI guidance safety",
command: "pnpm",
args: ["exec", "vitest", "run", "--project", "@paperclipai/server", "server/src/__tests__/cli-invocation-safety.test.ts"],
background: true,
},
];
return { prepare, tasks };
}