mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 20:44:16 +02:00
ci: skip full CI for release-notes-only pull requests (#16207)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each beta release ends with a pull request that adds one release notes file under `releases/` > - PR CI runs the full build, test, and e2e matrix for that pull request > - No build, test, or e2e lane reads `releases/v*.md` or `releases/beta/v*.md` > - The full matrix takes about 30 minutes, and lost runners or flaky e2e specs can make it run again > - This pull request lets the existing CI scope gate skip the heavy lanes when a pull request changes only release notes > - The benefit is that release notes pull requests get a green result in minutes, and code pull requests keep full CI ## Linked Issues or Issue Description **Issue type** CI efficiency enhancement. **Where is the issue?** The `Select stacked PR CI scope` step in `.github/workflows/ci-routing.yml`. This step sets the `full_ci` output for PR CI. **What's wrong?** The step sets `full_ci=false` only for middle pull requests in a stack. A pull request that changes only one release notes markdown file runs the full matrix. For example, #16053 changed one file in `releases/beta/`. Its CI ran two times: on the first run, 8 shards lost their runners, and on the second run, one Playwright spec flaked and needed a rerun. **Suggested fix** Read the pull request diff from the GitHub compare API in the scope step. When every changed path is release notes, set `full_ci=false`. The callers already support this value. I found no open pull request or issue for this change. ## What Changed - `ci-routing.yml`: the scope step (renamed to `Select PR CI scope`) also sets `full_ci=false` when every path in the diff matches `releases/v*.md` or `releases/beta/v*.md`. - The step reads the diff from `compare/<event base SHA>...<event head SHA>`. The SHAs come from the event, so a later push cannot change the decision for this run. - For a renamed file, the step checks the old path and the new path. - The step keeps full CI when the event is not `pull_request`, a SHA is malformed, the API call fails, the diff is empty, or the file list has 300 entries. The compare API can truncate a list at 300 entries. - The step does not change the stacked pull request rules. It checks the paths only when the stack rules select full CI. - `ci-routing.test.mjs`: the GitHub stub now returns `files` for compare calls. 17 new cases cover release notes paths, mixed diffs, renames, non-markdown files, nested folders, empty and truncated lists, API failure, non-PR events, malformed SHAs, and stack interaction. - `e2e-shard.test.mjs`: the test now uses the new step name. - `doc/DEVELOPING.md`: added a short description of the CI scope rules. ## Verification - `node --test .github/scripts/tests/ci-routing.test.mjs scripts/__tests__/e2e-shard.test.mjs`: 73 pass, 0 fail. - `node --test '.github/scripts/tests/*.test.mjs'`: all pass. - I ran the jq filter on real compare API responses. #16053 (one file in `releases/beta/`) returns `true`. #16082 (10 files) returns `false`. - After merge: open a pull request that changes only a `releases/beta/v*.md` file. The `Select trusted runner` job shows the notice `release notes only; full_ci=false`. The heavy lanes show as skipped. The `policy`, `verify`, and `e2e` checks pass. Note: `pr.yml` calls `pr-trusted.yml@master`, and that workflow calls `ci-routing.yml@master`. This pull request does not change its own CI run. The change applies after merge. ## Risks - Low risk. The skip uses the existing `full_ci=false` path. The `verify` and `e2e` aggregate checks already accept that value, and the `policy` job always runs. - A future job that reads release notes files would not run for notes-only pull requests. Today, only the release workflows read these files, and those workflows do not run on pull requests. - The step now uses `GH_TOKEN` to call one more read-only API endpoint. The workflow still does not check out or run pull request code, and its permissions do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Agentic coding with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
4 files changed
+131
-6
No files matched your search
@@ -29,7 +29,7 @@ test("routing exports both placement and stack scope to callers", () => {
|
||||
assert.match(jobs, /^ {6}full_ci: \$\{\{ steps\.scope\.outputs\.full_ci \}\}$/m);
|
||||
});
|
||||
|
||||
const githubStub = "#!/usr/bin/env bash\nset -Eeuo pipefail\n\nendpoint=\"${*: -1}\"\ncase \"$endpoint\" in\n */pulls/*)\n jq -cn \\\n --argjson author_id \"${TEST_PR_AUTHOR_ID:-34892728}\" \\\n --argjson repository_id \"${TEST_REPOSITORY_ID:-1170821064}\" \\\n --arg base_ref \"${TEST_BASE_REF:-master}\" \\\n --arg base_sha \"${TEST_PR_BASE_SHA:-$EVENT_BASE_SHA}\" \\\n --arg head_sha \"${TEST_PR_HEAD_SHA:-1111111111111111111111111111111111111111}\" \\\n --arg merge_sha \"${TEST_PR_MERGE_SHA:-4444444444444444444444444444444444444444}\" '{\n state: \"open\",\n user: {id: $author_id},\n base: {repo: {id: $repository_id}, ref: $base_ref, sha: $base_sha},\n head: {sha: $head_sha},\n merge_commit_sha: $merge_sha\n }'\n ;;\n */git/ref/heads/*)\n jq -cn \\\n --arg sha \"${TEST_LIVE_BASE_REF_SHA:-$EVENT_BASE_SHA}\" '{object: {sha: $sha}}'\n ;;\n */compare/*)\n jq -cn \\\n --arg merge_base_sha \"${TEST_BASE_COMPARISON_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\" \\\n --arg status \"${TEST_BASE_COMPARISON_STATUS:-ahead}\" '{\n merge_base_commit: {sha: $merge_base_sha},\n status: $status\n }'\n ;;\n */git/commits/*)\n requested_sha=\"${endpoint##*/}\"\n live_merge_sha=\"${TEST_PR_MERGE_SHA:-4444444444444444444444444444444444444444}\"\n synthetic_base_merge_sha=\"${TEST_SYNTHETIC_BASE_MERGE_SHA:-}\"\n if [[ \"$requested_sha\" == \"$EVENT_MERGE_SHA\" ]]; then\n tree_sha=\"${TEST_EVENT_MERGE_TREE_SHA:-5555555555555555555555555555555555555555}\"\n base_sha=\"${TEST_EVENT_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\"\n head_sha=\"${TEST_EVENT_MERGE_HEAD_SHA:-$EVENT_HEAD_SHA}\"\n elif [[ \"$requested_sha\" == \"$live_merge_sha\" ]]; then\n tree_sha=\"${TEST_LIVE_MERGE_TREE_SHA:-5555555555555555555555555555555555555555}\"\n base_sha=\"${TEST_LIVE_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\"\n head_sha=\"${TEST_LIVE_MERGE_HEAD_SHA:-$EVENT_HEAD_SHA}\"\n elif [[ -n \"$synthetic_base_merge_sha\" && \"$requested_sha\" == \"$synthetic_base_merge_sha\" ]]; then\n tree_sha=\"${TEST_SYNTHETIC_BASE_MERGE_TREE_SHA:-8888888888888888888888888888888888888888}\"\n base_sha=\"${TEST_SYNTHETIC_MASTER_PARENT_SHA:-9999999999999999999999999999999999999999}\"\n head_sha=\"${TEST_SYNTHETIC_BASE_REF_PARENT_SHA:-${TEST_LIVE_BASE_REF_SHA:-$EVENT_BASE_SHA}}\"\n else\n echo \"unexpected merge commit: $requested_sha\" >&2\n exit 1\n fi\n jq -cn \\\n --arg sha \"$requested_sha\" \\\n --arg tree_sha \"$tree_sha\" \\\n --arg base_sha \"$base_sha\" \\\n --arg head_sha \"$head_sha\" '{\n sha: $sha,\n tree: {sha: $tree_sha},\n parents: [{sha: $base_sha}, {sha: $head_sha}]\n }'\n ;;\n */actions/runs/*)\n jq -cn \\\n --argjson actor_id \"${TEST_TRIGGERING_ACTOR_ID:-34892728}\" \\\n --argjson repository_id \"${TEST_REPOSITORY_ID:-1170821064}\" \\\n --arg event_name \"${TEST_EVENT_NAME:-pull_request}\" '{\n triggering_actor: {id: $actor_id},\n repository: {id: $repository_id},\n event: $event_name\n }'\n ;;\n *)\n echo \"unexpected gh endpoint: $endpoint\" >&2\n exit 1\n ;;\nesac";
|
||||
const githubStub = "#!/usr/bin/env bash\nset -Eeuo pipefail\n\nendpoint=\"${*: -1}\"\ncase \"$endpoint\" in\n */pulls/*)\n jq -cn \\\n --argjson author_id \"${TEST_PR_AUTHOR_ID:-34892728}\" \\\n --argjson repository_id \"${TEST_REPOSITORY_ID:-1170821064}\" \\\n --arg base_ref \"${TEST_BASE_REF:-master}\" \\\n --arg base_sha \"${TEST_PR_BASE_SHA:-$EVENT_BASE_SHA}\" \\\n --arg head_sha \"${TEST_PR_HEAD_SHA:-1111111111111111111111111111111111111111}\" \\\n --arg merge_sha \"${TEST_PR_MERGE_SHA:-4444444444444444444444444444444444444444}\" '{\n state: \"open\",\n user: {id: $author_id},\n base: {repo: {id: $repository_id}, ref: $base_ref, sha: $base_sha},\n head: {sha: $head_sha},\n merge_commit_sha: $merge_sha\n }'\n ;;\n */git/ref/heads/*)\n jq -cn \\\n --arg sha \"${TEST_LIVE_BASE_REF_SHA:-$EVENT_BASE_SHA}\" '{object: {sha: $sha}}'\n ;;\n */compare/*)\n [[ -z \"${TEST_COMPARE_FAIL:-}\" ]] || exit 1\n default_files='[{\"filename\":\"server/src/index.ts\"}]'\n jq -cn \\\n --arg merge_base_sha \"${TEST_BASE_COMPARISON_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\" \\\n --arg status \"${TEST_BASE_COMPARISON_STATUS:-ahead}\" \\\n --argjson files \"${TEST_COMPARE_FILES:-$default_files}\" '{\n merge_base_commit: {sha: $merge_base_sha},\n status: $status,\n files: $files\n }'\n ;;\n */git/commits/*)\n requested_sha=\"${endpoint##*/}\"\n live_merge_sha=\"${TEST_PR_MERGE_SHA:-4444444444444444444444444444444444444444}\"\n synthetic_base_merge_sha=\"${TEST_SYNTHETIC_BASE_MERGE_SHA:-}\"\n if [[ \"$requested_sha\" == \"$EVENT_MERGE_SHA\" ]]; then\n tree_sha=\"${TEST_EVENT_MERGE_TREE_SHA:-5555555555555555555555555555555555555555}\"\n base_sha=\"${TEST_EVENT_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\"\n head_sha=\"${TEST_EVENT_MERGE_HEAD_SHA:-$EVENT_HEAD_SHA}\"\n elif [[ \"$requested_sha\" == \"$live_merge_sha\" ]]; then\n tree_sha=\"${TEST_LIVE_MERGE_TREE_SHA:-5555555555555555555555555555555555555555}\"\n base_sha=\"${TEST_LIVE_MERGE_BASE_SHA:-$EVENT_BASE_SHA}\"\n head_sha=\"${TEST_LIVE_MERGE_HEAD_SHA:-$EVENT_HEAD_SHA}\"\n elif [[ -n \"$synthetic_base_merge_sha\" && \"$requested_sha\" == \"$synthetic_base_merge_sha\" ]]; then\n tree_sha=\"${TEST_SYNTHETIC_BASE_MERGE_TREE_SHA:-8888888888888888888888888888888888888888}\"\n base_sha=\"${TEST_SYNTHETIC_MASTER_PARENT_SHA:-9999999999999999999999999999999999999999}\"\n head_sha=\"${TEST_SYNTHETIC_BASE_REF_PARENT_SHA:-${TEST_LIVE_BASE_REF_SHA:-$EVENT_BASE_SHA}}\"\n else\n echo \"unexpected merge commit: $requested_sha\" >&2\n exit 1\n fi\n jq -cn \\\n --arg sha \"$requested_sha\" \\\n --arg tree_sha \"$tree_sha\" \\\n --arg base_sha \"$base_sha\" \\\n --arg head_sha \"$head_sha\" '{\n sha: $sha,\n tree: {sha: $tree_sha},\n parents: [{sha: $base_sha}, {sha: $head_sha}]\n }'\n ;;\n */actions/runs/*)\n jq -cn \\\n --argjson actor_id \"${TEST_TRIGGERING_ACTOR_ID:-34892728}\" \\\n --argjson repository_id \"${TEST_REPOSITORY_ID:-1170821064}\" \\\n --arg event_name \"${TEST_EVENT_NAME:-pull_request}\" '{\n triggering_actor: {id: $actor_id},\n repository: {id: $repository_id},\n event: $event_name\n }'\n ;;\n *)\n echo \"unexpected gh endpoint: $endpoint\" >&2\n exit 1\n ;;\nesac";
|
||||
|
||||
function stepScript(name) {
|
||||
const start = workflow.indexOf(` - name: ${name}\n`);
|
||||
@@ -137,12 +137,82 @@ const scopeCases = [
|
||||
|
||||
for (const [name, stack, base, expected] of scopeCases) {
|
||||
test(`scope decision: ${name}`, () => {
|
||||
assert.equal(executeStep("Select stacked PR CI scope", {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify(stack), PR_BASE_REF: base,
|
||||
}).full_ci, expected);
|
||||
});
|
||||
}
|
||||
|
||||
const files = (...paths) => JSON.stringify(paths.map((filename) => ({ filename })));
|
||||
const manyReleaseNotes = JSON.stringify(Array.from({ length: 300 }, (_, index) => ({
|
||||
filename: `releases/beta/v2026.1012.${index}-beta.0.md`,
|
||||
})));
|
||||
|
||||
const pathScopeCases = [
|
||||
["stable release notes", { TEST_COMPARE_FILES: files("releases/v2026.1015.0.md") }, "false"],
|
||||
["beta release notes draft", { TEST_COMPARE_FILES: files("releases/beta/v2026.1012.1-beta.0.md") }, "false"],
|
||||
["stable and beta notes", { TEST_COMPARE_FILES: files("releases/v2026.1015.0.md", "releases/beta/v2026.1012.1-beta.0.md") }, "false"],
|
||||
["removed release notes", { TEST_COMPARE_FILES: JSON.stringify([{ filename: "releases/beta/v2026.1006.0-beta.0.md", status: "removed" }]) }, "false"],
|
||||
["release notes and code", { TEST_COMPARE_FILES: files("releases/v2026.1015.0.md", "server/src/index.ts") }, "true"],
|
||||
["code renamed into releases", { TEST_COMPARE_FILES: JSON.stringify([{ filename: "releases/v2026.1015.0.md", previous_filename: "scripts/release.sh" }]) }, "true"],
|
||||
["non-markdown file in releases", { TEST_COMPARE_FILES: files("releases/beta/v2026.1012.1-beta.0.json") }, "true"],
|
||||
["nested releases directory", { TEST_COMPARE_FILES: files("releases/beta/drafts/v2026.1012.1-beta.0.md") }, "true"],
|
||||
["markdown outside releases", { TEST_COMPARE_FILES: files("doc/RELEASING.md") }, "true"],
|
||||
["file entry without a name", { TEST_COMPARE_FILES: JSON.stringify([{ status: "modified" }]) }, "true"],
|
||||
["empty diff", { TEST_COMPARE_FILES: "[]" }, "true"],
|
||||
["possibly truncated file list", { TEST_COMPARE_FILES: manyReleaseNotes }, "true"],
|
||||
["compare lookup failure", { TEST_COMPARE_FAIL: "1", TEST_COMPARE_FILES: files("releases/v2026.1015.0.md") }, "true"],
|
||||
["non-PR event", { EVENT_NAME: "workflow_dispatch", TEST_COMPARE_FILES: files("releases/v2026.1015.0.md") }, "true"],
|
||||
["malformed head SHA", { EVENT_HEAD_SHA: "invalid", TEST_COMPARE_FILES: files("releases/v2026.1015.0.md") }, "true"],
|
||||
];
|
||||
|
||||
for (const [name, env, expected] of pathScopeCases) {
|
||||
test(`path scope decision: ${name}`, () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: "null", PR_BASE_REF: "master", ...env,
|
||||
}).full_ci, expected);
|
||||
});
|
||||
}
|
||||
|
||||
test("path scope keeps full CI for the top PR in a stack that touches code", () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify({ position: 11, size: 11, base: { ref: "master" } }),
|
||||
PR_BASE_REF: "stack-10",
|
||||
}).full_ci, "true");
|
||||
});
|
||||
|
||||
test("path scope keeps full CI for release notes on top of code-changing stacked PRs", () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify({ position: 11, size: 11, base: { ref: "master" } }),
|
||||
PR_BASE_REF: "stack-10",
|
||||
TEST_COMPARE_FILES: files("releases/v2026.1015.0.md"),
|
||||
}).full_ci, "true");
|
||||
});
|
||||
|
||||
test("path scope keeps full CI for release notes with malformed stack metadata", () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify({ position: 0, size: 2, base: { ref: "master" } }),
|
||||
PR_BASE_REF: "master",
|
||||
TEST_COMPARE_FILES: files("releases/v2026.1015.0.md"),
|
||||
}).full_ci, "true");
|
||||
});
|
||||
|
||||
test("path scope skips full CI for the lowest PR in a stack that only touches release notes", () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify({ position: 1, size: 3, base: { ref: "master" } }),
|
||||
PR_BASE_REF: "master",
|
||||
TEST_COMPARE_FILES: files("releases/v2026.1015.0.md"),
|
||||
}).full_ci, "false");
|
||||
});
|
||||
|
||||
test("path scope skips full CI for a single-PR stack that only touches release notes", () => {
|
||||
assert.equal(executeStep("Select PR CI scope", {
|
||||
STACK_JSON: JSON.stringify({ position: 1, size: 1, base: { ref: "master" } }),
|
||||
PR_BASE_REF: "master",
|
||||
TEST_COMPARE_FILES: files("releases/v2026.1015.0.md"),
|
||||
}).full_ci, "false");
|
||||
});
|
||||
|
||||
|
||||
function routingPlacement(vars = {}, github = {}) {
|
||||
const source = "[34892728]";
|
||||
|
||||
Reference in new issue
Block a user