mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 22:24:16 +01:00
## Thinking Path > - Paperclip manages AI agents and needs reliable checks for each contribution. > - CI selects affected suites and then affected Vitest files before allocating runners. > - Recent runs compared a tested merge against a stale event base. > - That included unrelated master changes and sometimes reached the compare API file cap. > - Ordinary documentation also forced every suite and invalidated test caches. > - This PR reads complete merge trees and exempts a small set of audited prose inputs. > - Tests retain conservative coverage when the diff or an input is uncertain. ## Linked Issues or Issue Description Refs #18348 and #18214. This is a follow-up to the merged affected-test selector. Related CI performance work: #16513 and #16500 address setup and suite execution. This PR changes diff collection and input classification. ## What Changed - Fetch the exact tested merge and its parents into a temporary bare repository. Read the complete tree diff against the actual first parent. Validate the second parent against the event PR head. Never execute PR code in the routing job. - Carry the immutable base, merge and PR-head identities into the plan. Validate the same complete diff again before file selection. - Use one prose policy across the planner, Vitest selector and task cache adapter. Exempt only `doc/DEVELOPING.md`, `doc/DATABASE.md`, and Markdown under `doc/plans/`. - Keep the uncached typecheck and CLI guidance guard for the first two documents. Keep policy checks for historical plans. Keep existing coverage for all other docs, runtime skills, fixtures and unknown inputs. - Extend candidate-workflow qualification to exercise mixed source and prose changes. ## Verification - Passed: 15 planner and Git routing tests, including a 353-file diff, stale base, rename, deletion, wrong parent and fetch-failure cases. - Passed: 83 routing, Vitest graph, exact selection, workflow and real Turbo cache checks. - Passed: `pnpm -r typecheck` and `pnpm build`. - Full local `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL startup failures in native-session and Muse tests. It did not pass or complete. The exact cause remains unverified. The changed files do not include those tests or their database setup. Linux CI results are recorded below. - Passed: real mixed Inbox/source/prose graph qualification selected 699/911 workspace test files, preserving the source-only selection. - Passed: both GitHub CI qualification jobs, including real mixed source/prose Vitest selection. Greptile reviewed commit `10ac6f5ec9` at 5/5 with no findings. - First full CI attempt: the [chat shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253342684) received a runner shutdown signal. The [Runner shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253341271) failed the unchanged Antigravity authentication success assertion. All 11 cases in that file passed when run directly. That Runner lane does not use the changed selector/cache path. The exact assertion-failure cause remains unverified. The chat shard passed on the one retry. The authentication assertion also passed on retry, but the same Runner shard failed a different unchanged test: `native-distribution-integrity.test.ts:242`, "retains spawned native bytes until the exact child exits", exceeded 5000ms. The [retry log](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124259174201) preserves the failure. The Runner shard and required verify aggregate are still red; all other checks passed. All 33 native-distribution tests then passed in a direct local run. This shows recovery in isolation, not the cause of the Linux timeout. No further blind reruns were issued. - Replayed 15 real recent PR merge commits through a blob-filtered bare fetch. Nine needed a corrected base. The four API comparisons that returned 300 filenames became complete diffs of 3, 57, 66 and 54 files. The 3-file case (#18326) becomes eligible for graph selection. Most other samples still correctly use full CI because they include inputs outside this narrow prose policy. - Cache proof: prose edits preserve a product-test cache hit; changing runtime API documentation invalidates it. ## Risks - Prose exceptions must be removed or revised if product code starts reading those files. The allowlist is deliberately small. Runtime docs and skills remain covered. - A network or Git failure keeps the full-suite fallback. Unexpected merge topology also keeps full CI. Stack and force-full policies remain in place. - This does not claim an end-to-end speedup yet. Many mixed-content PRs still require full CI. The routing fetch reads only shallow commit/tree objects; it does not check out or execute PR files. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and code execution. The exact served model ID and context-window size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
119 lines
6.6 KiB
JavaScript
119 lines
6.6 KiB
JavaScript
import { appendFileSync, readFileSync, realpathSync } from "node:fs";
|
|
import { execFileSync } from "node:child_process";
|
|
import { matchesGlob } from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { documentationInput, productInputs } from "./ci-inputs.mjs";
|
|
import { mergeDiff } from "./ci-diff.mjs";
|
|
|
|
const releaseNote = /^releases\/(beta\/)?v[^/]+\.md$/;
|
|
const validPath = (file) => typeof file === "string" && file.length > 0
|
|
&& !file.startsWith("/") && !file.split("/").includes("..") && !file.includes("\0");
|
|
|
|
function validateGlobs(globs) {
|
|
if (!Array.isArray(globs) || globs.some((glob) => !validPath(glob) || /[!{}[\]?$\\]/.test(glob))) {
|
|
throw new Error("Unsupported task input pattern");
|
|
}
|
|
}
|
|
|
|
export function comparisonPaths(comparison) {
|
|
if (!Array.isArray(comparison?.files) || comparison.files.length === 0 || comparison.files.length >= 300) {
|
|
throw new Error("Missing, empty, or possibly truncated comparison");
|
|
}
|
|
return comparison.files.flatMap((file) => {
|
|
if (!validPath(file.filename) || (file.status === "renamed" && !validPath(file.previous_filename))
|
|
|| (file.previous_filename !== undefined && !validPath(file.previous_filename))) {
|
|
throw new Error("Malformed changed file");
|
|
}
|
|
return file.previous_filename ? [file.previous_filename, file.filename] : [file.filename];
|
|
});
|
|
}
|
|
|
|
export function planTasks({ config, lanes, paths, mode = "enabled", forceReason = null }) {
|
|
if (!["enabled", "shadow", "full"].includes(mode)) throw new Error("Invalid CI mode");
|
|
validateGlobs(config.globalDependencies);
|
|
const tasks = config.tasks;
|
|
const inputs = new Map();
|
|
function taskInputs(name, parents = []) {
|
|
if (inputs.has(name)) return inputs.get(name);
|
|
if (parents.includes(name)) throw new Error("Cyclic task graph");
|
|
const task = tasks[`//#${name}`];
|
|
if (!task) throw new Error(`Missing task: ${name}`);
|
|
validateGlobs(task.inputs);
|
|
const all = [...task.inputs];
|
|
for (const dependency of task.dependsOn ?? []) {
|
|
if (!dependency.startsWith("//#")) throw new Error("Unsupported task dependency");
|
|
all.push(...taskInputs(dependency.slice(3), [...parents, name]));
|
|
}
|
|
inputs.set(name, [...new Set(all)]);
|
|
return inputs.get(name);
|
|
}
|
|
for (const entries of Object.values(lanes)) {
|
|
if (!Array.isArray(entries) || !entries.length) throw new Error("Empty CI lane");
|
|
for (const entry of entries) taskInputs(entry.task);
|
|
}
|
|
const changed = [...new Set(paths)];
|
|
if (!changed.length || changed.some((file) => !validPath(file))) forceReason ??= "Missing or invalid diff";
|
|
const matches = (file, globs) => globs.some((glob) => matchesGlob(file, glob));
|
|
const sourceChanges = productInputs(changed);
|
|
const globalChanges = sourceChanges.filter((file) => matches(file, config.globalDependencies));
|
|
if (globalChanges.length) forceReason ??= `Global CI input changed: ${globalChanges[0]}`;
|
|
const unknown = sourceChanges.filter((file) => !releaseNote.test(file)
|
|
&& !matches(file, config.globalDependencies) && ![...inputs.values()].some((globs) => matches(file, globs)));
|
|
if (unknown.length) forceReason ??= `Unclassified path: ${unknown[0]}`;
|
|
if (mode === "full") forceReason ??= "Full CI requested";
|
|
const reasons = Object.fromEntries([...inputs].map(([task, globs]) => [task,
|
|
forceReason ? [forceReason] : [...new Set([...globalChanges, ...sourceChanges.filter((file) => matches(file, globs))])],
|
|
]));
|
|
// This uncached lane owns the repository-wide CLI guidance check.
|
|
if (reasons["ci:typecheck"]) reasons["ci:typecheck"].push(...changed.filter((file) => documentationInput(file) === "guidance"));
|
|
const jobs = Object.fromEntries(Object.entries(lanes).map(([job, entries]) => {
|
|
const affected = entries.filter((entry) => reasons[entry.task].length > 0);
|
|
const selected = mode === "shadow" ? entries : affected;
|
|
return [job, { run: selected.length > 0, matrix: { include: selected.length ? selected : entries },
|
|
selected: selected.length, proposed: affected.length, total: entries.length }];
|
|
}));
|
|
return { version: 1, mode, forceReason, changed, documentation: changed.filter(documentationInput), reasons, jobs };
|
|
}
|
|
|
|
export function planSummary(plan) {
|
|
const rows = Object.entries(plan.jobs).map(([job, lane]) =>
|
|
`| ${job} | ${lane.proposed}/${lane.total} | ${lane.selected}/${lane.total} |`);
|
|
return ["### Incremental CI", "", `Mode: **${plan.mode}**${plan.forceReason ? ` — ${plan.forceReason}` : ""}`,
|
|
...(plan.diff ? [`Diff: \`${plan.diff.base}\` → \`${plan.diff.head}\` (complete Git tree comparison)`, ""] : []),
|
|
`Documentation inputs: ${(plan.documentation ?? []).join(", ") || "none"}`,
|
|
"", "| Suite | Affected jobs | Scheduled jobs |", "| --- | ---: | ---: |", ...rows,
|
|
"", "<details><summary>Selection reasons</summary>", "", "```json", JSON.stringify(plan.reasons, null, 2),
|
|
"```", "</details>", ""].join("\n");
|
|
}
|
|
|
|
if (process.argv[1] && realpathSync(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
|
try {
|
|
const config = JSON.parse(readFileSync(process.env.CI_TASK_CONFIG ?? "turbo.json", "utf8"));
|
|
const lanes = JSON.parse(readFileSync(process.env.CI_TASK_MATRIX ?? ".github/ci-task-matrix.json", "utf8"));
|
|
let paths = [], diff, forceReason = process.env.CI_FORCE_REASON || null;
|
|
try {
|
|
if (process.env.CI_DIFF_REPOSITORY) {
|
|
const result = mergeDiff(process.env.EVENT_MERGE_SHA, process.env.EVENT_HEAD_SHA, process.env.CI_DIFF_REPOSITORY);
|
|
paths = result.changed;
|
|
diff = result.diff;
|
|
} else if (process.env.CI_COMPARISON) {
|
|
paths = comparisonPaths(JSON.parse(readFileSync(process.env.CI_COMPARISON, "utf8")));
|
|
} else {
|
|
const [base, head = "HEAD"] = process.argv.slice(2);
|
|
if (!base) throw new Error("Usage: ci-plan.mjs <base> [head]");
|
|
paths = execFileSync("git", ["diff", "--no-renames", "--name-only", "-z", `${base}...${head}`, "--"],
|
|
{ encoding: "utf8" }).split("\0").filter(Boolean);
|
|
}
|
|
} catch (error) { forceReason ??= `Diff unavailable: ${error.message}`; }
|
|
const plan = planTasks({ config, lanes, paths, mode: process.env.CI_INCREMENTAL_MODE || "enabled", forceReason });
|
|
if (diff) plan.diff = diff;
|
|
if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `plan=${JSON.stringify(plan)}\n`);
|
|
else console.log(JSON.stringify(plan, null, 2));
|
|
if (process.env.GITHUB_STEP_SUMMARY) appendFileSync(process.env.GITHUB_STEP_SUMMARY, planSummary(plan));
|
|
} catch (error) {
|
|
console.error(`Incremental CI unavailable; selecting full CI: ${error.message}`);
|
|
if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, "plan=\n");
|
|
else process.exitCode = 1;
|
|
}
|
|
}
|