mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip coordinates agents that make and verify repository changes. > - Local verification currently defaults to the complete Vitest suite. > - Contributor instructions also require broad local checks at each PR handoff. > - CI already owns conservative affected-test selection and broad final-revision checks. > - This pull request adds local selection and pass reuse from those existing contracts. > - Agents can get focused feedback and use CI to close the remaining coverage gaps. ## Linked Issues or Issue Description **Subsystem affected** Developer verification scripts and contributor guidance. Related selection work: #18398 and #18464. **Problem or motivation** An agent that repeats an unchanged local verification step starts the complete suite again. The handoff guidance also duplicates broad CI coverage locally. **Proposed solution** Add `pnpm verify` for the actual working tree. Select affected Vitest specifications with the CI task policy and dependency graph. Reuse successful checks whose inputs are modeled. Run opaque checks fresh. Report exit 2 when broader verification is required. Update the contributor contract to accept applicable final-revision CI coverage. **Alternatives considered** Plain `vitest related` omits filesystem and opaque dependencies. Commit-only cache keys miss dirty files. Running full local suites at every handoff repeats CI work. **Roadmap alignment** This is repository development tooling. It does not change the product roadmap. ## What Changed - Add `pnpm verify`, `--explain`, `--base`, `--full`, and `--no-cache`. - Reuse CI selection and exact runner ownership for dirty, staged, committed, deleted, renamed, and untracked inputs. - Keep native server suites and serialized suites in their local owners. Keep six sequential chat shards. - Cache selected passes within one worktree. Hash source, fixtures, installed package contents, dependency bindings and installation metadata, dotenv files, child environment, toolchains, platform, and selection. - Run opaque consumers fresh. Preserve failed, interrupted, mutated-input, and partial-coverage outcomes. - Update AGENTS.md, CONTRIBUTING.md, and DEVELOPING.md guidance. Record the shared `verify` command as the sole exception to the package-script rule. Register the new tests in the existing CI contract-test command. - Add optional root/environment inputs to shared graph helpers. Existing CI defaults stay the same. ## Verification - Fourteen new integration tests pass locally across focused runs with real Vitest selection and execution; CI colors are covered. - 51 existing targeted selector/input, project-isolation, and stable-runner checks passed. Four relevant runner contracts also pass after the workspace change. - Repeated unchanged execution skips reusable tests. Source, environment, installed code, dependency bindings, and new untracked tests invalidate results. - Installed-code mutation with a restored modification time still invalidates a cached pass. Internal skill-directory links preserve reuse. - Workspace B/C tests prove discovery, reusable/fresh partition execution, pass reuse, and external-input failure. Chat selection produces exactly six single-worker partitions. - A deliberate unrelated failing test is omitted by narrow selection and caught by `--full`. - Red/green regressions prove fingerprints preserve file boundaries and reject edits made between diff discovery and snapshotting. - Opaque external-file changes still fail on a subsequent run. Source mutation and process interruption cannot publish reusable passes. - `pnpm verify --explain` works in the checkout and requests broad CI coverage for this tooling change. No duplicate full local suite was started. - [Final-revision CI](https://github.com/paperclipai/paperclip/actions/runs/42117291789) passed on `ed85d96ddff82c05c325948680b593b0f48c473d` (attempt 2). The scoped Canary retry preserved all passing test/build results. - [Greptile review](https://github.com/paperclipai/paperclip/pull/19345#issuecomment-6670050175) is 5/5; all four review threads are resolved. - The additional Paperclip review concluded neutral because its execution stopped; it produced no assessment. ## Risks - Pass reuse depends on conservative dependency classification. Opaque consumers, external database/live settings, and non-isolated projects retain fresh or broad coverage. - Local passes cover selected Vitest checks. Required CI, build/typecheck, release, browser, and platform acceptance remain separate obligations. - Cross-worktree caching and cross-agent scheduling are outside this change. - The initial installation scan reads dependency contents; stat-validated, worktree-local digest memoization avoids rereading unchanged contents. Subsequent scans still inspect file identity and nanosecond change/modification times. - No dependency lockfile or workflow changes. ## Model Used OpenAI Codex, `gpt-6.1-sol`, max reasoning, with local shell and file tools. Context window size is not exposed by this host. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
87 lines
4.0 KiB
JavaScript
87 lines
4.0 KiB
JavaScript
import { spawn } from "node:child_process";
|
|
import { mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
|
|
import path from "node:path";
|
|
import { createHash, randomUUID } from "node:crypto";
|
|
import { selectionHash } from "../ci/test-selection-manifest.mjs";
|
|
import { protocolVersion } from "./inputs.mjs";
|
|
|
|
export const cacheDirectory = root => path.join(root, ".turbo/local-verify");
|
|
const planHash = plan => createHash("sha256").update(JSON.stringify(plan)).digest("hex");
|
|
|
|
export function readPass(root, key) {
|
|
try {
|
|
const result = JSON.parse(readFileSync(path.join(cacheDirectory(root), `${key}.json`), "utf8"));
|
|
return result.version === protocolVersion && result.key === key && result.status === "passed" && result.planHash === planHash(result.plan)
|
|
&& ["selected", "full"].includes(result.plan?.kind) && Array.isArray(result.plan.jobs) ? result : null;
|
|
} catch { return null; }
|
|
}
|
|
|
|
export function readPlan(root, key) {
|
|
try {
|
|
const result = JSON.parse(readFileSync(path.join(cacheDirectory(root), `plan-${key}.json`), "utf8"));
|
|
return result.version === protocolVersion && result.key === key && result.status === "planned" && result.planHash === planHash(result.plan)
|
|
&& result.plan?.kind === "selected" && Array.isArray(result.plan.jobs) ? result.plan : null;
|
|
} catch { return null; }
|
|
}
|
|
|
|
export function mayReuseAttempt(root, key) {
|
|
try {
|
|
const result = JSON.parse(readFileSync(path.join(cacheDirectory(root), `attempt-${key}.json`), "utf8"));
|
|
return result.version === protocolVersion && result.key === key && result.status === "passed";
|
|
} catch (error) { return error.code === "ENOENT"; }
|
|
}
|
|
|
|
export function writeRecord(root, name, record) {
|
|
const directory = cacheDirectory(root);
|
|
mkdirSync(directory, { recursive: true });
|
|
const temporary = path.join(directory, `${name}.${randomUUID()}.tmp`);
|
|
writeFileSync(temporary, JSON.stringify({ ...record, planHash: planHash(record.plan) }, null, 2), { mode: 0o600 });
|
|
renameSync(temporary, path.join(directory, `${name}.json`));
|
|
}
|
|
|
|
export const invalidatePass = (root, key) => rmSync(path.join(cacheDirectory(root), `${key}.json`), { force: true });
|
|
|
|
async function execute(command, args, root, env, label) {
|
|
return new Promise(resolve => {
|
|
const grouped = process.platform !== "win32";
|
|
const child = spawn(command, args, { cwd: root, env, stdio: "inherit", detached: grouped });
|
|
let stopped;
|
|
let finished = false;
|
|
const interrupt = signal => {
|
|
stopped = signal;
|
|
try {
|
|
if (grouped && child.pid) process.kill(-child.pid, signal);
|
|
else child.kill(signal);
|
|
} catch (error) { if (error.code !== "ESRCH") throw error; }
|
|
};
|
|
const onInterrupt = () => interrupt("SIGINT"), onTerminate = () => interrupt("SIGTERM");
|
|
process.on("SIGINT", onInterrupt); process.on("SIGTERM", onTerminate);
|
|
const finish = code => {
|
|
if (finished) return;
|
|
finished = true;
|
|
process.off("SIGINT", onInterrupt); process.off("SIGTERM", onTerminate);
|
|
resolve(stopped ? stopped === "SIGINT" ? 130 : 143 : code);
|
|
};
|
|
child.once("error", error => { console.error(error.message); finish(1); });
|
|
child.once("close", code => finish(code ?? 1));
|
|
console.log(`[verify] Running ${label}`);
|
|
});
|
|
}
|
|
|
|
export async function runPlan(root, env, plan) {
|
|
const directory = cacheDirectory(root);
|
|
mkdirSync(directory, { recursive: true });
|
|
const manifestFile = path.join(directory, `selection-${randomUUID()}.json`);
|
|
if (plan.manifest) writeFileSync(manifestFile, JSON.stringify(plan.manifest));
|
|
try {
|
|
for (const job of plan.jobs) {
|
|
const code = job.args === null ? await execute("pnpm", ["test:run"], root, env, job.id)
|
|
: await execute(process.execPath, [path.join(root, "scripts/run-vitest-stable.mjs"), ...job.args], root,
|
|
job.selected ? { ...env, PAPERCLIP_CI_TEST_SELECTION: manifestFile,
|
|
PAPERCLIP_CI_SELECTION_HASH: selectionHash(plan.manifest) } : env, job.id);
|
|
if (code !== 0) return code;
|
|
}
|
|
return 0;
|
|
} finally { rmSync(manifestFile, { force: true }); }
|
|
}
|