mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner support suite proves the plugin SDK public types from pinned sources in `tests/runner-e2e/source-sdk-public-inputs.json` > - That registry pins the committed `pnpm-lock.yaml` blob and a `lockSha256` that must equal the `outputSha256` in `tests/runner-e2e/report-lock.json` > - The lockfile bot refresh #20615 changed the lock and `report-lock.json`, but the bot does not write the SDK registry > - The bot merges without CI, so the drift guard from #20579 could not block it, and every PR now fails `ci / Runner support suite` > - The guard also checked only the source blobs, so it did not name the `lockSha256` drift, and the sync script could not update `lockSha256` > - This pull request regenerates the registry, derives `lockSha256` from the reviewed policy, and makes the bot resync and check the registry in its own commit > - The benefit is that master CI is green again, and the next lockfile refresh cannot leave the SDK proof stale ## Linked Issues or Issue Description Refs #20615. Refs #20579. Refs #20550. Refs #20613. Overlaps #20618, which refreshes only the registry data. This PR has the same data change and also closes the gap that let the drift land. **What happened?** After #20615 merged, `ci / Runner support suite` failed on master and on every PR (for example runs 42877750315 and 42877750957, and PR #20613). `source-sdk-public-type.test.ts` and `source-sdk-evidence.test.ts` failed with "Reviewed SDK reference policy does not match public proof". `source-sdk-public-inputs.test.mjs` failed with "pnpm-lock.yaml (changed)". **Expected behavior** A lockfile refresh keeps the SDK public proof registry in step with the lock and the reviewed report-lock policy. If the registry drifts, the failure names the stale pin and the fix command. **Steps to reproduce** 1. Check out master at `3bf7af6c77`. 2. Run `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk`. 3. See the "Reviewed SDK reference policy does not match public proof" failure. ## What Changed - Root cause: #20615 changed `report-lock.json` from output `2360d5e9…` to `b3208306…` and changed the `pnpm-lock.yaml` blob from `851e65af…` to `6d825f97…`. The registry kept the old values. `refresh-lockfile.yml` commits only the lockfiles and the report-lock proof, and it merges without CI. - `source-sdk-public-inputs.json`: regenerated with `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --write`. Only `inputs["pnpm-lock.yaml"]` and `lockSha256` change. The compiler, parser and Zod pins do not change. - `source-sdk-public-inputs.mjs`: the sync now copies `lockSha256` from the committed `report-lock.json` `outputSha256`. It refuses when the committed lock is not that policy's reviewed input or output. This is the same rule that `prepareReviewedSdkReference` applies. Only the lockfile bot can write that policy, so a contributor cannot choose `lockSha256`. - `staleSdkPublicInputs()`: also reports a `lockSha256` that differs from the committed report-lock output. The guard test then names both stale pins and the `--write` command. - `source-sdk-reviewed-fixture.ts`: the "Reviewed SDK reference policy does not match public proof" error now names the two values and the fix command. The check itself is unchanged. - `refresh-lockfile.yml`: after the lock commit, the bot runs the SDK sync with `--write`, amends the commit, and runs `--check` before it pushes. `head_sha` is still read after the amend, so the bot merges the exact commit it checked. - Tests: new sync tests for a lockfile refresh and for an unreviewed lock, and a workflow-order test in `lockfile-refresh-merge.test.mjs`. - Docs: `doc/RELEASE-AUTOMATION-SETUP.md` §8.1 and `tests/runner-e2e/OPERATIONS.md` describe the resync. ## Verification - Reproduced on master: `source-sdk-public-type.test.ts` and `source-sdk-evidence.test.ts` failed with the policy mismatch. - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk prepare-report-lock sdk-canary workflow-security`: 9 files, 135 tests pass. - Before the registry change, the extended guard reported `pnpm-lock.yaml (changed), tests/runner-e2e/report-lock.json outputSha256 (lockSha256 changed)` with the fix command. - `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --check` reports `changed: false`. - `node --test .github/scripts/tests/lockfile-refresh-merge.test.mjs .github/scripts/tests/lockfile-refresh-workflows.test.mjs .github/scripts/tests/lockfile-trust.test.mjs`: 39 tests pass. - `pnpm test:e2e:runner:unit` on a host with load near 28: 6 files timed out, and none of them are the SDK proof files. All 15 of those files pass when run alone. CI runs the full suite. ## Risks - The bot step now runs one more repository script. If that script refuses (for example, an unreviewed committed lock), the refresh PR is not pushed. Before this change, the bot merged a refresh that made master red. - `lockSha256` now comes from the reviewed policy, not from the old registry value. The proof check is unchanged. It still needs the lock bytes to hash to `lockSha256`. - If #20618 merges first, this branch rebases cleanly, because the registry change is identical. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit) and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
131 lines
10 KiB
JavaScript
131 lines
10 KiB
JavaScript
import { strict as assert } from "node:assert";
|
|
import { execFileSync } from "node:child_process";
|
|
import { createHash } from "node:crypto";
|
|
import { mkdir, mkdtemp, readFile, realpath, rm, symlink, writeFile } from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { afterEach, test } from "vitest";
|
|
import { fileURLToPath } from "node:url";
|
|
import { staleSdkPublicInputs, synchronizeSdkPublicInputs } from "./source-sdk-public-inputs.mjs";
|
|
|
|
const registry = "tests/runner-e2e/source-sdk-public-inputs.json";
|
|
const reportLock = "tests/runner-e2e/report-lock.json";
|
|
const reportPolicy = (inputSha256, outputSha256) => `${JSON.stringify({ inputSha256, outputSha256, patchSha256: "0".repeat(64), basis: "synthetic" }, null, 2)}\n`;
|
|
const commit = (root, message) => { git(root, "add", "-A"); git(root, "-c", "user.name=Synthetic unit", "-c", "user.email=synthetic@example.invalid", "commit", "-qm", message); return git(root, "rev-parse", "HEAD").trim(); };
|
|
const roots = [];
|
|
const digest = bytes => createHash("sha256").update(bytes).digest("hex");
|
|
const blob = bytes => createHash("sha1").update(`blob ${Buffer.byteLength(bytes)}\0`).update(bytes).digest("hex");
|
|
const git = (root, ...args) => execFileSync("git", ["-c", "core.hooksPath=/dev/null", "-c", "commit.gpgsign=false", ...args], { cwd: root, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
|
|
async function put(root, file, bytes) { await mkdir(path.dirname(path.join(root, file)), { recursive: true }); await writeFile(path.join(root, file), bytes); }
|
|
async function fixture() {
|
|
const root = await realpath(await mkdtemp(path.join(os.tmpdir(), "sdk-input-sync-"))); roots.push(root);
|
|
const files = ["package.json", "packages/plugins/sdk/package.json", "packages/plugins/sdk/tsconfig.json",
|
|
"packages/shared/package.json", "packages/shared/tsconfig.json", "pnpm-lock.yaml", "scripts/ensure-plugin-build-deps.mjs",
|
|
"tsconfig.base.json", "tsconfig.json", "packages/shared/src/schema.ts", "packages/plugins/sdk/src/types.ts"];
|
|
for (const file of files) await put(root, file, `synthetic original ${file}\n`);
|
|
const baseline = { inputs: Object.fromEntries(files.map(file => [file, blob(`synthetic stale ${file}\n`)])),
|
|
lockSha256: digest("synthetic authenticated restored lock"), compilerPackageSha256: "a".repeat(64),
|
|
parserPackageSha256: "b".repeat(64), parserEntrySha256: "c".repeat(64),
|
|
zodPackageSha256: "d".repeat(64), zodEntrySha256: "e".repeat(64), zodTypeFiles: { "index.d.cts": "f".repeat(64) } };
|
|
await put(root, registry, `${JSON.stringify(baseline, null, 2)}\n`);
|
|
// The reviewed report-lock policy: committed lock in, authenticated restored lock out.
|
|
await put(root, reportLock, reportPolicy(digest("synthetic original pnpm-lock.yaml\n"), baseline.lockSha256));
|
|
git(root, "init", "-q"); git(root, "add", ".");
|
|
git(root, "-c", "user.name=Synthetic unit", "-c", "user.email=synthetic@example.invalid", "commit", "-qm", "synthetic source");
|
|
return { root, baseline, revision: git(root, "rev-parse", "HEAD").trim() };
|
|
}
|
|
afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); });
|
|
|
|
test("synchronizes the committed source graph, preserves every runtime pin and is idempotent", async () => {
|
|
const { root, baseline } = await fixture();
|
|
await put(root, "packages/shared/src/added.ts", "synthetic added source\n");
|
|
git(root, "add", "."); git(root, "-c", "user.name=Synthetic unit", "-c", "user.email=synthetic@example.invalid", "commit", "-qm", "additional source");
|
|
const revision = git(root, "rev-parse", "HEAD").trim();
|
|
assert.equal((await synchronizeSdkPublicInputs(root, revision)).changed, true);
|
|
assert.deepEqual(JSON.parse(await readFile(path.join(root, registry), "utf8")), baseline, "check does not write");
|
|
const changed = await synchronizeSdkPublicInputs(root, revision, { write: true });
|
|
const updated = JSON.parse(await readFile(path.join(root, registry), "utf8"));
|
|
assert.equal(updated.inputs["packages/shared/src/added.ts"], blob("synthetic added source\n"));
|
|
for (const [file, pin] of Object.entries(updated.inputs)) assert.equal(pin, git(root, "rev-parse", `${revision}:${file}`).trim());
|
|
const { inputs: oldInputs, ...runtime } = baseline, { inputs: newInputs, ...afterRuntime } = updated;
|
|
assert.deepEqual(afterRuntime, runtime); assert.equal(changed.runtimePinsChanged, false);
|
|
assert.equal(changed.registrySha256, digest(await readFile(path.join(root, registry))));
|
|
assert.equal((await synchronizeSdkPublicInputs(root, revision)).changed, false);
|
|
});
|
|
for (const mutation of ["dirty", "missing", "extra", "symlink", "registry", "revision"]) {
|
|
test(`refuses ${mutation} source custody without writing the registry`, async () => {
|
|
const { root, revision } = await fixture();
|
|
const target = path.join(root, "packages/shared/src/schema.ts");
|
|
if (mutation === "dirty") await writeFile(target, "uncommitted synthetic source");
|
|
if (mutation === "missing") await rm(target);
|
|
if (mutation === "extra") await put(root, "packages/shared/src/uncommitted.ts", "uncommitted");
|
|
if (mutation === "symlink") { await rm(target); await symlink("../package.json", target); }
|
|
if (mutation === "registry") { const current = JSON.parse(await readFile(path.join(root, registry), "utf8")); current.parserEntrySha256 = "0".repeat(64); await writeFile(path.join(root, registry), JSON.stringify(current)); }
|
|
const before = await readFile(path.join(root, registry));
|
|
await assert.rejects(synchronizeSdkPublicInputs(root, mutation === "revision" ? "0".repeat(40) : revision, { write: true }));
|
|
assert.deepEqual(await readFile(path.join(root, registry)), before);
|
|
});
|
|
}
|
|
test("admits only the separately pinned restored lock overlay without changing its identities", async () => {
|
|
const { root, baseline, revision } = await fixture();
|
|
await writeFile(path.join(root, "pnpm-lock.yaml"), "synthetic authenticated restored lock");
|
|
await synchronizeSdkPublicInputs(root, revision, { write: true });
|
|
const updated = JSON.parse(await readFile(path.join(root, registry), "utf8"));
|
|
assert.equal(updated.lockSha256, baseline.lockSha256);
|
|
assert.equal(updated.inputs["pnpm-lock.yaml"], git(root, "rev-parse", `${revision}:pnpm-lock.yaml`).trim());
|
|
await writeFile(path.join(root, "pnpm-lock.yaml"), "unknown lock overlay");
|
|
await assert.rejects(synchronizeSdkPublicInputs(root, revision));
|
|
});
|
|
|
|
// A lockfile refresh rewrites the lock and report-lock.json together (#20615).
|
|
// The sync carries the new reviewed output hash into lockSha256; nothing else.
|
|
test("a lockfile refresh moves lockSha256 to the committed reviewed report-lock output", async () => {
|
|
const { root, baseline, revision: first } = await fixture();
|
|
await synchronizeSdkPublicInputs(root, first, { write: true });
|
|
commit(root, "synced");
|
|
const refreshed = "synthetic refreshed pnpm-lock.yaml\n";
|
|
await put(root, "pnpm-lock.yaml", refreshed);
|
|
await put(root, reportLock, reportPolicy(baseline.lockSha256, digest(refreshed)));
|
|
const revision = commit(root, "lockfile refresh");
|
|
assert.deepEqual(await staleSdkPublicInputs(root), [
|
|
{ file: "pnpm-lock.yaml", change: "changed" },
|
|
{ file: `${reportLock} outputSha256`, change: "lockSha256 changed" },
|
|
]);
|
|
const result = await synchronizeSdkPublicInputs(root, revision, { write: true });
|
|
assert.equal(result.lockPinChanged, true); assert.equal(result.runtimePinsChanged, false);
|
|
const updated = JSON.parse(await readFile(path.join(root, registry), "utf8"));
|
|
assert.equal(updated.lockSha256, digest(refreshed));
|
|
assert.equal(updated.inputs["pnpm-lock.yaml"], blob(refreshed));
|
|
const { inputs: _old, lockSha256: _oldLock, ...runtime } = baseline, { inputs: _new, lockSha256: _newLock, ...afterRuntime } = updated;
|
|
assert.deepEqual(afterRuntime, runtime);
|
|
assert.deepEqual(await staleSdkPublicInputs(root), []);
|
|
});
|
|
test("refuses a committed lock that the report-lock policy does not review", async () => {
|
|
const { root } = await fixture();
|
|
await put(root, "pnpm-lock.yaml", "synthetic unreviewed lock\n");
|
|
const revision = commit(root, "unreviewed lock");
|
|
const before = await readFile(path.join(root, registry));
|
|
await assert.rejects(synchronizeSdkPublicInputs(root, revision, { write: true }), /not reviewed by tests\/runner-e2e\/report-lock\.json/);
|
|
assert.deepEqual(await readFile(path.join(root, registry)), before);
|
|
});
|
|
|
|
// The pins only prove the SDK public types while they match the committed
|
|
// source. A PR that edits packages/shared/src (or another pinned input) without
|
|
// refreshing them breaks every SDK-dependent Runner cell before its server
|
|
// starts, so fail here, cheaply, with the inputs and the fix command.
|
|
test("the committed SDK public source pins match this checkout's committed tree", async () => {
|
|
const root = fileURLToPath(new URL("../..", import.meta.url));
|
|
const stale = await staleSdkPublicInputs(root);
|
|
assert.deepEqual(stale, [], `${registry} is stale for ${stale.length} pinned input(s): ${stale.slice(0, 10).map(row => `${row.file} (${row.change})`).join(", ")}${stale.length > 10 ? ", ..." : ""}. `
|
|
+ "Commit your change, run `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --write`, and commit the updated registry.");
|
|
});
|
|
test("the drift guard names changed, unpinned and removed inputs and clears after a resync", async () => {
|
|
const { root, revision } = await fixture();
|
|
assert.deepEqual((await staleSdkPublicInputs(root)).map(row => row.change), Array(11).fill("changed"));
|
|
await synchronizeSdkPublicInputs(root, revision, { write: true });
|
|
assert.deepEqual(await staleSdkPublicInputs(root), []);
|
|
await put(root, "packages/shared/src/added.ts", "synthetic added source\n"); await rm(path.join(root, "packages/plugins/sdk/src/types.ts"));
|
|
git(root, "add", "-A"); git(root, "-c", "user.name=Synthetic unit", "-c", "user.email=synthetic@example.invalid", "commit", "-qm", "drift");
|
|
assert.deepEqual(await staleSdkPublicInputs(root), [{ file: "packages/plugins/sdk/src/types.ts", change: "removed" }, { file: "packages/shared/src/added.ts", change: "unpinned" }]);
|
|
});
|