mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 23:43:16 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Release workflows publish npm packages and multi-platform images. > - A master push can finish its image before the npm canary dispatch starts. > - The dispatch currently builds that same source again on two runners. > - This PR verifies and reuses the signed master image by digest. > - Missing images retain the native build path. Channel promotion still follows npm. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Docker workflow dispatched after npm canary publication. **Current behavior** Every canary dispatch builds both architectures, even when master has already published the same source. This consumes runner capacity and delays the image channel update. **Proposed behavior** Resolve the full-SHA master image. Verify its platform index, digest and signed source identity. Retag that immutable digest. Build normally when the image does not exist. **Reason and benefit** Remove duplicate image builds while keeping missing-image recovery and the current npm channel check. A recent duplicate used about 25 runner-minutes. The saving depends on whether the master image exists. ## What Changed - Add a canary-only reuse job before the native matrix. - Require signed provenance from the canonical Docker workflow on master at the exact source and signer commit. Read the proof from GHCR. - Reuse the validated digest or build both platforms when the full-SHA image is absent. Stop on registry and verification errors. - Keep the serialized npm dist-tag promotion after either path. Keep other build lanes independent. - Add failure tests, register them in CI and update release documentation. ## Verification - Passed 603 tests: `node --test '.github/scripts/tests/*.test.mjs' scripts/standard-image-contract.test.mjs scripts/reuse-canary-image.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/cloud-source-verification.test.mjs`. - Passed `actionlint -shellcheck='' .github/workflows/docker.yml .github/workflows/pr-trusted.yml` and `git diff --check`. - Full actionlint reports the same six existing shellcheck findings as the unchanged master workflow. No new findings. - A published master image passed the exact provenance command. This was a read-only check. No images were retagged. - Gitleaks and manual diff inspection found no secrets or private references. - Attempted `pnpm -r typecheck`, `pnpm test:run` and `pnpm build`. They cannot start because this isolated checkout has no installed dependencies (`tsc` and `tsx` are absent). Repository CI passed on `3c46f0152d4db94a7f393bc79f466aa6c2c745e2` after the documented browser-test rerun. All checks are complete. The original failure and prior baseline occurrence remain recorded in the PR comments; its root cause remains unknown. ## Risks - The image can appear just before its attestation. Verification then fails closed. Rerun after the master producer completes. - Reuse preserves the first build's resolved dependencies and tool versions. It intentionally does not rebuild the same source to refresh floating inputs. - A skipped or unfinished master image still needs a full build. No speedup is claimed for that path. - End-to-end reuse and runner savings require observation after merge. No live publication was performed. ## Model Used OpenAI `gpt-6.1-sol`, through the Codex harness in T3 Code, with xhigh reasoning effort and shell/code tools. The context window size is not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
94 lines
5.0 KiB
JavaScript
94 lines
5.0 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { createHash } from "node:crypto";
|
|
import test from "node:test";
|
|
import { standardImageDigest, resolveStandardImageDigest } from "./standard-image-contract.mjs";
|
|
import { isNode24ImageTag } from "./node-image-policy.mjs";
|
|
|
|
test("Node image policy accepts Node 24 major tags and exact security pins", () => {
|
|
for (const tag of ["24-trixie-slim", "24-bookworm-slim", "24-alpine3.23", "24.11.0-bookworm", "24.21.0-bookworm",
|
|
`24.21.0-bookworm@sha256:${"a".repeat(64)}`, `24-bookworm@sha256:${"0".repeat(64)}`]) {
|
|
assert.equal(isNode24ImageTag(tag), true, tag);
|
|
}
|
|
});
|
|
|
|
test("Node image policy rejects other majors, partial versions and malformed pins", () => {
|
|
for (const tag of ["240-bookworm", "25.21.0-bookworm", "v24.21.0-bookworm", "latest",
|
|
"24.0.0-bookworm", "24.10.0-bookworm", "24.10.99-bookworm",
|
|
"24.21-bookworm", "24.21.0", "24.21.0-", "24..21.0-bookworm", "24.021.0-bookworm",
|
|
"24.21.0-bookworm@sha256:short", `24.21.0-bookworm@@sha256:${"a".repeat(64)}`,
|
|
`24.21.0-bookworm@sha256:${"a".repeat(64)}junk`, "${NODE_VERSION}-bookworm", "24-$(whoami)"]) {
|
|
assert.equal(isNode24ImageTag(tag), false, tag);
|
|
}
|
|
});
|
|
const descriptor = architecture => ({ platform: { os: "linux", architecture }, digest: `sha256:${"a".repeat(64)}`, size: 100 });
|
|
const index = manifests => Buffer.from(JSON.stringify({ schemaVersion: 2, mediaType: "application/vnd.oci.image.index.v1+json", manifests }));
|
|
test("signs the exact registry bytes, including both supported platforms", () => {
|
|
const bytes = index([descriptor("amd64"), descriptor("arm64")]);
|
|
assert.equal(standardImageDigest(bytes), `sha256:${createHash("sha256").update(bytes).digest("hex")}`);
|
|
});
|
|
test("rejects incomplete, ambiguous, or malformed platform manifests", () => {
|
|
for (const manifests of [[descriptor("amd64")], [descriptor("amd64"), descriptor("amd64"), descriptor("arm64")],
|
|
[{ ...descriptor("amd64"), digest: "mutable-tag" }, descriptor("arm64")],
|
|
[{ ...descriptor("amd64"), size: -1 }, descriptor("arm64")]]) assert.throws(() => standardImageDigest(index(manifests)));
|
|
});
|
|
|
|
|
|
test("lookup hashes registry bytes and rejects formatting or header mismatches", async () => {
|
|
const raw = index([descriptor("amd64"), descriptor("arm64")]);
|
|
const expected = standardImageDigest(raw);
|
|
let appendNewline = false;
|
|
const fetchImpl = async (url, options) => {
|
|
assert.equal(options.redirect, "error");
|
|
if (url.includes("/token?")) return Response.json({ token: "public-fixture" });
|
|
assert.ok(url.endsWith(`sha-${"a".repeat(40)}`));
|
|
assert.equal(options.headers.authorization, "Bearer public-fixture");
|
|
return new Response(appendNewline ? Buffer.concat([raw, Buffer.from("\n")]) : raw, { headers: { "docker-content-digest": expected } });
|
|
};
|
|
assert.equal(await resolveStandardImageDigest("a".repeat(40), fetchImpl), expected);
|
|
appendNewline = true;
|
|
await assert.rejects(resolveStandardImageDigest("a".repeat(40), fetchImpl), /Registry digest/);
|
|
});
|
|
|
|
|
|
test("bounded retries cover token failures, propagation lag and rate limits", async () => {
|
|
const raw = index([descriptor("amd64"), descriptor("arm64")]);
|
|
const expected = standardImageDigest(raw);
|
|
const waits = [];
|
|
let tokens = 0, manifests = 0;
|
|
const fetchImpl = async url => {
|
|
if (url.includes("/token?")) {
|
|
if (++tokens === 1) throw new Error("transient connection reset");
|
|
if (tokens === 2) return new Response(null, { status: 503 });
|
|
return Response.json({ token: "public-fixture" });
|
|
}
|
|
if (++manifests === 1) return new Response(null, { status: 404 });
|
|
if (manifests === 2) return new Response(null, { status: 429, headers: { "retry-after": "120" } });
|
|
return new Response(raw, { headers: { "docker-content-digest": expected } });
|
|
};
|
|
assert.equal(await resolveStandardImageDigest("a".repeat(40), fetchImpl, { sleep: async ms => { waits.push(ms); } }), expected);
|
|
assert.deepEqual(waits, [1000, 2000, 1000, 30000]);
|
|
assert.equal(tokens, 3); assert.equal(manifests, 3);
|
|
});
|
|
|
|
test("permanent registry errors fail immediately and transient errors exhaust a finite budget", async () => {
|
|
for (const [status, expectedCalls] of [[401, 1], [403, 1], [503, 5]]) {
|
|
let calls = 0;
|
|
await assert.rejects(resolveStandardImageDigest("a".repeat(40), async () => { calls++; return new Response(null, { status }); }, { sleep: async () => {} }), /Registry lookup failed/);
|
|
assert.equal(calls, expectedCalls);
|
|
}
|
|
});
|
|
|
|
test("optional lookup falls back only for a missing manifest", async () => {
|
|
for (const status of [401, 403, 404, 503]) {
|
|
let calls = 0;
|
|
const lookup = resolveStandardImageDigest("a".repeat(40), async url => {
|
|
if (url.includes("/token?")) return Response.json({ token: "public-fixture" });
|
|
calls++;
|
|
return new Response(null, { status });
|
|
}, { missing: "return-null", sleep: async () => {} });
|
|
if (status === 404) assert.equal(await lookup, null);
|
|
else await assert.rejects(lookup, /Registry lookup failed/);
|
|
assert.equal(calls, status === 503 ? 5 : 1);
|
|
}
|
|
});
|