Files
PaperClipAI/scripts/standard-image-contract.mjs
T
Devin FoleyandPaperclip 105a67b52c Reuse signed master images for npm canaries (#18404)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Release workflows publish npm packages and multi-platform images.
> - A master push can finish its image before the npm canary dispatch
starts.
> - The dispatch currently builds that same source again on two runners.
> - This PR verifies and reuses the signed master image by digest.
> - Missing images retain the native build path. Channel promotion still
follows npm.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The Docker workflow dispatched after npm canary publication.

**Current behavior**

Every canary dispatch builds both architectures, even when master has
already published the same source. This consumes runner capacity and
delays the image channel update.

**Proposed behavior**

Resolve the full-SHA master image. Verify its platform index, digest and
signed source identity. Retag that immutable digest. Build normally when
the image does not exist.

**Reason and benefit**

Remove duplicate image builds while keeping missing-image recovery and
the current npm channel check. A recent duplicate used about 25
runner-minutes. The saving depends on whether the master image exists.

## What Changed

- Add a canary-only reuse job before the native matrix.
- Require signed provenance from the canonical Docker workflow on master
at the exact source and signer commit. Read the proof from GHCR.
- Reuse the validated digest or build both platforms when the full-SHA
image is absent. Stop on registry and verification errors.
- Keep the serialized npm dist-tag promotion after either path. Keep
other build lanes independent.
- Add failure tests, register them in CI and update release
documentation.

## Verification

- Passed 603 tests: `node --test '.github/scripts/tests/*.test.mjs'
scripts/standard-image-contract.test.mjs
scripts/reuse-canary-image.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs
scripts/cloud-source-verification.test.mjs`.
- Passed `actionlint -shellcheck='' .github/workflows/docker.yml
.github/workflows/pr-trusted.yml` and `git diff --check`.
- Full actionlint reports the same six existing shellcheck findings as
the unchanged master workflow. No new findings.
- A published master image passed the exact provenance command. This was
a read-only check. No images were retagged.
- Gitleaks and manual diff inspection found no secrets or private
references.
- Attempted `pnpm -r typecheck`, `pnpm test:run` and `pnpm build`. They
cannot start because this isolated checkout has no installed
dependencies (`tsc` and `tsx` are absent). Repository CI passed on
`3c46f0152d4db94a7f393bc79f466aa6c2c745e2` after the documented
browser-test rerun. All checks are complete. The original failure and
prior baseline occurrence remain recorded in the PR comments; its root
cause remains unknown.

## Risks

- The image can appear just before its attestation. Verification then
fails closed. Rerun after the master producer completes.
- Reuse preserves the first build's resolved dependencies and tool
versions. It intentionally does not rebuild the same source to refresh
floating inputs.
- A skipped or unfinished master image still needs a full build. No
speedup is claimed for that path.
- End-to-end reuse and runner savings require observation after merge.
No live publication was performed.

## Model Used

OpenAI `gpt-6.1-sol`, through the Codex harness in T3 Code, with xhigh
reasoning effort and shell/code tools. The context window size is not
exposed by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 16:54:04 -07:00

88 lines
4.4 KiB
JavaScript

import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { appendFile, readFile } from "node:fs/promises";
import { pathToFileURL } from "node:url";
/** Validate the multi-platform index before it becomes a signed build contract. */
export function standardImageDigest(bytes) {
const index = JSON.parse(bytes.toString("utf8"));
assert.equal(index.schemaVersion, 2);
assert.ok(["application/vnd.oci.image.index.v1+json", "application/vnd.docker.distribution.manifest.list.v2+json"].includes(index.mediaType));
assert.ok(Array.isArray(index.manifests));
for (const architecture of ["amd64", "arm64"]) {
const matches = index.manifests.filter(item => item.platform?.os === "linux" && item.platform?.architecture === architecture);
assert.equal(matches.length, 1, `Expected exactly one linux/${architecture} image`);
assert.match(matches[0].digest, /^sha256:[a-f0-9]{64}$/);
assert.ok(Number.isSafeInteger(matches[0].size) && matches[0].size > 0);
}
return `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
}
/** Read registry bytes directly; CLI stdout formatting is not artifact content. */
export async function resolveStandardImageDigest(sha, fetchImpl = fetch, {
sleep = ms => new Promise(resolve => setTimeout(resolve, ms)),
missing = "error",
} = {}) {
assert.match(sha ?? "", /^[a-f0-9]{40}$/);
const request = async (url, headers, consume, allowNotFound = false) => {
for (let attempt = 1; attempt <= 5; attempt++) {
let response;
try {
response = await fetchImpl(url, { headers, redirect: "error", signal: AbortSignal.timeout(30_000) });
} catch (error) {
if (attempt === 5) throw error;
await sleep(attempt * 1000);
continue;
}
if (!response.ok) {
await response.body?.cancel();
if (allowNotFound && response.status === 404 && missing === "return-null") return null;
const transient = [408, 425, 429].includes(response.status) || response.status >= 500 || (allowNotFound && response.status === 404);
assert.ok(transient && attempt < 5, `Registry lookup failed: HTTP ${response.status}`);
const retryAfter = Number(response.headers.get("retry-after"));
await sleep(Math.min(30_000, Math.max(attempt * 1000, Number.isFinite(retryAfter) ? retryAfter * 1000 : 0)));
continue;
}
try { return await consume(response); }
catch (error) {
// Interrupted response bodies are transient too. The size bound is an
// integrity limit and fails immediately, as does validation below.
if (error instanceof RangeError || attempt === 5) throw error;
await sleep(attempt * 1000);
}
}
throw new Error("Registry retry budget exhausted");
};
const { token } = await request("https://ghcr.io/token?service=ghcr.io&scope=repository:paperclipai/paperclip:pull", {}, response => response.json());
assert.ok(typeof token === "string" && token, "Missing public pull token");
const manifest = await request(`https://ghcr.io/v2/paperclipai/paperclip/manifests/sha-${sha}`, {
authorization: `Bearer ${token}`, accept: "application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json",
}, async response => {
assert.ok(response.body, "Standard image index has no body");
const reader = response.body.getReader();
const chunks = [];
let length = 0;
try {
while (true) {
const next = await reader.read();
if (next.done) break;
length += next.value.length;
if (length > 1024 * 1024) throw new RangeError("Standard index exceeds its size limit");
chunks.push(next.value);
}
} finally { await reader.cancel(); }
return { raw: Buffer.concat(chunks), expected: response.headers.get("docker-content-digest") };
}, true);
if (manifest === null) return null;
const { raw, expected } = manifest;
const digest = standardImageDigest(raw);
assert.equal(digest, expected, "Registry digest does not match the index bytes");
return digest;
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
const digest = process.argv[2] === "--resolve" ? await resolveStandardImageDigest(process.argv[3]) : standardImageDigest(await readFile(process.argv[2]));
if (process.env.GITHUB_OUTPUT) await appendFile(process.env.GITHUB_OUTPUT, `digest=${digest}\n`);
console.log(digest);
}