mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 21:05:21 +02:00
Build isolated preview artifacts for exact-source deployments (#13041)
## Thinking Path > - Paperclip manages AI agents and their work. > - Managed deployments need a cloud image and a database migration package. > - Branch commits can lack both artifacts until a normal release runs. > - Operators need to test an exact commit without advancing release aliases. > - This pull request adds a preview build mode to the existing release workflow. > - Builds use an immutable source SHA and publish isolated, reusable artifacts. ## Linked Issues or Issue Description **Subsystem affected** Release automation, cloud Docker images, and shared/database npm packages. **Problem or motivation** An operator cannot deploy an unpublished branch with new migrations using only the normal release artifacts. Publishing it through a normal lane would also advance shared release aliases. **Proposed solution** Dispatch the trusted release workflow on master with a full source SHA and a request UUID. Build missing SHA images and, when needed, deterministic preview shared/DB packages. Publish packages under the preview dist-tag with exact workspace pins. Reuse matching artifacts on retries. **Roadmap alignment** This extends release tooling for operator validation. It does not add a core product feature or duplicate a planned product capability. Related PR searches found no duplicate preview deployment workflow. ## What Changed - Add the preview channel, request correlation, artifact checks, and result artifact. - Compile source packages in a separate job from the npm publisher. The publisher uses trusted master code and disables package lifecycle scripts. - Publish only SHA cloud image tags. Preserve release aliases. Use full-SHA tags and no shared build cache. - Verify full source identity for reused packages and images. Both image and npm publishers use isolated jobs and the externally master-restricted npm-canary environment. Fail on registry authentication errors, outages, or artifact identity mismatches. - Let bundled-package preparation use patches from the requested source checkout. - Document publishing configuration, artifact contracts, and deployment order. ## Verification - Passed `pnpm -r typecheck` and `pnpm build`. - Passed `pnpm test:release-registry`: 107 tests, including eight preview tests. - Passed `actionlint -shellcheck= .github/workflows/release.yml`. - Built real shared and DB preview tarballs from an isolated exact-SHA checkout. Verified package source identity and all 244 SQL files and journal entries. - Verified the full revision behind an existing published SHA cloud image. - `pnpm test:run` exposed missing local embedded PostgreSQL library symlinks. The package's postinstall repair restored initdb; all 12 previously affected suites passed on rerun (95 tests). Additional local matrix reruns are in progress. The complete PR CI matrix is green, including general/serialized tests, e2e, typecheck, build, release registry, canary dry run, and the required verify gate. - Live preview publication and staging deployment require this workflow on master and the compatible control-plane backend. They have not run yet. No production deployment was performed. ## Risks Preview npm versions are immutable public artifacts. Both packages must retain their trusted publisher for release.yml in environment npm-canary. Source builds must remain separated from privileged npm publishing. The deploying control plane must verify source identity, integrity, and migration compatibility before use. Normal release jobs retain their existing conditions. Roll back by stopping preview dispatches and reverting the workflow/tooling. Published preview versions remain isolated from normal release tags. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution, and test runs. The session does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the feature template - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [ ] I have run the full tests locally and they pass - [x] I have added tests for the new behavior - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
b97101893f
commit
0cc796b7bd
6 files changed
+652
-8
No files matched your search
@@ -116,8 +116,8 @@ export function selectBundledDependencyPatches(
|
||||
return selectedPatches;
|
||||
}
|
||||
|
||||
export function applyBundledDependencyPatches(destinationDir, bundledDependencies) {
|
||||
const rootPackage = JSON.parse(readFileSync(resolve(repoRoot, "package.json"), "utf8"));
|
||||
export function applyBundledDependencyPatches(destinationDir, bundledDependencies, sourceRoot = repoRoot) {
|
||||
const rootPackage = JSON.parse(readFileSync(resolve(sourceRoot, "package.json"), "utf8"));
|
||||
const patchedDependencies = rootPackage.pnpm?.patchedDependencies ?? {};
|
||||
|
||||
for (const { packageName, patchPath } of selectBundledDependencyPatches(
|
||||
@@ -129,14 +129,14 @@ export function applyBundledDependencyPatches(destinationDir, bundledDependencie
|
||||
"patch",
|
||||
["-p1", "--forward", "-d", resolve(destinationDir, "node_modules", packageName)],
|
||||
{
|
||||
input: readFileSync(resolve(repoRoot, patchPath)),
|
||||
input: readFileSync(resolve(sourceRoot, patchPath)),
|
||||
stdio: ["pipe", "inherit", "inherit"],
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export function prepareBundledPackage(sourceDir, destinationDir) {
|
||||
export function prepareBundledPackage(sourceDir, destinationDir, { sourceRoot = repoRoot } = {}) {
|
||||
const sourcePackagePath = resolve(sourceDir, "package.json");
|
||||
const sourcePackage = JSON.parse(readFileSync(sourcePackagePath, "utf8"));
|
||||
const bundledDependencies = sourcePackage.bundleDependencies ?? sourcePackage.bundledDependencies ?? [];
|
||||
@@ -166,7 +166,7 @@ export function prepareBundledPackage(sourceDir, destinationDir) {
|
||||
{ cwd: destinationDir, stdio: "inherit" },
|
||||
);
|
||||
writeFileSync(deployedPackagePath, `${JSON.stringify(publishManifest, null, 2)}\n`);
|
||||
applyBundledDependencyPatches(destinationDir, bundledDependencies);
|
||||
applyBundledDependencyPatches(destinationDir, bundledDependencies, sourceRoot);
|
||||
|
||||
if (bundledDependencies.includes("acpx")) {
|
||||
const acpxPackage = JSON.parse(
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
#!/usr/bin/env node
|
||||
// Trusted release tooling. Packaging runs without publish credentials; publishing
|
||||
// accepts only the two fixed package artifacts and never executes their scripts.
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { readFileSync, writeFileSync, mkdirSync, cpSync, renameSync, appendFileSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { gunzipSync } from "node:zlib";
|
||||
import { createHash } from "node:crypto";
|
||||
import { materializePublishManifest, prepareBundledPackage } from "./prepare-bundled-package.mjs";
|
||||
|
||||
export const versionFor = (sha) => {
|
||||
if (!/^[0-9a-f]{40}$/.test(sha ?? "")) throw new Error("Preview builds require a full immutable commit SHA.");
|
||||
return `0.0.0-preview.g${sha}`;
|
||||
};
|
||||
export function validateRequest(sha, requestId) {
|
||||
versionFor(sha);
|
||||
if (!/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/.test(requestId ?? "")) throw new Error("A correlation UUID is required.");
|
||||
}
|
||||
|
||||
export function previewManifest(pkg, sha) {
|
||||
if (!["@paperclipai/shared", "@paperclipai/db"].includes(pkg.name)) throw new Error("Unexpected preview package.");
|
||||
const version = versionFor(sha);
|
||||
const exact = structuredClone(pkg);
|
||||
for (const section of ["dependencies", "optionalDependencies", "peerDependencies"]) {
|
||||
for (const [name, specifier] of Object.entries(exact[section] ?? {})) {
|
||||
if (typeof specifier === "string" && specifier.startsWith("workspace:")) exact[section][name] = version;
|
||||
}
|
||||
}
|
||||
const result = materializePublishManifest({ ...exact, version });
|
||||
result.gitHead = sha;
|
||||
result.paperclipPreviewCommit = sha;
|
||||
if (pkg.name === "@paperclipai/db") result.dependencies = { ...result.dependencies, "@paperclipai/shared": version };
|
||||
return result;
|
||||
}
|
||||
|
||||
export function assertMetadata(pkg, name, sha) {
|
||||
if (pkg?.publishConfig !== undefined || pkg?.name !== name || pkg.version !== versionFor(sha) || pkg.gitHead !== sha || pkg.paperclipPreviewCommit !== sha ||
|
||||
(name === "@paperclipai/db" && pkg.dependencies?.["@paperclipai/shared"] !== versionFor(sha))) {
|
||||
throw new Error("Preview package identity or dependency pin mismatch.");
|
||||
}
|
||||
}
|
||||
|
||||
export function tarManifest(bytes) {
|
||||
const tar = gunzipSync(bytes, { maxOutputLength: 128 * 1024 * 1024 });
|
||||
let manifest;
|
||||
for (let offset = 0; offset + 512 <= tar.length;) {
|
||||
const h = tar.subarray(offset, offset + 512);
|
||||
if (h.every((v) => v === 0)) break;
|
||||
const field = (start, size) => h.subarray(start, start + size).toString("utf8").split("\0")[0].trim();
|
||||
const sizeText = field(124, 12);
|
||||
if (!/^[0-7]+$/.test(sizeText)) throw new Error("Invalid package archive.");
|
||||
const size = Number.parseInt(sizeText, 8);
|
||||
if (offset + 512 + size > tar.length) throw new Error("Truncated package archive.");
|
||||
const name = `${field(345, 155) ? field(345, 155) + "/" : ""}${field(0, 100)}`;
|
||||
if (!name.startsWith("package/") || name.split("/").some((part) => part === "." || part === "..") || ![0, 48, 53].includes(h[156])) throw new Error("Unsupported package archive entry.");
|
||||
if (name === "package/package.json") {
|
||||
if (manifest || ![0, 48].includes(h[156])) throw new Error("Invalid package manifest entry.");
|
||||
manifest = JSON.parse(tar.subarray(offset + 512, offset + 512 + size).toString("utf8"));
|
||||
}
|
||||
offset += 512 + Math.ceil(size / 512) * 512;
|
||||
}
|
||||
if (!manifest) throw new Error("Missing package manifest.");
|
||||
return manifest;
|
||||
}
|
||||
|
||||
export async function packageExists(name, sha, fetchImpl = fetch) {
|
||||
const response = await fetchImpl(`https://registry.npmjs.org/${encodeURIComponent(name)}/${versionFor(sha)}`, { signal: AbortSignal.timeout(30_000) });
|
||||
if (response.status === 404) return false;
|
||||
if (!response.ok) throw new Error(`npm lookup failed: HTTP ${response.status}`);
|
||||
const pkg = await response.json();
|
||||
assertMetadata(pkg, name, sha);
|
||||
if (!pkg.dist?.integrity || !pkg.dist?.tarball) throw new Error("Published preview has no immutable distribution pin.");
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function imageExists(sha, fetchImpl = fetch) {
|
||||
versionFor(sha);
|
||||
const tokenRes = await fetchImpl("https://ghcr.io/token?service=ghcr.io&scope=repository:paperclipai/paperclip:pull", { signal: AbortSignal.timeout(30_000) });
|
||||
if (!tokenRes.ok) throw new Error(`GHCR lookup failed: HTTP ${tokenRes.status}`);
|
||||
const { token } = await tokenRes.json();
|
||||
if (typeof token !== "string") throw new Error("GHCR did not return a pull token.");
|
||||
const base = "https://ghcr.io/v2/paperclipai/paperclip";
|
||||
const headers = { Authorization: `Bearer ${token}`, Accept: "application/vnd.oci.image.index.v1+json, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json" };
|
||||
const get = (url) => fetchImpl(url, { headers, redirect: "error", signal: AbortSignal.timeout(30_000) });
|
||||
let res = await get(`${base}/manifests/sha-${sha}-cloud`);
|
||||
if (res.status === 404) return false;
|
||||
if (!res.ok) throw new Error(`GHCR lookup failed: HTTP ${res.status}`);
|
||||
let manifest = await res.json();
|
||||
const digest = (value) => {
|
||||
if (typeof value !== "string" || !/^sha256:[0-9a-f]{64}$/.test(value)) throw new Error("Invalid image digest.");
|
||||
return value;
|
||||
};
|
||||
if (Array.isArray(manifest.manifests)) {
|
||||
const amd64 = manifest.manifests.find((entry) => entry.platform?.os === "linux" && entry.platform?.architecture === "amd64");
|
||||
if (!amd64) throw new Error("Cloud image has no Linux amd64 manifest.");
|
||||
res = await get(`${base}/manifests/${digest(amd64.digest)}`);
|
||||
if (!res.ok) throw new Error(`GHCR manifest lookup failed: HTTP ${res.status}`);
|
||||
manifest = await res.json();
|
||||
}
|
||||
res = await fetchImpl(`${base}/blobs/${digest(manifest.config?.digest)}`, { headers, redirect: "manual", signal: AbortSignal.timeout(30_000) });
|
||||
// Registry blob storage may redirect to its signed storage URL. Follow only
|
||||
// with no Authorization header, so the GHCR token cannot leave the registry.
|
||||
if ([301, 302, 307, 308].includes(res.status)) {
|
||||
const location = new URL(res.headers.get("location"));
|
||||
if (location.protocol !== "https:" || location.username || location.password) throw new Error("Invalid registry blob redirect.");
|
||||
res = await fetchImpl(location.href, { redirect: "error", signal: AbortSignal.timeout(30_000) });
|
||||
}
|
||||
if (!res.ok) throw new Error(`GHCR config lookup failed: HTTP ${res.status}`);
|
||||
const config = await res.json();
|
||||
if (config.config?.Labels?.["org.opencontainers.image.revision"] !== sha) throw new Error("Existing SHA image tag does not match the requested full commit.");
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Publication loads image data, but never runs a container or source scripts. */
|
||||
export async function publishImage(file, sha, { exec = execFileSync, fetchImpl = fetch } = {}) {
|
||||
versionFor(sha);
|
||||
const image = `ghcr.io/paperclipai/paperclip:sha-${sha}-cloud`;
|
||||
if (await imageExists(sha, fetchImpl)) { console.log("Reusing the verified SHA cloud image."); return; }
|
||||
exec("docker", ["load", "--input", path.resolve(file)], { encoding: "utf8", maxBuffer: 8 * 1024 * 1024 });
|
||||
const [metadata] = JSON.parse(exec("docker", ["image", "inspect", image], { encoding: "utf8", maxBuffer: 8 * 1024 * 1024 }));
|
||||
if (metadata?.Config?.Labels?.["org.opencontainers.image.revision"] !== sha || metadata.Os !== "linux" || metadata.Architecture !== "amd64" ||
|
||||
!/^sha256:[0-9a-f]{64}$/.test(metadata.Id ?? "")) throw new Error("Built image identity or platform does not match the request.");
|
||||
// Push only this verified image ID under the one permitted tag, regardless
|
||||
// of any additional tag names present in the untrusted Docker archive.
|
||||
exec("docker", ["tag", metadata.Id, image], { stdio: "inherit" });
|
||||
exec("docker", ["push", image], { stdio: "inherit" });
|
||||
}
|
||||
|
||||
export function packPreview(source, output, sha, { exec = execFileSync } = {}) {
|
||||
versionFor(sha);
|
||||
source = path.resolve(source); output = path.resolve(output);
|
||||
if (exec("git", ["rev-parse", "HEAD"], { cwd: source, encoding: "utf8" }).trim() !== sha) throw new Error("Source checkout differs from the requested commit.");
|
||||
mkdirSync(output, { recursive: true });
|
||||
for (const short of ["shared", "db"]) {
|
||||
exec("pnpm", ["--filter", `@paperclipai/${short}`, "build"], { cwd: source, stdio: "inherit" });
|
||||
const packageDir = path.join(source, "packages", short);
|
||||
const originalText = readFileSync(path.join(packageDir, "package.json"), "utf8");
|
||||
const original = JSON.parse(originalText);
|
||||
const pkg = previewManifest(original, sha);
|
||||
const staging = path.join(output, `package-${short}`);
|
||||
if ((pkg.bundleDependencies ?? []).length) {
|
||||
// The established helper materializes patched embedded-postgres instead
|
||||
// of publishing pnpm's dependency symlinks.
|
||||
writeFileSync(path.join(packageDir, "package.json"), JSON.stringify(pkg));
|
||||
try { prepareBundledPackage(packageDir, staging, { sourceRoot: source }); }
|
||||
finally { writeFileSync(path.join(packageDir, "package.json"), originalText); }
|
||||
} else {
|
||||
mkdirSync(staging, { recursive: true });
|
||||
cpSync(path.join(packageDir, "dist"), path.join(staging, "dist"), { recursive: true });
|
||||
writeFileSync(path.join(staging, "package.json"), JSON.stringify(pkg));
|
||||
}
|
||||
const packed = JSON.parse(exec("npx", ["--yes", "npm@10.9.7", "pack", "--ignore-scripts", "--json", "--pack-destination", output], { cwd: staging, encoding: "utf8", maxBuffer: 8 * 1024 * 1024 }));
|
||||
renameSync(path.join(output, path.basename(packed[0].filename)), path.join(output, `${short}.tgz`));
|
||||
assertMetadata(tarManifest(readFileSync(path.join(output, `${short}.tgz`))), `@paperclipai/${short}`, sha);
|
||||
}
|
||||
}
|
||||
|
||||
export async function publishPreview(dir, sha, { fetchImpl = fetch, exec = execFileSync, sleep = (ms) => new Promise((r) => setTimeout(r, ms)) } = {}) {
|
||||
for (const short of ["shared", "db"]) {
|
||||
const name = `@paperclipai/${short}`;
|
||||
const file = path.resolve(dir, `${short}.tgz`);
|
||||
const bytes = readFileSync(file);
|
||||
assertMetadata(tarManifest(bytes), name, sha);
|
||||
if (await packageExists(name, sha, fetchImpl)) { console.log(`Reusing ${name}@${versionFor(sha)}`); continue; }
|
||||
console.log(`Publishing ${name}@${versionFor(sha)} (${createHash("sha256").update(bytes).digest("hex").slice(0, 12)})`);
|
||||
// No package checkout, lifecycle scripts, npmrc, or branch code runs here.
|
||||
exec("npm", ["publish", file, "--tag", "preview", "--access", "public", "--ignore-scripts", "--provenance", "--registry", "https://registry.npmjs.org"], { stdio: "inherit" });
|
||||
let published = false;
|
||||
for (let attempt = 0; attempt < 60; attempt++) {
|
||||
if (await packageExists(name, sha, fetchImpl)) { published = true; break; }
|
||||
await sleep(10_000);
|
||||
}
|
||||
if (!published) throw new Error("npm accepted the preview but it is not yet visible. Retry reuses published packages.");
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
const [command, ...args] = process.argv.slice(2);
|
||||
try {
|
||||
if (command === "plan") {
|
||||
const [sha, requestId, migrator] = args;
|
||||
validateRequest(sha, requestId);
|
||||
if (process.env.GITHUB_REF !== "refs/heads/master") throw new Error("Preview workflow definitions must run from master.");
|
||||
const image = !await imageExists(sha);
|
||||
const packages = migrator === "true" && !(await packageExists("@paperclipai/shared", sha) && await packageExists("@paperclipai/db", sha));
|
||||
appendFileSync(process.env.GITHUB_OUTPUT, `image=${image}\npackages=${packages}\n`);
|
||||
} else if (command === "pack") packPreview(...args);
|
||||
else if (command === "publish") await publishPreview(...args);
|
||||
else if (command === "publish-image") await publishImage(...args);
|
||||
else if (command === "result") {
|
||||
const [sha, requestId] = args;
|
||||
validateRequest(sha, requestId);
|
||||
if (!await imageExists(sha)) throw new Error("Cloud image is still missing.");
|
||||
if (process.env.PREVIEW_MIGRATOR === "true" && !(await packageExists("@paperclipai/shared", sha) && await packageExists("@paperclipai/db", sha))) throw new Error("Preview packages are still missing.");
|
||||
mkdirSync("stack-deploy-result", { recursive: true });
|
||||
writeFileSync("stack-deploy-result/result.json", JSON.stringify({ version: 1, stage: "build", requestId, sha, status: "ready" }) + "\n");
|
||||
} else throw new Error("Expected plan, pack, publish, publish-image, or result.");
|
||||
} catch (error) { console.error(error.message); process.exitCode = 1; }
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync, mkdtempSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { gzipSync } from "node:zlib";
|
||||
import { previewManifest, assertMetadata, validateRequest, versionFor, tarManifest, packageExists, imageExists, publishPreview, publishImage } from "./preview-artifacts.mjs";
|
||||
|
||||
const sha = "a".repeat(40);
|
||||
const id = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa";
|
||||
const manifest = (name) => previewManifest({ name, version: "0.0.0", dependencies: name.endsWith("/db") ? { "@paperclipai/shared": "workspace:*" } : {}, publishConfig: { exports: { ".": "./dist/index.js" } } }, sha);
|
||||
function pack(pkg) {
|
||||
const b = Buffer.from(JSON.stringify(pkg)); const h = Buffer.alloc(512);
|
||||
h.write("package/package.json"); h.write(b.length.toString(8).padStart(11, "0"), 124, 11); h[156] = 48;
|
||||
const padded = Buffer.alloc(Math.ceil(b.length / 512) * 512); b.copy(padded);
|
||||
return gzipSync(Buffer.concat([h, padded, Buffer.alloc(1024)]));
|
||||
}
|
||||
const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
|
||||
|
||||
test("preview request requires immutable SHA and correlation UUID", () => {
|
||||
validateRequest(sha, id);
|
||||
for (const ref of ["master", "origin/master", "a".repeat(7), "$(unsafe)", "A".repeat(40)]) assert.throws(() => versionFor(ref));
|
||||
assert.throws(() => validateRequest(sha, "not-a-request"));
|
||||
});
|
||||
|
||||
test("preview manifests carry exact source, isolated versions and shared dependency", () => {
|
||||
const pkg = manifest("@paperclipai/db");
|
||||
assert.equal(pkg.version, `0.0.0-preview.g${sha}`);
|
||||
assert.equal(pkg.dependencies["@paperclipai/shared"], pkg.version);
|
||||
assert.deepEqual(pkg.exports, { ".": "./dist/index.js" });
|
||||
assertMetadata(pkg, "@paperclipai/db", sha);
|
||||
assert.throws(() => assertMetadata({ ...pkg, gitHead: "b".repeat(40) }, pkg.name, sha));
|
||||
assert.throws(() => assertMetadata({ ...pkg, dependencies: { "@paperclipai/shared": "latest" } }, pkg.name, sha));
|
||||
assert.deepEqual(tarManifest(pack(pkg)), pkg);
|
||||
});
|
||||
|
||||
test("only 404 means an artifact is missing; auth and outages are fatal", async () => {
|
||||
assert.equal(await packageExists("@paperclipai/db", sha, async () => json({}, 404)), false);
|
||||
await assert.rejects(packageExists("@paperclipai/db", sha, async () => json({}, 403)));
|
||||
await assert.rejects(packageExists("@paperclipai/db", sha, async () => json({}, 503)));
|
||||
await assert.rejects(imageExists(sha, async () => json({}, 503)));
|
||||
assert.equal(await imageExists(sha, async (url) => url.includes("/token?") ? json({ token: "test-pull-token" }) : json({}, 404)), false);
|
||||
await assert.rejects(packageExists("@paperclipai/db", sha, async () => json({ ...manifest("@paperclipai/db"), gitHead: "b".repeat(40) })));
|
||||
});
|
||||
|
||||
test("publishing reuses existing previews and never executes package lifecycle hooks", async () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), "preview-publish-test-"));
|
||||
const published = new Set(["@paperclipai/shared"]);
|
||||
const calls = [];
|
||||
try {
|
||||
for (const short of ["shared", "db"]) writeFileSync(path.join(dir, `${short}.tgz`), pack({ ...manifest(`@paperclipai/${short}`), scripts: { prepublishOnly: "do-not-run" } }));
|
||||
await publishPreview(dir, sha, {
|
||||
fetchImpl: async (url) => {
|
||||
const name = decodeURIComponent(new URL(url).pathname.split("/")[1]);
|
||||
return published.has(name) ? json({ ...manifest(name), dist: { integrity: "test-integrity", tarball: "https://registry.npmjs.org/package.tgz" } }) : json({}, 404);
|
||||
},
|
||||
exec: (command, args) => { calls.push({ command, args }); published.add("@paperclipai/db"); },
|
||||
sleep: async () => {},
|
||||
});
|
||||
assert.equal(calls.length, 1);
|
||||
assert.equal(calls[0].command, "npm");
|
||||
assert.ok(calls[0].args.includes("--ignore-scripts"));
|
||||
assert.equal(calls[0].args[calls[0].args.indexOf("--tag") + 1], "preview");
|
||||
assert.ok(!calls[0].args.includes("canary"));
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
test("preview workflow separates branch compilation from trusted publishing", () => {
|
||||
const workflow = readFileSync(new URL("../.github/workflows/release.yml", import.meta.url), "utf8");
|
||||
const builder = workflow.split(" package_preview:")[1].split(" publish_preview:")[0];
|
||||
const publisher = workflow.split(" publish_preview:")[1].split(" image_preview:")[0];
|
||||
const image = workflow.split(" image_preview:")[1].split(" publish_image_preview:")[0];
|
||||
const imagePublisher = workflow.split(" publish_image_preview:")[1].split(" result_preview:")[0];
|
||||
assert.doesNotMatch(builder, /id-token: write|packages: write|secrets\./);
|
||||
assert.doesNotMatch(publisher, /ref: \$\{\{ inputs.source_ref|working-directory: source|pnpm install/);
|
||||
assert.match(publisher, /environment: npm-canary/);
|
||||
assert.match(image, /PAPERCLIP_BUILD_COMMIT=\$\{\{ inputs.source_ref \}\}/);
|
||||
assert.doesNotMatch(image, /cache-(?:to|from):|canary-cloud|latest-cloud|packages: write|secrets\./);
|
||||
assert.doesNotMatch(imagePublisher, /ref: \$\{\{ inputs.source_ref|docker\/build-push-action|pnpm install/);
|
||||
assert.match(imagePublisher, /publish-image/);
|
||||
assert.match(imagePublisher, /environment: npm-canary/);
|
||||
assert.match(imagePublisher, /github.ref == 'refs\/heads\/master'/);
|
||||
assert.match(publisher, /github.ref == 'refs\/heads\/master'/);
|
||||
assert.doesNotMatch(workflow.split(" verify_canary:")[0], /uses: [^\n]+@v\d/);
|
||||
assert.match(workflow, /Stack deploy \{0\} build/);
|
||||
});
|
||||
|
||||
|
||||
test("existing image reuse verifies the full revision behind the immutable tag", async () => {
|
||||
const digest = "sha256:" + "b".repeat(64);
|
||||
for (const revision of [sha, "c".repeat(40)]) {
|
||||
const fetchImpl = async (url) => url.includes("/token?") ? json({ token: "test-pull-token" }) :
|
||||
url.includes("/blobs/") ? json({ config: { Labels: { "org.opencontainers.image.revision": revision } } }) :
|
||||
url.endsWith(digest) ? json({ config: { digest } }) : json({ manifests: [{ digest, platform: { os: "linux", architecture: "amd64" } }] });
|
||||
if (revision === sha) assert.equal(await imageExists(sha, fetchImpl), true);
|
||||
else await assert.rejects(imageExists(sha, fetchImpl), /full commit/);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
test("image publisher verifies source and platform before pushing exactly one immutable tag", async () => {
|
||||
for (const revision of [sha, "c".repeat(40)]) {
|
||||
const calls = [];
|
||||
const operation = publishImage("preview-image.tar", sha, {
|
||||
fetchImpl: async (url) => url.includes("/token?") ? json({ token: "test-pull-token" }) : json({}, 404),
|
||||
exec: (command, args) => {
|
||||
calls.push({ command, args });
|
||||
if (args[0] === "image") return JSON.stringify([{ Id: "sha256:" + "b".repeat(64), Os: "linux", Architecture: "amd64", Config: { Labels: { "org.opencontainers.image.revision": revision } } }]);
|
||||
return "";
|
||||
},
|
||||
});
|
||||
if (revision === sha) {
|
||||
await operation;
|
||||
assert.deepEqual(calls.filter((call) => call.args[0] === "push").map((call) => call.args), [["push", `ghcr.io/paperclipai/paperclip:sha-${sha}-cloud`]]);
|
||||
} else { await assert.rejects(operation, /identity/); assert.ok(!calls.some((call) => call.args[0] === "push")); }
|
||||
assert.ok(!calls.some((call) => ["run", "build"].includes(call.args[0])));
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
test("commits sharing a short prefix use separate full-SHA image addresses", async () => {
|
||||
const urls = [];
|
||||
const fetchImpl = async (url) => { urls.push(url); return url.includes("/token?") ? json({ token: "test-pull-token" }) : json({}, 404); };
|
||||
const other = sha.slice(0, 7) + "b".repeat(33);
|
||||
await imageExists(sha, fetchImpl);
|
||||
await imageExists(other, fetchImpl);
|
||||
assert.deepEqual(urls.filter((url) => url.includes("/manifests/")), [sha, other].map((commit) => `https://ghcr.io/v2/paperclipai/paperclip/manifests/sha-${commit}-cloud`));
|
||||
});
|
||||
Reference in new issue
Block a user