mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
ci: retire recurring public cloud image builds (#13827)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Core publishes standard images and source verification for downstream services. > - Managed services can now compose private images from the signed standard image. > - Core still builds a second public cloud image on every master push and release. > - That duplicate producer consumes build capacity and retains an obsolete readiness contract. > - This pull request retires recurring cloud publication while preserving the standard producer and rollback artifacts. ## Linked Issues or Issue Description Refs #13797 and #13789. Related: #12856 changes image dependency packaging; it does not retire this producer. **What existing behavior does this improve?** Core's recurring Docker publication and Cloud readiness workflow. **Current behavior** Master pushes call the legacy cloud publisher from Cloud readiness. Release tags and manual Docker runs call it too. Canary promotion also requires the legacy image. **Proposed behavior** Publish standard Core images and retain `Cloud source verified v1`. Let downstream services build their managed image. Keep explicit commit previews and existing images available. ## What Changed - Remove `docker-cloud.yml`, its master and release callers, and its unused cache selector. - Remove the legacy image/migrator wait and `Cloud deployable v1` job. Keep the full source verification workflow and exact source-proof name. - Make canary promotion inspect and promote the standard image only. - Preserve signed standard-image publication, direct migrator publication, and explicit `release.yml` previews. The preview path still uses the Dockerfile `cloud` target. - Update workflow, preview, build-stamp, and packaging tests. Exercise the promotion shell with mocked registry commands, including missing-image and missing-tag cases. - Document frozen legacy aliases, consumer requirements, preview compatibility, and rollback retention. ## Verification - All 377 workflow tests pass: `node --test .github/scripts/tests/*.test.mjs`. - All 129 release-registry tests pass: `pnpm test:release-registry`. - Focused source-proof, standard-image, preview, and workflow tests pass: 256 tests. - Focused image packaging/build-stamp tests pass: 16 tests. - Actionlint passes on all three changed workflow files. `git diff --check` passes. - Full local `pnpm build` and `pnpm -r typecheck` pass. - The policy follow-up updates an old assertion that required the removed readiness job. All 37 source-proof/release-workflow tests pass locally. - Full local `pnpm test:run` did not complete successfully while the Mac ran out of disk space. No full-suite pass is claimed. Removed 1.2 GiB of generated Cargo output from this isolated worktree with `cargo clean`. GitHub CI passed on the final head: 52 successful checks and 2 optional skips. - Fresh Greptile review for `4f5fe1951f0bd7f7739cf6655d395ff78f1ed944`: **5/5**, successful current-head check, zero review threads. - September 23 refresh: the unchanged PR head merges cleanly with current master `db8f8fe5b73a2697684a30261b0d306a9c631aba`. In an isolated temporary worktree, all 377 workflow tests and 29 release/preview tests pass on the combined tree. `git diff --cached --check` passes. - Refreshed Actionlint workflow validation passes with ShellCheck disabled. Full Actionlint reports the same 10 existing ShellCheck diagnostics as master, with no added diagnostics. No source changes or new PR commits were needed. - The full local build/typecheck and current-head Linux CI results above remain the verification for the unchanged PR head. They were not rerun for this metadata-only refresh. No image publication or tenant deployment was initiated for this refresh. ## Risks **Deployment prerequisite satisfied (September 23):** The combined cleanup release is deployed to staging and production, and production Support is verified. Active managed-fleet automation uses standard-image composition. Explicit immutable previews remain supported by the retained preview publisher. This PR is ready for maintainer review; keep auto-merge disabled and wait for explicit merge authorization. - A consumer still selecting `Cloud deployable v1` will stop advancing at the last legacy-ready commit. Confirm active automatic consumers use the standard-image composition contract before merge. - Legacy cloud release-channel aliases stop advancing. Standard self-hosted aliases continue. - This PR deletes no registry images, cache tags, migrators, credentials, or runner infrastructure. Existing immutable releases remain usable for rollback. - Explicit legacy previews remain for commit-specific operator deployments. Retiring that compatibility path requires a separate consumer migration. - These changes affect CI publication, not database schema or application behavior. ## Model Used OpenAI Codex, GPT-6. The runtime does not expose a more specific model identifier or context-window size. Used repository inspection, reasoning, code editing, shell tools, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip authored and GitHub committed
1 parent
db8f8fe5b7
commit
8ee8f1fd6e
23 files changed
+197
-1187
No files matched your search
@@ -1,144 +1,31 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { gzipSync } from "node:zlib";
|
||||
import { waitForCloudArtifacts, verifyManifestProvenance, migratorPublished } from "../../../scripts/cloud-readiness.mjs";
|
||||
import { artifactBase, descriptor } from "../../../scripts/cloud-migrator-artifacts.mjs";
|
||||
import { previewManifest } from "../../../scripts/preview-artifacts.mjs";
|
||||
|
||||
const sha = "a".repeat(40);
|
||||
const version = `0.0.0-preview.g${sha}`;
|
||||
const digest = `sha256:${"b".repeat(64)}`;
|
||||
const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
|
||||
const producer = { id: 123, head_sha: sha, head_branch: "master", path: ".github/workflows/cloud-migrator-artifacts.yml",
|
||||
head_repository: { id: 1170821064, full_name: "paperclipai/paperclip" }, event: "push", status: "completed", conclusion: "success" };
|
||||
function bundle() {
|
||||
const packages = {}; const files = new Map();
|
||||
const entries = { "": { dependencies: { "@paperclipai/db": version } } };
|
||||
for (const name of ["db", "shared"]) {
|
||||
const metadata = previewManifest({ name: `@paperclipai/${name}`, dependencies: {} }, sha);
|
||||
const bytes = Buffer.from(JSON.stringify(metadata));
|
||||
const header = Buffer.alloc(512); header.write("package/package.json"); header.write(bytes.length.toString(8).padStart(11, "0"), 124, 11); header[156] = 48;
|
||||
const padded = Buffer.alloc(Math.ceil(bytes.length / 512) * 512); bytes.copy(padded);
|
||||
const archive = gzipSync(Buffer.concat([header, padded, Buffer.alloc(1024)]));
|
||||
const pin = descriptor(archive, "tgz"); packages[name] = pin; files.set(pin.url, archive);
|
||||
entries[`node_modules/@paperclipai/${name}`] = { version, resolved: pin.url, integrity: pin.integrity, dependencies: metadata.dependencies };
|
||||
}
|
||||
const lock = Buffer.from(JSON.stringify({ lockfileVersion: 3, packages: entries }));
|
||||
const manifest = { version: 1, sourceSha: sha, packageVersion: version, packages, lockfile: descriptor(lock, "json") };
|
||||
files.set(manifest.lockfile.url, lock);
|
||||
const bytes = Buffer.from(JSON.stringify(manifest) + "\n");
|
||||
files.set(`${artifactBase}/${sha}/manifest.json`, bytes);
|
||||
return { manifest, bytes, files };
|
||||
}
|
||||
function registry({ missing = new Set(), failure, wrongImage = false, run = producer, objects = bundle() } = {}) {
|
||||
return async (url, options) => {
|
||||
assert.ok(!url.startsWith("https://registry.npmjs.org/"), "readiness must never wait for npm");
|
||||
if (failure) return json({}, failure);
|
||||
if (url.startsWith("https://api.github.com/")) {
|
||||
assert.match(url, new RegExp(`head_sha=${sha}&per_page=100&page=1$`));
|
||||
return json({ total_count: missing.has("migrator") ? 0 : 1, workflow_runs: missing.has("migrator") ? [] : [run] });
|
||||
}
|
||||
if (url.startsWith(artifactBase)) {
|
||||
assert.equal(options.headers?.Authorization, undefined, "GitHub credentials stay off the artifact origin");
|
||||
return objects.files.has(url) ? new Response(objects.files.get(url)) : json({}, 403);
|
||||
}
|
||||
if (url.includes("/token?")) return json({ token: "fixture" });
|
||||
if (url.includes("/manifests/")) return missing.has("image") ? json({}, 404) : json({ config: { digest } });
|
||||
if (url.includes("/blobs/")) return json({ config: { Labels: { "org.opencontainers.image.revision": wrongImage ? "c".repeat(40) : sha } } });
|
||||
throw new Error(`Unexpected request: ${url}`);
|
||||
};
|
||||
}
|
||||
const noSignature = async () => {}; // Signature enforcement is exercised separately below.
|
||||
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
|
||||
|
||||
test("readiness rechecks image and publisher, then verifies the exact signed bundle with no npm requests", async () => {
|
||||
const missing = new Set(["image", "migrator"]); const objects = bundle(); let clock = 0; let signatures = 0;
|
||||
const result = await waitForCloudArtifacts(sha, {
|
||||
fetchImpl: registry({ missing, objects }), token: "fixture", now: () => clock, intervalMs: 10, timeoutMs: 100, log: () => {},
|
||||
verifyProvenance: async (bytes, source) => { assert.deepEqual(bytes, objects.bytes); assert.equal(source, sha); signatures++; },
|
||||
sleep: async (ms) => {
|
||||
clock += ms;
|
||||
if (clock === 10) missing.delete("image");
|
||||
if (clock === 20) { missing.delete("migrator"); missing.add("image"); }
|
||||
if (clock === 30) missing.delete("image");
|
||||
},
|
||||
});
|
||||
assert.equal(clock, 30); assert.equal(signatures, 1);
|
||||
assert.deepEqual(result, { version: 1, sha, packageVersion: version });
|
||||
});
|
||||
|
||||
test("missing or in-progress publishers time out with a precise inventory and bounded sleep", async () => {
|
||||
for (const fixture of [{ missing: new Set(["migrator"]) }, { run: { ...producer, status: "in_progress", conclusion: null } }]) {
|
||||
let clock = 0; const sleeps = [];
|
||||
await assert.rejects(waitForCloudArtifacts(sha, {
|
||||
fetchImpl: registry(fixture), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {}, verifyProvenance: noSignature,
|
||||
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
|
||||
}), /timed out.*missing: migrator/);
|
||||
assert.deepEqual(sleeps, [20, 5]);
|
||||
}
|
||||
});
|
||||
|
||||
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true },
|
||||
...["failure", "cancelled", "skipped"].map((conclusion) => ({ run: { ...producer, conclusion } })),
|
||||
...[{ head_sha: "b".repeat(40) }, { head_branch: "feature" }, { path: ".github/workflows/evil.yml" },
|
||||
{ head_repository: { id: 123, full_name: "someone/paperclip" } }, { event: "pull_request" }].map((wrong) => ({ run: { ...producer, ...wrong } }))]) {
|
||||
test(`upstream errors, failed publication and identity mismatches fail immediately: ${JSON.stringify(fixture)}`, async () => {
|
||||
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(fixture), verifyProvenance: noSignature,
|
||||
sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {} }));
|
||||
});
|
||||
}
|
||||
|
||||
test("successful publication cannot hide inaccessible or corrupt archives or an invalid signature", async () => {
|
||||
for (const corrupt of [false, true]) {
|
||||
const objects = bundle();
|
||||
if (corrupt) objects.files.set(objects.manifest.packages.db.url, Buffer.from("corrupt"));
|
||||
else objects.files.delete(objects.manifest.packages.db.url);
|
||||
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry({ objects }), verifyProvenance: noSignature, log: () => {} }), /download failed|immutable pin/);
|
||||
}
|
||||
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(), verifyProvenance: async () => { throw new Error("invalid signature"); }, log: () => {} }), /invalid signature/);
|
||||
});
|
||||
|
||||
test("CLI verifies the exact bytes, source, master workflow and hosted runner and cleans up on failure", () => {
|
||||
let temporary;
|
||||
assert.throws(() => verifyManifestProvenance(Buffer.from("exact manifest\n"), sha, { exec: (cmd, args) => {
|
||||
assert.equal(cmd, "gh"); assert.deepEqual(args.slice(0, 2), ["attestation", "verify"]); temporary = args[2];
|
||||
assert.equal(readFileSync(temporary, "utf8"), "exact manifest\n");
|
||||
for (const [flag, value] of [["--repo", "paperclipai/paperclip"], ["--source-digest", sha], ["--source-ref", "refs/heads/master"],
|
||||
["--cert-identity", "https://github.com/paperclipai/paperclip/.github/workflows/cloud-migrator-artifacts.yml@refs/heads/master"]]) assert.equal(args[args.indexOf(flag) + 1], value);
|
||||
assert.ok(args.includes("--deny-self-hosted-runners")); throw new Error("verification rejected");
|
||||
} }), /verification rejected/);
|
||||
assert.equal(existsSync(temporary), false);
|
||||
});
|
||||
|
||||
test("invalid source and timing configuration are rejected before registry access", async () => {
|
||||
const fetchImpl = async () => assert.fail("invalid inputs must not reach a registry");
|
||||
await assert.rejects(waitForCloudArtifacts("master", { fetchImpl }), /full immutable commit SHA/);
|
||||
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
|
||||
});
|
||||
|
||||
test("versioned readiness retains every source gate and removes duplicate automatic npm publication", () => {
|
||||
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
|
||||
test("retirement preserves exact-source verification without issuing legacy deployment readiness", () => {
|
||||
assert.match(workflow, /push:\s*\n\s*branches: \[master\]/);
|
||||
assert.match(workflow, /uses: \.\/\.github\/workflows\/release-verify.yml\s+with:\s+ref: \$\{\{ github.sha \}\}/);
|
||||
assert.match(workflow, /uses: \.\/\.github\/workflows\/docker-cloud.yml/);
|
||||
assert.match(workflow, /attestations: read/); assert.match(workflow, /GH_TOKEN: \$\{\{ github.token \}\}/);
|
||||
const ready = workflow.split(" ready:")[1];
|
||||
assert.match(ready, /name: Cloud deployable v1/); assert.match(ready, /needs: \[verify, image, artifacts\]/);
|
||||
assert.match(ready, /if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'/);
|
||||
assert.doesNotMatch(ready, /^\s*(?:if:.*always\(|continue-on-error:)/m);
|
||||
assert.doesNotMatch(workflow, /secrets: inherit|id-token: write|actions: write|checks: write|uses: .*@v\d\b/);
|
||||
assert.equal(existsSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url)), false);
|
||||
const proof = workflow.split(" source_verified:")[1];
|
||||
assert.ok(proof);
|
||||
assert.match(proof, /name: Cloud source verified v1/);
|
||||
assert.match(proof, /needs: \[verify\]/);
|
||||
assert.match(proof, /if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'/);
|
||||
assert.doesNotMatch(proof, /^\s*(?:if:.*always\(|continue-on-error:)/m);
|
||||
assert.doesNotMatch(workflow, /Cloud deployable v1|docker-cloud.yml|cloud-readiness.mjs|^ (image|artifacts|ready):/m);
|
||||
assert.doesNotMatch(workflow, /packages: write|secrets: inherit|id-token: write|actions: write|checks: write|uses: .*@v\d\b/);
|
||||
assert.equal(existsSync(new URL("../../workflows/docker-cloud.yml", import.meta.url)), false);
|
||||
});
|
||||
|
||||
|
||||
test("later manual failures or pending retries cannot hide an earlier successful immutable publication", async () => {
|
||||
for (const latest of [{ status: "completed", conclusion: "failure" }, { status: "in_progress", conclusion: null }]) {
|
||||
let calls = 0;
|
||||
assert.equal(await migratorPublished(sha, async (url) => {
|
||||
calls++;
|
||||
if (url.endsWith("page=1")) return json({ total_count: 101, workflow_runs: Array.from({ length: 100 }, (_, i) => ({ ...producer, ...latest, id: 200 + i, event: "workflow_dispatch" })) });
|
||||
assert.ok(url.endsWith("page=2")); return json({ total_count: 101, workflow_runs: [producer] });
|
||||
}), true);
|
||||
assert.equal(calls, 2);
|
||||
}
|
||||
test("standard image provenance and independent exact-source migrators remain available", () => {
|
||||
const docker = readFileSync(new URL("../../workflows/docker.yml", import.meta.url), "utf8");
|
||||
assert.match(docker, /target: production/);
|
||||
assert.match(docker, /type=raw,value=sha-\$\{\{ github.sha \}\}/);
|
||||
assert.match(docker, /run: node scripts\/standard-image-contract.mjs --resolve "\$GITHUB_SHA"/);
|
||||
assert.match(docker, /subject-digest: \$\{\{ steps.standard.outputs.digest \}\}/);
|
||||
const migrator = readFileSync(new URL("../../workflows/cloud-migrator-artifacts.yml", import.meta.url), "utf8");
|
||||
assert.match(migrator, /push:\s*\n\s*branches: \[master\]/);
|
||||
assert.match(migrator, /uses: actions\/attest@/);
|
||||
assert.doesNotMatch(docker, /build-and-push-cloud|docker-cloud.yml|canary-cloud/);
|
||||
});
|
||||
@@ -1,35 +0,0 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { runInNewContext } from "node:vm";
|
||||
|
||||
const workflow = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
|
||||
// Exercise the workflow's actual boolean expression. Its string comparisons and
|
||||
// boolean operators have the same results in JS for these canonical contexts.
|
||||
const expression = workflow.match(/^ runs-on: \$\{\{ (.+) \}\}$/m)?.[1];
|
||||
assert.ok(expression, "cloud routing must remain an explicit job expression");
|
||||
const timeoutExpression = workflow.match(/^ timeout-minutes: \$\{\{ (.+) \}\}$/m)?.[1];
|
||||
assert.ok(timeoutExpression, "AWS jobs must finish before the Fleet instance lifetime");
|
||||
const fleet = "runs-on/fleet=paperclip-cloud-build-x64/env=public-ci";
|
||||
const base = { repository: "paperclipai/paperclip", repository_id: "1170821064", ref: "refs/heads/master", event_name: "push" };
|
||||
for (const { name, github = {}, enabled = "true", expected = "ubuntu-latest" } of [
|
||||
{ name: "canonical master push", expected: fleet },
|
||||
{ name: "manual master build", github: { event_name: "workflow_dispatch" }, expected: fleet },
|
||||
{ name: "disabled switch", enabled: "false" },
|
||||
{ name: "missing switch", enabled: "" },
|
||||
{ name: "invalid switch", enabled: "yes" },
|
||||
{ name: "fork", github: { repository: "someone/paperclip", repository_id: "123" } },
|
||||
{ name: "wrong repository identity", github: { repository_id: "123" } },
|
||||
{ name: "pull request", github: { event_name: "pull_request", ref: "refs/pull/123/merge" } },
|
||||
{ name: "privileged PR event", github: { event_name: "pull_request_target" } },
|
||||
{ name: "release tag", github: { ref: "refs/tags/v2026.911.0" } },
|
||||
{ name: "branch push", github: { ref: "refs/heads/feature" } },
|
||||
{ name: "manual branch build", github: { event_name: "workflow_dispatch", ref: "refs/heads/feature" } },
|
||||
{ name: "workflow completion event", github: { event_name: "workflow_run" } },
|
||||
]) {
|
||||
test(`cloud runner routing: ${name}`, () => {
|
||||
const context = { github: { ...base, ...github }, vars: { AWS_CLOUD_BUILDS_ENABLED: enabled } };
|
||||
assert.equal(runInNewContext(expression, context), expected);
|
||||
assert.equal(runInNewContext(timeoutExpression, context), expected === fleet ? 40 : 60);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { spawnSync } from "node:child_process";
|
||||
|
||||
const workflow = readFileSync(new URL("../../workflows/docker.yml", import.meta.url), "utf8");
|
||||
const job = workflow.split(" promote_canary_channel:\n")[1];
|
||||
const script = job.split(" run: |\n")[1].split("\n").map(line => line.replace(/^ {10}/, "")).join("\n");
|
||||
const sha = "a".repeat(40);
|
||||
|
||||
test("canary promotion waits for the standard manifest and keeps its serialized channel", () => {
|
||||
assert.match(job, /needs: \[merge-and-push\]/);
|
||||
assert.match(job, /group: docker-canary-channel-promotion/);
|
||||
assert.match(job, /cancel-in-progress: false/);
|
||||
});
|
||||
|
||||
for (const [name, commitPresent, imagePresent] of [
|
||||
["promotes the current npm canary without a cloud image", true, true],
|
||||
["waits when the standard image is missing", true, false],
|
||||
["waits when the npm canary tag has not resolved", false, false],
|
||||
]) {
|
||||
test(name, () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), "standard-canary-promotion-"));
|
||||
const log = path.join(dir, "calls.jsonl");
|
||||
const fixture = `#!${process.execPath}
|
||||
const fs = require("node:fs");
|
||||
const command = require("node:path").basename(process.argv[1]);
|
||||
const args = process.argv.slice(2);
|
||||
fs.appendFileSync(process.env.TEST_CALLS, JSON.stringify({ command, args }) + "\\n");
|
||||
if (command === "curl") process.stdout.write(JSON.stringify({ canary: "2026.922.0-canary.1" }));
|
||||
else if (command === "gh") { if (process.env.COMMIT_PRESENT !== "true") process.exit(1); process.stdout.write(process.env.TEST_SHA); }
|
||||
else if (args.slice(0, 3).join(" ") === "buildx imagetools inspect") process.exit(process.env.IMAGE_PRESENT === "true" ? 0 : 1);
|
||||
else if (args.slice(0, 3).join(" ") !== "buildx imagetools create") process.exit(99);
|
||||
`;
|
||||
try {
|
||||
for (const command of ["curl", "gh", "docker"]) writeFileSync(path.join(dir, command), fixture, { mode: 0o755 });
|
||||
const result = spawnSync("bash", ["-e", "-o", "pipefail", "-c", script], {
|
||||
encoding: "utf8", env: {
|
||||
...process.env, PATH: `${dir}${path.delimiter}${process.env.PATH}`,
|
||||
IMAGE: "ghcr.io/paperclipai/paperclip", GITHUB_REPOSITORY: "paperclipai/paperclip",
|
||||
TEST_CALLS: log, TEST_SHA: sha, COMMIT_PRESENT: String(commitPresent), IMAGE_PRESENT: String(imagePresent),
|
||||
},
|
||||
});
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const calls = readFileSync(log, "utf8").trim().split("\n").map(line => JSON.parse(line));
|
||||
assert.ok(calls.find(call => call.command === "gh").args.some(arg => arg.includes("canary%2Fv2026.922.0-canary.1")));
|
||||
const docker = calls.filter(call => call.command === "docker").map(call => call.args);
|
||||
assert.deepEqual(docker, [
|
||||
...(commitPresent ? [["buildx", "imagetools", "inspect", "ghcr.io/paperclipai/paperclip:sha-aaaaaaa"]] : []),
|
||||
...(commitPresent && imagePresent ? [["buildx", "imagetools", "create", "-t", "ghcr.io/paperclipai/paperclip:canary", "ghcr.io/paperclipai/paperclip:sha-aaaaaaa"]] : []),
|
||||
]);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { spawnSync } from "node:child_process";
|
||||
|
||||
const workflow = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
|
||||
const step = workflow.split(" - name: Free runner disk")[1].split(" - name: Login to GitHub Container Registry")[0];
|
||||
const script = step.split(" run: |\n")[1].split("\n").map((line) => line.replace(/^ {10}/, "")).join("\n");
|
||||
const threshold = 64 * 1024 * 1024;
|
||||
|
||||
for (const { name, dockerFree, workspaceFree, dfStatus = "0", infoStatus = "0", cleanup } of [
|
||||
{ name: "ample free space", dockerFree: threshold + 1, workspaceFree: threshold + 1, cleanup: false },
|
||||
{ name: "exactly the headroom threshold", dockerFree: threshold, workspaceFree: threshold, cleanup: false },
|
||||
{ name: "Docker filesystem below threshold", dockerFree: threshold - 1, workspaceFree: threshold + 1, cleanup: true },
|
||||
{ name: "workspace filesystem below threshold", dockerFree: threshold + 1, workspaceFree: threshold - 1, cleanup: true },
|
||||
{ name: "invalid Docker measurement", dockerFree: "unknown", workspaceFree: threshold + 1, cleanup: true },
|
||||
{ name: "invalid workspace measurement", dockerFree: threshold + 1, workspaceFree: "unknown", cleanup: true },
|
||||
{ name: "failed df command", dockerFree: threshold + 1, workspaceFree: threshold + 1, dfStatus: "1", cleanup: true },
|
||||
{ name: "failed Docker inspection", dockerFree: threshold + 1, workspaceFree: threshold + 1, infoStatus: "1", cleanup: true },
|
||||
]) {
|
||||
test(`cloud disk cleanup: ${name}`, () => {
|
||||
const dir = mkdtempSync(path.join(tmpdir(), "cloud-disk-test-"));
|
||||
const log = path.join(dir, "commands.log");
|
||||
// Every mutating command is a recording fixture; no real SDKs, caches,
|
||||
// images, or directories are deleted when the workflow shell executes.
|
||||
const fixture = `#!/bin/bash
|
||||
printf '%s %s\\n' "\${0##*/}" "$*" >> "$COMMAND_LOG"
|
||||
case "\${0##*/}" in
|
||||
df)
|
||||
printf 'Filesystem 1024-blocks Used Available Capacity Mounted on\\n'
|
||||
if [ "$1" = '-Pk' ]; then
|
||||
printf '/dev/docker 200000000 1 %s 1%% /docker\\n' "$DOCKER_FREE"
|
||||
printf '/dev/workspace 200000000 1 %s 1%% /workspace\\n' "$WORKSPACE_FREE"
|
||||
exit "$DF_STATUS"
|
||||
fi
|
||||
;;
|
||||
docker)
|
||||
if [ "$1" = 'info' ]; then
|
||||
printf '/docker-data\\n'
|
||||
exit "$INFO_STATUS"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
`;
|
||||
try {
|
||||
for (const command of ["df", "docker", "pnpm", "sudo"]) {
|
||||
writeFileSync(path.join(dir, command), fixture, { mode: 0o755 });
|
||||
}
|
||||
const result = spawnSync("bash", ["-c", script], {
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, PATH: `${dir}:${process.env.PATH}`, GITHUB_WORKSPACE: "/workspace", COMMAND_LOG: log,
|
||||
DOCKER_FREE: String(dockerFree), WORKSPACE_FREE: String(workspaceFree), DF_STATUS: dfStatus, INFO_STATUS: infoStatus },
|
||||
});
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
const commands = readFileSync(log, "utf8");
|
||||
if (infoStatus === "0") assert.match(commands, /df -Pk \/docker-data \/workspace/);
|
||||
assert.equal(commands.includes("pnpm store prune"), cleanup);
|
||||
assert.equal(commands.includes("sudo rm -rf /usr/share/dotnet"), cleanup);
|
||||
assert.equal(commands.includes("docker system prune -af"), cleanup);
|
||||
assert.equal(result.stdout.includes("skipping cleanup"), !cleanup);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
});
|
||||
}
|
||||
@@ -6,7 +6,6 @@ const workflows = [
|
||||
'.github/workflows/refresh-lockfile.yml',
|
||||
'.github/workflows/pr-trusted.yml',
|
||||
'.github/workflows/docker.yml',
|
||||
'.github/workflows/docker-cloud.yml',
|
||||
];
|
||||
|
||||
test('lockfile repair workflows resolve dependencies instead of updating metadata only', async () => {
|
||||
|
||||
@@ -87,9 +87,7 @@ test("Cloud readiness bookkeeping never waits for the AWS verification fleet", (
|
||||
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
|
||||
const bodies = new Map();
|
||||
for (const [name, needs] of [
|
||||
["artifacts", null],
|
||||
["source_verified", "[verify]"],
|
||||
["ready", "[verify, image, artifacts]"],
|
||||
]) {
|
||||
const body = workflow.match(new RegExp(`^ ${name}:\\n([\\s\\S]*?)(?=^ [a-z_]+:|(?![\\s\\S]))`, "m"))?.[1];
|
||||
assert.ok(body, `missing ${name} job`);
|
||||
@@ -100,8 +98,6 @@ test("Cloud readiness bookkeeping never waits for the AWS verification fleet", (
|
||||
assert.match(body, /^ +SOURCE_SHA: \$\{\{ github.sha \}\}$/m);
|
||||
assert.equal(body.match(/^ needs: (.+)$/m)?.[1] ?? null, needs, `${name} prerequisites`);
|
||||
}
|
||||
assert.match(bodies.get("artifacts"), /^ run: node scripts\/cloud-readiness.mjs "\$SOURCE_SHA"$/m);
|
||||
assert.match(bodies.get("source_verified"), /^ run: node --test scripts\/cloud-source-verification.test.mjs$/m);
|
||||
assert.match(bodies.get("source_verified"), /echo "Cloud source verified v1: \$SOURCE_SHA"/);
|
||||
assert.match(bodies.get("ready"), /echo "Cloud deployable v1: \$SOURCE_SHA"/);
|
||||
});
|
||||
Reference in new issue
Block a user