fix(ci): remove npm propagation from cloud readiness (#13456)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud needs a verified image and matching database
migrator before it can deploy a merge.
> - New npm package versions can take minutes to become downloadable
after the package build finishes.
> - The direct producer now publishes signed archives and a complete
dependency lockfile for each master commit.
> - This pull request makes readiness verify those artifacts and removes
the duplicate automatic npm migrator run.
> - Deployment still requires all source checks, exact image identity,
migration compatibility, and pinned dependencies.

## Linked Issues or Issue Description

Refs: #13455, #13454, #13192

**What existing behavior does this improve?**
The time from a master merge to the `Cloud deployable v1` signal.

**Current behavior**
Readiness polls npm metadata for the new DB and shared versions. An
automatic dispatcher also starts a separate npm-only migrator workflow.
A measured source built its packages at 06:22:41 UTC on 2026-09-15, but
both npm archives were not downloadable until 06:31:56 UTC.

**Proposed behavior**
Wait for the successful exact-source direct producer, verify its signed
manifest and all pinned downloads, and publish readiness only after the
existing source and image jobs pass. Keep manual npm migrators and
branch previews available.

**Reason and benefit**
Remove new-version npm propagation from merge-to-deployable time. The
gain depends on whether image building or source verification finishes
later; it is not a fixed subtraction from every run.

## What Changed

- Require a successful producer from the canonical repository, exact
commit, master ref, expected workflow, and approved event.
- Verify the manifest's GitHub attestation with the hosted GitHub CLI.
Enforce the exact source SHA, master workflow identity, and hosted
runner.
- Download and validate both archives and the complete dependency
lockfile after publication succeeds. Reject invalid signatures,
inaccessible objects, corrupt bytes, and source mismatches.
- Remove automatic npm-only migrator dispatch. Retain manual release and
branch-preview publication.
- Document the cloud feature-switch prerequisite and coordinated
rollback.

## Verification

- `node --test .github/scripts/tests/*.test.mjs`: 405 pass.
- Focused readiness, routing, preview, and artifact tests: 249 pass.
- Workflow lint and `git diff --check`: pass.
- `pnpm test:release-registry`: 139 pass after installing this
worktree's dependencies.
- All latest-head GitHub CI checks passed. Greptile is 5/5 with no
unresolved comments.
- Application source is unchanged. Common-source local typecheck and
build passed. The full local application suite has the documented macOS
read-only-directory rename limitation from #13454 (13 failures in two
unchanged suites); Linux CI is the final application gate.
- Live readiness verification of master
da77a0c28c passed in 6.52 seconds,
including the real GitHub CLI signature policy and all artifact
downloads.
- Cloud consumer resolution with the certificate encoding fix passed in
5.45 seconds with zero npm metadata requests or npm processes. The
consumer is deployed and enabled in staging and production; their live
resolution APIs passed in 2.34 and 2.38 seconds. Both report the
expected fixed harness commit. A fresh tenant deployment follows this
cutover merge.

## Risks

- `Cloud deployable v1` no longer promises npm preview availability.
Enable the cloud direct-artifact consumer in staging and production
before merging this change.
- Artifact storage and GitHub attestations become required services for
new direct releases. Missing or invalid evidence fails explicitly.
- Restore the old npm dispatcher and readiness gate together before
disabling the consumer switch. Retain artifacts referenced by existing
releases.
- This change does not expand AWS runner access. The producer and
readiness bookkeeping use GitHub-hosted runners. Existing PR allowlists
and source verification gates remain enforced.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused checks; full
application host limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip authored and GitHub committed 2026-09-15 01:14:43 -07:00
1 parent 9ed55f6931
commit 4cc387f907
9 files changed
+211 -122

No files matched your search

+102 -49
View File
@@ -1,20 +1,47 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { waitForCloudArtifacts } from "../../../scripts/cloud-readiness.mjs";
import { existsSync, readFileSync } from "node:fs";
import { gzipSync } from "node:zlib";
import { waitForCloudArtifacts, verifyManifestProvenance, migratorPublished } from "../../../scripts/cloud-readiness.mjs";
import { artifactBase, descriptor } from "../../../scripts/cloud-migrator-artifacts.mjs";
import { previewManifest } from "../../../scripts/preview-artifacts.mjs";
const sha = "a".repeat(40);
const version = `0.0.0-preview.g${sha}`;
const digest = `sha256:${"b".repeat(64)}`;
const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
function registry({ missing = new Set(), failure, wrongImage = false, wrongPackage = false } = {}) {
return async (url) => {
const producer = { id: 123, head_sha: sha, head_branch: "master", path: ".github/workflows/cloud-migrator-artifacts.yml",
head_repository: { id: 1170821064, full_name: "paperclipai/paperclip" }, event: "push", status: "completed", conclusion: "success" };
function bundle() {
const packages = {}; const files = new Map();
const entries = { "": { dependencies: { "@paperclipai/db": version } } };
for (const name of ["db", "shared"]) {
const metadata = previewManifest({ name: `@paperclipai/${name}`, dependencies: {} }, sha);
const bytes = Buffer.from(JSON.stringify(metadata));
const header = Buffer.alloc(512); header.write("package/package.json"); header.write(bytes.length.toString(8).padStart(11, "0"), 124, 11); header[156] = 48;
const padded = Buffer.alloc(Math.ceil(bytes.length / 512) * 512); bytes.copy(padded);
const archive = gzipSync(Buffer.concat([header, padded, Buffer.alloc(1024)]));
const pin = descriptor(archive, "tgz"); packages[name] = pin; files.set(pin.url, archive);
entries[`node_modules/@paperclipai/${name}`] = { version, resolved: pin.url, integrity: pin.integrity, dependencies: metadata.dependencies };
}
const lock = Buffer.from(JSON.stringify({ lockfileVersion: 3, packages: entries }));
const manifest = { version: 1, sourceSha: sha, packageVersion: version, packages, lockfile: descriptor(lock, "json") };
files.set(manifest.lockfile.url, lock);
const bytes = Buffer.from(JSON.stringify(manifest) + "\n");
files.set(`${artifactBase}/${sha}/manifest.json`, bytes);
return { manifest, bytes, files };
}
function registry({ missing = new Set(), failure, wrongImage = false, run = producer, objects = bundle() } = {}) {
return async (url, options) => {
assert.ok(!url.startsWith("https://registry.npmjs.org/"), "readiness must never wait for npm");
if (failure) return json({}, failure);
if (url.startsWith("https://registry.npmjs.org/")) {
const name = decodeURIComponent(new URL(url).pathname.split("/")[1]);
if (missing.has(name.split("/")[1])) return json({}, 404);
const pkg = previewManifest({ name, version: "0.0.0" }, sha);
return json({ ...pkg, ...(wrongPackage ? { gitHead: "c".repeat(40) } : {}), dist: { integrity: "sha512-fixture", tarball: "https://registry.npmjs.org/fixture.tgz" } });
if (url.startsWith("https://api.github.com/")) {
assert.match(url, new RegExp(`head_sha=${sha}&per_page=100&page=1$`));
return json({ total_count: missing.has("migrator") ? 0 : 1, workflow_runs: missing.has("migrator") ? [] : [run] });
}
if (url.startsWith(artifactBase)) {
assert.equal(options.headers?.Authorization, undefined, "GitHub credentials stay off the artifact origin");
return objects.files.has(url) ? new Response(objects.files.get(url)) : json({}, 403);
}
if (url.includes("/token?")) return json({ token: "fixture" });
if (url.includes("/manifests/")) return missing.has("image") ? json({}, 404) : json({ config: { digest } });
@@ -22,70 +49,96 @@ function registry({ missing = new Set(), failure, wrongImage = false, wrongPacka
throw new Error(`Unexpected request: ${url}`);
};
}
const noSignature = async () => {}; // Signature enforcement is exercised separately below.
test("readiness requires the image and both exact-source packages on the successful poll", async () => {
const missing = new Set(["image", "shared", "db"]);
let clock = 0;
const states = [];
test("readiness rechecks image and publisher, then verifies the exact signed bundle with no npm requests", async () => {
const missing = new Set(["image", "migrator"]); const objects = bundle(); let clock = 0; let signatures = 0;
const result = await waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing }), now: () => clock, intervalMs: 10, timeoutMs: 100, log: (message) => states.push(message),
fetchImpl: registry({ missing, objects }), token: "fixture", now: () => clock, intervalMs: 10, timeoutMs: 100, log: () => {},
verifyProvenance: async (bytes, source) => { assert.deepEqual(bytes, objects.bytes); assert.equal(source, sha); signatures++; },
sleep: async (ms) => {
clock += ms;
if (clock === 10) missing.delete("image");
if (clock === 20) missing.delete("shared");
if (clock === 30) { missing.delete("db"); missing.add("image"); }
if (clock === 40) missing.delete("image");
if (clock === 20) { missing.delete("migrator"); missing.add("image"); }
if (clock === 30) missing.delete("image");
},
});
assert.equal(clock, 40, "an artifact disappearing before the final poll must prevent readiness");
assert.deepEqual(result, { version: 1, sha, packageVersion: `0.0.0-preview.g${sha}` });
assert.match(states.at(-1), /Cloud artifacts available/);
assert.equal(clock, 30); assert.equal(signatures, 1);
assert.deepEqual(result, { version: 1, sha, packageVersion: version });
});
test("missing artifacts time out with a precise inventory and bounded sleep", async () => {
let clock = 0;
const sleeps = [];
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing: new Set(["db"]) }), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {},
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
}), /timed out.*missing: db/);
assert.deepEqual(sleeps, [20, 5]);
});
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true }, { wrongPackage: true }]) {
test(`registry errors and identity mismatches fail without waiting: ${JSON.stringify(fixture)}`, async () => {
test("missing or in-progress publishers time out with a precise inventory and bounded sleep", async () => {
for (const fixture of [{ missing: new Set(["migrator"]) }, { run: { ...producer, status: "in_progress", conclusion: null } }]) {
let clock = 0; const sleeps = [];
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry(fixture), sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {},
}));
fetchImpl: registry(fixture), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {}, verifyProvenance: noSignature,
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
}), /timed out.*missing: migrator/);
assert.deepEqual(sleeps, [20, 5]);
}
});
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true },
...["failure", "cancelled", "skipped"].map((conclusion) => ({ run: { ...producer, conclusion } })),
...[{ head_sha: "b".repeat(40) }, { head_branch: "feature" }, { path: ".github/workflows/evil.yml" },
{ head_repository: { id: 123, full_name: "someone/paperclip" } }, { event: "pull_request" }].map((wrong) => ({ run: { ...producer, ...wrong } }))]) {
test(`upstream errors, failed publication and identity mismatches fail immediately: ${JSON.stringify(fixture)}`, async () => {
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(fixture), verifyProvenance: noSignature,
sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {} }));
});
}
test("successful publication cannot hide inaccessible or corrupt archives or an invalid signature", async () => {
for (const corrupt of [false, true]) {
const objects = bundle();
if (corrupt) objects.files.set(objects.manifest.packages.db.url, Buffer.from("corrupt"));
else objects.files.delete(objects.manifest.packages.db.url);
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry({ objects }), verifyProvenance: noSignature, log: () => {} }), /download failed|immutable pin/);
}
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(), verifyProvenance: async () => { throw new Error("invalid signature"); }, log: () => {} }), /invalid signature/);
});
test("CLI verifies the exact bytes, source, master workflow and hosted runner and cleans up on failure", () => {
let temporary;
assert.throws(() => verifyManifestProvenance(Buffer.from("exact manifest\n"), sha, { exec: (cmd, args) => {
assert.equal(cmd, "gh"); assert.deepEqual(args.slice(0, 2), ["attestation", "verify"]); temporary = args[2];
assert.equal(readFileSync(temporary, "utf8"), "exact manifest\n");
for (const [flag, value] of [["--repo", "paperclipai/paperclip"], ["--source-digest", sha], ["--source-ref", "refs/heads/master"],
["--cert-identity", "https://github.com/paperclipai/paperclip/.github/workflows/cloud-migrator-artifacts.yml@refs/heads/master"]]) assert.equal(args[args.indexOf(flag) + 1], value);
assert.ok(args.includes("--deny-self-hosted-runners")); throw new Error("verification rejected");
} }), /verification rejected/);
assert.equal(existsSync(temporary), false);
});
test("invalid source and timing configuration are rejected before registry access", async () => {
const fetchImpl = async () => assert.fail("invalid inputs must not reach a registry");
await assert.rejects(waitForCloudArtifacts("master", { fetchImpl }), /full immutable commit SHA/);
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) {
await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
}
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
});
test("the versioned readiness job requires successful source, image and artifact jobs", () => {
test("versioned readiness retains every source gate and removes duplicate automatic npm publication", () => {
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
assert.match(workflow, /push:\s*\n\s*branches: \[master\]/);
assert.match(workflow, /group: cloud-readiness-\$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/release-verify.yml\s+with:\s+ref: \$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/docker-cloud.yml/);
assert.match(workflow, /attestations: read/); assert.match(workflow, /GH_TOKEN: \$\{\{ github.token \}\}/);
const ready = workflow.split(" ready:")[1];
assert.match(ready, /name: Cloud deployable v1/);
assert.match(ready, /needs: \[verify, image, artifacts\]/);
assert.match(ready, /name: Cloud deployable v1/); assert.match(ready, /needs: \[verify, image, artifacts\]/);
assert.match(ready, /if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'/);
assert.doesNotMatch(ready, /^\s*(?:if:.*always\(|continue-on-error:)/m);
assert.doesNotMatch(workflow, /secrets: inherit|id-token: write|actions: write|checks: write|uses: .*@v\d\b/);
const cloud = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
assert.doesNotMatch(cloud, /^ push:/m, "the master image must build only once");
const migrator = readFileSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url), "utf8");
assert.match(migrator, /push:\s*\n\s*branches: \[master\]/);
assert.match(migrator, /SOURCE_SHA: \$\{\{ github.sha \}\}/);
assert.match(migrator, /gh workflow run release.yml .*--ref master/);
assert.match(migrator, /--field channel=cloud-migrator/);
assert.match(migrator, /--field source_ref="\$SOURCE_SHA"/);
assert.equal(existsSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url)), false);
});
test("later manual failures or pending retries cannot hide an earlier successful immutable publication", async () => {
for (const latest of [{ status: "completed", conclusion: "failure" }, { status: "in_progress", conclusion: null }]) {
let calls = 0;
assert.equal(await migratorPublished(sha, async (url) => {
calls++;
if (url.endsWith("page=1")) return json({ total_count: 101, workflow_runs: Array.from({ length: 100 }, (_, i) => ({ ...producer, ...latest, id: 200 + i, event: "workflow_dispatch" })) });
assert.ok(url.endsWith("page=2")); return json({ total_count: 101, workflow_runs: [producer] });
}), true);
assert.equal(calls, 2);
}
});
@@ -11,7 +11,7 @@ const base = {
};
const expectedJobs = {
"cloud-readiness.yml": [],
"cloud-artifacts.yml": ["dispatch_migrator"],
"cloud-migrator-artifacts.yml": [],
"release-verify.yml": ["typecheck", "general_tests", "serialized_tests", "runner_workflow_evals", "verify_paperclip_runner", "build"],
"runner-chaos-evals.yml": ["chaos_and_recovery"],
"release.yml": ["plan_preview", "package_preview"],
-34
View File
@@ -1,34 +0,0 @@
name: Cloud artifacts
on:
push:
branches: [master]
workflow_dispatch:
permissions: {}
jobs:
dispatch_migrator:
name: Start exact-source cloud migrator publication
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 5
permissions:
actions: write
steps:
# This separate workflow starts at merge, outside the full npm release's
# concurrency group. Publication stays in release.yml so npm recognizes
# the established trusted-publisher identity and npm-canary environment.
# No source checkout or package code runs with the dispatch credential.
- name: Dispatch the migrator-only release
env:
GH_TOKEN: ${{ github.token }}
SOURCE_SHA: ${{ github.sha }}
run: |
set -euo pipefail
request_id="$(cat /proc/sys/kernel/random/uuid)"
gh workflow run release.yml --repo "$GITHUB_REPOSITORY" --ref master \
--field channel=cloud-migrator \
--field source_ref="$SOURCE_SHA" \
--field request_id="$request_id"
echo "Started Cloud migrator $SOURCE_SHA in release.yml (request $request_id)." >> "$GITHUB_STEP_SUMMARY"
+8 -3
View File
@@ -37,6 +37,8 @@ jobs:
timeout-minutes: 35
permissions:
contents: read
actions: read
attestations: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
@@ -46,6 +48,7 @@ jobs:
node-version: 24
- name: Wait for verified image and exact-source migrator
env:
GH_TOKEN: ${{ github.token }}
SOURCE_SHA: ${{ github.sha }}
run: node scripts/cloud-readiness.mjs "$SOURCE_SHA"
@@ -91,6 +94,8 @@ jobs:
env:
SOURCE_SHA: ${{ github.sha }}
run: |
echo "Cloud deployable v1: $SOURCE_SHA" >> "$GITHUB_STEP_SUMMARY"
echo "Source verification passed; the full-SHA image and exact-source migrator are available." >> "$GITHUB_STEP_SUMMARY"
echo "Deployment tooling must still resolve and pin the image and migrator and validate migration compatibility." >> "$GITHUB_STEP_SUMMARY"
{
echo "Cloud deployable v1: $SOURCE_SHA"
echo "Source verification passed; the full-SHA image and exact-source migrator are available."
echo "Deployment tooling must still resolve and pin the image and migrator and validate migration compatibility."
} >> "$GITHUB_STEP_SUMMARY"
+26 -14
View File
@@ -9,11 +9,12 @@ The `Cloud readiness` workflow starts for every master push. Its versioned
image, including Sentry resolution and orphan reaping, then publishes the
full-SHA cloud tag. Cloud readiness owns the master trigger so there is one
cloud build per push. Release tags and manual Docker runs retain their callers.
- The full-SHA image and both exact-source npm packages are visible. The
packages are `@paperclipai/shared` and `@paperclipai/db` at
`0.0.0-preview.g<FULL_SHA>`, published through the migrator-only release lane.
Registry metadata must match the full commit, and the database package must
pin the matching shared package.
- The full-SHA image is visible and the exact-source `Cloud migrator artifacts`
workflow has succeeded. Readiness verifies the manifest's GitHub attestation
against the full SHA, canonical master workflow, and GitHub-hosted runner,
then downloads and validates both package archives and the prepared dependency
lockfile. The database package pins the matching shared package. New-version
npm metadata and tarball propagation are outside this path.
The Cloud workflow builds the image with `USER_UID=1001` and `USER_GID=1001`,
matching the managed runtime. This avoids a startup user remap, which can walk
@@ -56,16 +57,19 @@ before that bot's PR merges. Verification must install and test that commit
without waiting for another merge. The generated lockfile stays in the job's
workspace; these checks do not commit it back to the repository.
The artifact wait runs for up to 30 minutes and reports what is missing. Only
an HTTP 404 means publication is pending; authorization errors, upstream outages,
and identity mismatches fail the job. A failed, cancelled, or skipped prerequisite
The artifact wait runs for up to 30 minutes and reports what is missing. A
missing image or an exact-source publisher with no successful run yet means publication
is pending. An earlier successful push or manual run remains valid after a failed
retry because publication is immutable. If all matching runs failed, readiness
fails. An invalid signature, inaccessible or corrupt
bundle, authorization error, or identity mismatch fails the job. A failed, cancelled, or skipped prerequisite
cannot produce a successful readiness job. Retry the failed publication or build,
then rerun the failed readiness workflow jobs to check the same commit again.
## Consumer contract
`Cloud deployable v1` is a source-and-artifact readiness signal. A deployment
consumer must still resolve and pin the image digest and npm integrity/lockfile,
consumer must still resolve and pin the image digest and migrator integrity/lockfile,
validate migration contents and compatibility, and apply its target health gates.
The check creates no release record and deploys no instance. A full-SHA tag by
itself, or a successful migrator dispatch, is not this readiness signal.
@@ -78,9 +82,16 @@ Do not trust a similarly named check from another workflow or a manual branch ru
Order candidates by master ancestry, not job completion time: an older commit
finishing late must not roll a fleet backward. Fail closed on API errors.
Existing npm canary discovery is unchanged by this producer workflow. Consumers
can adopt the versioned signal separately after the workflow has landed and
successfully verified a real master commit.
Cloud consumers must enable `CLOUD_HARNESS_DIRECT_MIGRATOR_ARTIFACTS` before
this gate is adopted: readiness no longer promises preview npm availability.
The automatic npm-only migrator dispatcher has been removed. Manual
`release.yml` runs with `channel=cloud-migrator`, branch previews, and stable
releases retain their npm publisher for legacy consumers and rollback.
For rollback, restore the npm dispatcher and gate together before disabling the
cloud direct-artifact switch. Already-created releases retain their immutable
archive URLs and lockfiles; keep those objects available. The master producer
can be retried independently without republishing or overwriting a valid bundle.
## Timing and rollout
@@ -151,9 +162,10 @@ its trusted-publisher identity.
Before enabling the switch, deploy the separate Fleet and restrict its GitHub
runner group to repository ID `1170821064` and these workflows at
`refs/heads/master`: `cloud-readiness.yml`, `cloud-artifacts.yml`,
`refs/heads/master`: `cloud-readiness.yml`,
`release-verify.yml`, `runner-chaos-evals.yml`, and `release.yml`. Do not authorize
PR-controlled workflow versions. PR placement retains its independent pinned
PR-controlled workflow versions. The direct migrator producer always uses
GitHub-hosted runners and needs no AWS runner-group authorization. PR placement retains its independent pinned
workflow and six-account author/actor allowlist.
Disable the switch and rerun the whole workflow to restore GitHub-hosted
+6 -4
View File
@@ -170,10 +170,12 @@ The deploy policies are checked in under `.github/cloud-migrator-deploy/`:
There is no lifecycle expiry on this prefix. Keep referenced artifacts for
rollback; deleting them can prevent a fresh migrator install for an old release.
This producer rollout is additive. npm preview publication and the current
cloud readiness gate remain active until the cloud consumer supports the new
manifest. A later cutover must preserve source verification, image identity,
migration compatibility, and the cloud runner's integrity checks.
Cloud readiness consumes the signed direct bundle after its exact-source
publisher succeeds. It retains source verification, image identity, and the
cloud runner's integrity and migration compatibility checks. The cloud direct
artifact switch must be enabled before adopting this gate. Automatic npm-only
migrator dispatch is removed; explicit npm previews and manual migrator runs
remain available. See `doc/cloud-build-readiness.md` for coordinated rollback.
Local verification:
+4 -2
View File
@@ -150,10 +150,12 @@ async function download(url, fetchImpl) {
return Buffer.concat(chunks);
}
export async function verifyPublished(sha, fetchImpl = fetch) {
export async function verifyPublished(sha, fetchImpl = fetch, { verifyProvenance } = {}) {
versionFor(sha);
const manifest = JSON.parse(await download(`${artifactBase}/${sha}/manifest.json`, fetchImpl));
const bytes = await download(`${artifactBase}/${sha}/manifest.json`, fetchImpl);
const manifest = JSON.parse(bytes);
assertManifest(manifest, sha);
if (verifyProvenance) await verifyProvenance(bytes, sha);
await Promise.all(names.map(async (name) => {
const bytes = await download(manifest.packages[name].url, fetchImpl);
verifyBytes(bytes, manifest.packages[name]);
+63 -7
View File
@@ -1,10 +1,64 @@
#!/usr/bin/env node
import { pathToFileURL } from "node:url";
import { imageExists, packageExists, versionFor } from "./preview-artifacts.mjs";
import { execFileSync } from "node:child_process";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { imageExists, versionFor } from "./preview-artifacts.mjs";
import { verifyPublished } from "./cloud-migrator-artifacts.mjs";
const repository = "paperclipai/paperclip";
const workflow = ".github/workflows/cloud-migrator-artifacts.yml";
export async function migratorPublished(sha, fetchImpl, token) {
let pending = false;
const failures = [];
for (let page = 1; page <= 10; page++) {
const response = await fetchImpl(`https://api.github.com/repos/${repository}/actions/workflows/cloud-migrator-artifacts.yml/runs?branch=master&head_sha=${sha}&per_page=100&page=${page}`, {
headers: { Accept: "application/vnd.github+json", ...(token ? { Authorization: `Bearer ${token}` } : {}) },
redirect: "error", signal: AbortSignal.timeout(30_000),
});
if (!response.ok) throw new Error(`Migrator producer lookup failed: HTTP ${response.status}`);
const body = await response.json();
if (!Array.isArray(body.workflow_runs) || !Number.isSafeInteger(body.total_count) || body.total_count < 0 ||
(page === 1 && (body.total_count === 0) !== (body.workflow_runs.length === 0))) throw new Error("Invalid migrator producer response.");
if (body.total_count === 0) return false;
for (const run of body.workflow_runs) {
if (run.head_sha !== sha || run.head_branch !== "master" || run.path !== workflow ||
run.head_repository?.id !== 1170821064 || run.head_repository.full_name !== repository ||
!["push", "workflow_dispatch"].includes(run.event)) throw new Error("Migrator producer identity mismatch.");
// Publication is immutable. A later failed manual run must not hide a
// successful exact-source publisher; the signed bundle is checked next.
if (run.status === "completed" && run.conclusion === "success") return true;
if (run.status !== "completed") pending = true;
else failures.push(`${run.id}: ${run.conclusion}`);
}
if (page * 100 >= body.total_count) {
if (pending) return false;
throw new Error(`Migrator producers failed: ${failures.join(", ")}.`);
}
}
throw new Error("Too many migrator producer runs to establish publication.");
}
export function verifyManifestProvenance(bytes, sha, { exec = execFileSync } = {}) {
versionFor(sha);
const scratch = mkdtempSync(path.join(os.tmpdir(), "cloud-readiness-attestation-"));
try {
const file = path.join(scratch, "manifest.json");
writeFileSync(file, bytes);
exec("gh", ["attestation", "verify", file, "--repo", repository,
"--source-digest", sha, "--source-ref", "refs/heads/master",
"--cert-identity", `https://github.com/${repository}/${workflow}@refs/heads/master`,
"--deny-self-hosted-runners"], { stdio: "inherit", timeout: 60_000 });
} finally { rmSync(scratch, { recursive: true, force: true }); }
}
/** Read-only availability gate. Deployment still resolves and pins artifacts. */
export async function waitForCloudArtifacts(sha, {
fetchImpl = fetch,
token = process.env.GH_TOKEN,
verifyProvenance = verifyManifestProvenance,
now = () => performance.now(),
sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
timeoutMs = 30 * 60_000,
@@ -17,17 +71,19 @@ export async function waitForCloudArtifacts(sha, {
}
const deadline = now() + timeoutMs;
let previous;
let missing = ["image", "shared", "db"];
let missing = ["image", "migrator"];
while (now() < deadline) {
// Recheck every artifact on the successful poll. Only an explicit 404
// means publication is pending; identity errors and upstream outages fail.
// Recheck the image and exact-source publisher on the successful poll.
// Only a missing/pending producer waits; failed publication fails closed.
const results = await Promise.all([
imageExists(sha, fetchImpl),
packageExists("@paperclipai/shared", sha, fetchImpl),
packageExists("@paperclipai/db", sha, fetchImpl),
migratorPublished(sha, fetchImpl, token),
]);
missing = ["image", "shared", "db"].filter((_, index) => !results[index]);
missing = ["image", "migrator"].filter((_, index) => !results[index]);
if (missing.length === 0) {
// Verify the exact signed bytes and all pinned downloads after the
// publisher succeeds. An inaccessible or corrupt artifact cannot pass.
await verifyPublished(sha, fetchImpl, { verifyProvenance });
log(`Cloud artifacts available for ${sha}: verified image and exact-source migrator ${version}.`);
return { version: 1, sha, packageVersion: version };
}
+1 -8
View File
@@ -170,15 +170,8 @@ test("preview workflow separates branch compilation from trusted publishing", ()
assert.match(workflow, /Stack deploy \{0\} build/);
});
test("merge dispatch uses the existing publisher outside full-release concurrency without claiming image readiness", () => {
const dispatcher = readFileSync(new URL("../.github/workflows/cloud-artifacts.yml", import.meta.url), "utf8");
test("manual migrator and branch preview retain their npm publisher and concurrency", () => {
const release = readFileSync(new URL("../.github/workflows/release.yml", import.meta.url), "utf8");
assert.match(dispatcher, /branches: \[master\]/);
assert.match(dispatcher, /github.ref == 'refs\/heads\/master'/);
assert.match(dispatcher, /SOURCE_SHA: \$\{\{ github.sha \}\}/);
assert.match(dispatcher, /gh workflow run release.yml .*--ref master/);
assert.match(dispatcher, /--field channel=cloud-migrator/);
assert.doesNotMatch(dispatcher, /actions\/checkout|id-token: write|packages: write|secrets\./);
assert.match(release, /\(inputs.channel == 'preview' \|\| inputs.channel == 'cloud-migrator'\) && format\('\{0\}-\{1\}', inputs.channel, inputs.source_ref\)/);
const publisher = release.split(" publish_preview:")[1].split(" image_preview:")[0];
assert.match(publisher, /group: preview-package-publish-\$\{\{ inputs.source_ref \}\}/);