Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-06 20:34:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(ci): remove npm propagation from cloud readiness (#13456)

Browse Source 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud needs a verified image and matching database
migrator before it can deploy a merge.
> - New npm package versions can take minutes to become downloadable
after the package build finishes.
> - The direct producer now publishes signed archives and a complete
dependency lockfile for each master commit.
> - This pull request makes readiness verify those artifacts and removes
the duplicate automatic npm migrator run.
> - Deployment still requires all source checks, exact image identity,
migration compatibility, and pinned dependencies.

## Linked Issues or Issue Description

Refs: #13455, #13454, #13192

**What existing behavior does this improve?**
The time from a master merge to the `Cloud deployable v1` signal.

**Current behavior**
Readiness polls npm metadata for the new DB and shared versions. An
automatic dispatcher also starts a separate npm-only migrator workflow.
A measured source built its packages at 06:22:41 UTC on 2026-09-15, but
both npm archives were not downloadable until 06:31:56 UTC.

**Proposed behavior**
Wait for the successful exact-source direct producer, verify its signed
manifest and all pinned downloads, and publish readiness only after the
existing source and image jobs pass. Keep manual npm migrators and
branch previews available.

**Reason and benefit**
Remove new-version npm propagation from merge-to-deployable time. The
gain depends on whether image building or source verification finishes
later; it is not a fixed subtraction from every run.

## What Changed

- Require a successful producer from the canonical repository, exact
commit, master ref, expected workflow, and approved event.
- Verify the manifest's GitHub attestation with the hosted GitHub CLI.
Enforce the exact source SHA, master workflow identity, and hosted
runner.
- Download and validate both archives and the complete dependency
lockfile after publication succeeds. Reject invalid signatures,
inaccessible objects, corrupt bytes, and source mismatches.
- Remove automatic npm-only migrator dispatch. Retain manual release and
branch-preview publication.
- Document the cloud feature-switch prerequisite and coordinated
rollback.

## Verification

- `node --test .github/scripts/tests/*.test.mjs`: 405 pass.
- Focused readiness, routing, preview, and artifact tests: 249 pass.
- Workflow lint and `git diff --check`: pass.
- `pnpm test:release-registry`: 139 pass after installing this
worktree's dependencies.
- All latest-head GitHub CI checks passed. Greptile is 5/5 with no
unresolved comments.
- Application source is unchanged. Common-source local typecheck and
build passed. The full local application suite has the documented macOS
read-only-directory rename limitation from #13454 (13 failures in two
unchanged suites); Linux CI is the final application gate.
- Live readiness verification of master
da77a0c28c passed in 6.52 seconds,
including the real GitHub CLI signature policy and all artifact
downloads.
- Cloud consumer resolution with the certificate encoding fix passed in
5.45 seconds with zero npm metadata requests or npm processes. The
consumer is deployed and enabled in staging and production; their live
resolution APIs passed in 2.34 and 2.38 seconds. Both report the
expected fixed harness commit. A fresh tenant deployment follows this
cutover merge.

## Risks

- `Cloud deployable v1` no longer promises npm preview availability.
Enable the cloud direct-artifact consumer in staging and production
before merging this change.
- Artifact storage and GitHub attestations become required services for
new direct releases. Missing or invalid evidence fails explicitly.
- Restore the old npm dispatcher and readiness gate together before
disabling the consumer switch. Retain artifacts referenced by existing
releases.
- This change does not expand AWS runner access. The producer and
readiness bookkeeping use GitHub-hosted runners. Existing PR allowlists
and source verification gates remain enforced.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused checks; full
application host limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin FoleyandPaperclip authored and GitHub committed 2026-09-15 01:14:43 -07:00
1 parent 9ed55f6931
commit 4cc387f907
9 files changed
+211 -122
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/scripts/tests/cloud-readiness.test.mjs
+102 -49
View File
@@ -1,20 +1,47 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { waitForCloudArtifacts } from "../../../scripts/cloud-readiness.mjs";
import { existsSync, readFileSync } from "node:fs";
import { gzipSync } from "node:zlib";
import { waitForCloudArtifacts, verifyManifestProvenance, migratorPublished } from "../../../scripts/cloud-readiness.mjs";
import { artifactBase, descriptor } from "../../../scripts/cloud-migrator-artifacts.mjs";
import { previewManifest } from "../../../scripts/preview-artifacts.mjs";
const sha = "a".repeat(40);
const version = `0.0.0-preview.g${sha}`;
const digest = `sha256:${"b".repeat(64)}`;
const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
function registry({ missing = new Set(), failure, wrongImage = false, wrongPackage = false } = {}) {
return async (url) => {
const producer = { id: 123, head_sha: sha, head_branch: "master", path: ".github/workflows/cloud-migrator-artifacts.yml",
head_repository: { id: 1170821064, full_name: "paperclipai/paperclip" }, event: "push", status: "completed", conclusion: "success" };
function bundle() {
const packages = {}; const files = new Map();
const entries = { "": { dependencies: { "@paperclipai/db": version } } };
for (const name of ["db", "shared"]) {
const metadata = previewManifest({ name: `@paperclipai/${name}`, dependencies: {} }, sha);
const bytes = Buffer.from(JSON.stringify(metadata));
const header = Buffer.alloc(512); header.write("package/package.json"); header.write(bytes.length.toString(8).padStart(11, "0"), 124, 11); header[156] = 48;
const padded = Buffer.alloc(Math.ceil(bytes.length / 512) * 512); bytes.copy(padded);
const archive = gzipSync(Buffer.concat([header, padded, Buffer.alloc(1024)]));
const pin = descriptor(archive, "tgz"); packages[name] = pin; files.set(pin.url, archive);
entries[`node_modules/@paperclipai/${name}`] = { version, resolved: pin.url, integrity: pin.integrity, dependencies: metadata.dependencies };
}
const lock = Buffer.from(JSON.stringify({ lockfileVersion: 3, packages: entries }));
const manifest = { version: 1, sourceSha: sha, packageVersion: version, packages, lockfile: descriptor(lock, "json") };
files.set(manifest.lockfile.url, lock);
const bytes = Buffer.from(JSON.stringify(manifest) + "\n");
files.set(`${artifactBase}/${sha}/manifest.json`, bytes);
return { manifest, bytes, files };
}
function registry({ missing = new Set(), failure, wrongImage = false, run = producer, objects = bundle() } = {}) {
return async (url, options) => {
assert.ok(!url.startsWith("https://registry.npmjs.org/"), "readiness must never wait for npm");
if (failure) return json({}, failure);
if (url.startsWith("https://registry.npmjs.org/")) {
const name = decodeURIComponent(new URL(url).pathname.split("/")[1]);
if (missing.has(name.split("/")[1])) return json({}, 404);
const pkg = previewManifest({ name, version: "0.0.0" }, sha);
return json({ ...pkg, ...(wrongPackage ? { gitHead: "c".repeat(40) } : {}), dist: { integrity: "sha512-fixture", tarball: "https://registry.npmjs.org/fixture.tgz" } });
if (url.startsWith("https://api.github.com/")) {
assert.match(url, new RegExp(`head_sha=${sha}&per_page=100&page=1$`));
return json({ total_count: missing.has("migrator") ? 0 : 1, workflow_runs: missing.has("migrator") ? [] : [run] });
}
if (url.startsWith(artifactBase)) {
assert.equal(options.headers?.Authorization, undefined, "GitHub credentials stay off the artifact origin");
return objects.files.has(url) ? new Response(objects.files.get(url)) : json({}, 403);
}
if (url.includes("/token?")) return json({ token: "fixture" });
if (url.includes("/manifests/")) return missing.has("image") ? json({}, 404) : json({ config: { digest } });
@@ -22,70 +49,96 @@ function registry({ missing = new Set(), failure, wrongImage = false, wrongPacka
throw new Error(`Unexpected request: ${url}`);
};
}
const noSignature = async () => {}; // Signature enforcement is exercised separately below.
test("readiness requires the image and both exact-source packages on the successful poll", async () => {
const missing = new Set(["image", "shared", "db"]);
let clock = 0;
const states = [];
test("readiness rechecks image and publisher, then verifies the exact signed bundle with no npm requests", async () => {
const missing = new Set(["image", "migrator"]); const objects = bundle(); let clock = 0; let signatures = 0;
const result = await waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing }), now: () => clock, intervalMs: 10, timeoutMs: 100, log: (message) => states.push(message),
fetchImpl: registry({ missing, objects }), token: "fixture", now: () => clock, intervalMs: 10, timeoutMs: 100, log: () => {},
verifyProvenance: async (bytes, source) => { assert.deepEqual(bytes, objects.bytes); assert.equal(source, sha); signatures++; },
sleep: async (ms) => {
clock += ms;
if (clock === 10) missing.delete("image");
if (clock === 20) missing.delete("shared");
if (clock === 30) { missing.delete("db"); missing.add("image"); }
if (clock === 40) missing.delete("image");
if (clock === 20) { missing.delete("migrator"); missing.add("image"); }
if (clock === 30) missing.delete("image");
},
});
assert.equal(clock, 40, "an artifact disappearing before the final poll must prevent readiness");
assert.deepEqual(result, { version: 1, sha, packageVersion: `0.0.0-preview.g${sha}` });
assert.match(states.at(-1), /Cloud artifacts available/);
assert.equal(clock, 30); assert.equal(signatures, 1);
assert.deepEqual(result, { version: 1, sha, packageVersion: version });
});
test("missing artifacts time out with a precise inventory and bounded sleep", async () => {
let clock = 0;
const sleeps = [];
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing: new Set(["db"]) }), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {},
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
}), /timed out.*missing: db/);
assert.deepEqual(sleeps, [20, 5]);
});
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true }, { wrongPackage: true }]) {
test(`registry errors and identity mismatches fail without waiting: ${JSON.stringify(fixture)}`, async () => {
test("missing or in-progress publishers time out with a precise inventory and bounded sleep", async () => {
for (const fixture of [{ missing: new Set(["migrator"]) }, { run: { ...producer, status: "in_progress", conclusion: null } }]) {
let clock = 0; const sleeps = [];
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry(fixture), sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {},
}));
fetchImpl: registry(fixture), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {}, verifyProvenance: noSignature,
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
}), /timed out.*missing: migrator/);
assert.deepEqual(sleeps, [20, 5]);
}
});
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true },
...["failure", "cancelled", "skipped"].map((conclusion) => ({ run: { ...producer, conclusion } })),
...[{ head_sha: "b".repeat(40) }, { head_branch: "feature" }, { path: ".github/workflows/evil.yml" },
{ head_repository: { id: 123, full_name: "someone/paperclip" } }, { event: "pull_request" }].map((wrong) => ({ run: { ...producer, ...wrong } }))]) {
test(`upstream errors, failed publication and identity mismatches fail immediately: ${JSON.stringify(fixture)}`, async () => {
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(fixture), verifyProvenance: noSignature,
sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {} }));
});
}
test("successful publication cannot hide inaccessible or corrupt archives or an invalid signature", async () => {
for (const corrupt of [false, true]) {
const objects = bundle();
if (corrupt) objects.files.set(objects.manifest.packages.db.url, Buffer.from("corrupt"));
else objects.files.delete(objects.manifest.packages.db.url);
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry({ objects }), verifyProvenance: noSignature, log: () => {} }), /download failed|immutable pin/);
}
await assert.rejects(waitForCloudArtifacts(sha, { fetchImpl: registry(), verifyProvenance: async () => { throw new Error("invalid signature"); }, log: () => {} }), /invalid signature/);
});
test("CLI verifies the exact bytes, source, master workflow and hosted runner and cleans up on failure", () => {
let temporary;
assert.throws(() => verifyManifestProvenance(Buffer.from("exact manifest\n"), sha, { exec: (cmd, args) => {
assert.equal(cmd, "gh"); assert.deepEqual(args.slice(0, 2), ["attestation", "verify"]); temporary = args[2];
assert.equal(readFileSync(temporary, "utf8"), "exact manifest\n");
for (const [flag, value] of [["--repo", "paperclipai/paperclip"], ["--source-digest", sha], ["--source-ref", "refs/heads/master"],
["--cert-identity", "https://github.com/paperclipai/paperclip/.github/workflows/cloud-migrator-artifacts.yml@refs/heads/master"]]) assert.equal(args[args.indexOf(flag) + 1], value);
assert.ok(args.includes("--deny-self-hosted-runners")); throw new Error("verification rejected");
} }), /verification rejected/);
assert.equal(existsSync(temporary), false);
});
test("invalid source and timing configuration are rejected before registry access", async () => {
const fetchImpl = async () => assert.fail("invalid inputs must not reach a registry");
await assert.rejects(waitForCloudArtifacts("master", { fetchImpl }), /full immutable commit SHA/);
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) {
await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
}
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
});
test("the versioned readiness job requires successful source, image and artifact jobs", () => {
test("versioned readiness retains every source gate and removes duplicate automatic npm publication", () => {
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
assert.match(workflow, /push:\s*\n\s*branches: \[master\]/);
assert.match(workflow, /group: cloud-readiness-\$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/release-verify.yml\s+with:\s+ref: \$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/docker-cloud.yml/);
assert.match(workflow, /attestations: read/); assert.match(workflow, /GH_TOKEN: \$\{\{ github.token \}\}/);
const ready = workflow.split(" ready:")[1];
assert.match(ready, /name: Cloud deployable v1/);
assert.match(ready, /needs: \[verify, image, artifacts\]/);
assert.match(ready, /name: Cloud deployable v1/); assert.match(ready, /needs: \[verify, image, artifacts\]/);
assert.match(ready, /if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'/);
assert.doesNotMatch(ready, /^\s*(?:if:.*always\(|continue-on-error:)/m);
assert.doesNotMatch(workflow, /secrets: inherit|id-token: write|actions: write|checks: write|uses: .*@v\d\b/);
const cloud = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
assert.doesNotMatch(cloud, /^ push:/m, "the master image must build only once");
const migrator = readFileSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url), "utf8");
assert.match(migrator, /push:\s*\n\s*branches: \[master\]/);
assert.match(migrator, /SOURCE_SHA: \$\{\{ github.sha \}\}/);
assert.match(migrator, /gh workflow run release.yml .*--ref master/);
assert.match(migrator, /--field channel=cloud-migrator/);
assert.match(migrator, /--field source_ref="\$SOURCE_SHA"/);
assert.equal(existsSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url)), false);
});
test("later manual failures or pending retries cannot hide an earlier successful immutable publication", async () => {
for (const latest of [{ status: "completed", conclusion: "failure" }, { status: "in_progress", conclusion: null }]) {
let calls = 0;
assert.equal(await migratorPublished(sha, async (url) => {
calls++;
if (url.endsWith("page=1")) return json({ total_count: 101, workflow_runs: Array.from({ length: 100 }, (_, i) => ({ ...producer, ...latest, id: 200 + i, event: "workflow_dispatch" })) });
assert.ok(url.endsWith("page=2")); return json({ total_count: 101, workflow_runs: [producer] });
}), true);
assert.equal(calls, 2);
}
});
.github/scripts/tests/post-merge-runner-routing.test.mjs
+1 -1
View File
@@ -11,7 +11,7 @@ const base = {
};
const expectedJobs = {
"cloud-readiness.yml": [],
"cloud-artifacts.yml": ["dispatch_migrator"],
"cloud-migrator-artifacts.yml": [],
"release-verify.yml": ["typecheck", "general_tests", "serialized_tests", "runner_workflow_evals", "verify_paperclip_runner", "build"],
"runner-chaos-evals.yml": ["chaos_and_recovery"],
"release.yml": ["plan_preview", "package_preview"],