mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every master push runs a canary, and only one canary at a time may run the release workflow > - After #21554 and #21555 a canary held that lock for 35.5 min, and 13-16 min of it was waiting for npm to show the new packages > - The wait was required: the next canary computed `canary.N` from npm's version list, so starting before npm showed the previous canary would reuse its number > - This pull request numbers a canary by its commit's position on master and moves the registry confirmation, the git tag, the Docker dispatch and the onboarding smoke to a separate `canary-finalize.yml` run > - The benefit is that the release lock is held for about 20-25 min per canary instead of 35, with the same checks before a canary is tagged ## Linked Issues or Issue Description **What existing behavior does this improve?** Canary throughput. In run 43715037497 (2026-11-15) `publish_canary` took 23.7 min, and 13.3 min of that was "Waiting for registry visibility" (13:08:50 to 13:22:11). The next canary could not start until that run ended. **Current behavior** `release.sh canary` sets `N` to (highest published `-canary.N` for today) + 1, then publishes, waits for npm to show every package, checks dist-tags, installs `paperclipai@canary`, and creates `canary/v<version>`. `publish_canary` pushes the tag, dispatches `docker.yml`, and `smoke_canary_onboarding` runs in the same run. **Proposed behavior** `N` = `git rev-list --count HEAD` (full history required), or the registry number + 1 when that is higher (a rerun of the same commit). `release.sh canary --defer-registry-confirmation` ends after npm accepts every publish. `publish_canary` dispatches `canary-finalize.yml`, which does the confirmation, tag, Docker dispatch and smoke outside the release lock. Nightly, beta and stable numbering and flow are unchanged. Refs #21554, #21555 ## What Changed - `scripts/release-lib.sh`: `next_canary_version_from_source`. A publishing canary refuses a shallow checkout; a dry run (the PR Canary Dry Run job) falls back to the registry number. - `scripts/release.sh`: canary uses the new number; new canary-only `--defer-registry-confirmation` flag skips the registry wait, the dist-tag check and the local tag, and writes the version to `PAPERCLIP_RELEASE_VERSION_FILE`. - `scripts/confirm-canary-publication.sh` (new): waits for registry visibility, checks provenance, checks dist-tags with `--allow-newer-canary`, installs `paperclipai@<version>`, then creates and pushes `canary/v<version>`. A tag already on the same commit is accepted, so reruns are safe. - `scripts/verify-canary-provenance.mjs` (new): every package's npm SLSA provenance must name `.github/workflows/release.yml` on `refs/heads/master` in `paperclipai/paperclip` at the source commit. A dispatch with invented inputs cannot tag anything. - `scripts/verify-release-registry-state.mjs`: `--allow-newer-canary` accepts a `canary` dist-tag that a newer canary already moved; older, foreign or missing tags still fail. - `.github/workflows/canary-finalize.yml` (new, `workflow_dispatch`, master only): `confirm_canary` (contents: write, actions: write) and `smoke_canary_onboarding` (downloads the publisher's lockfile artifact by run id). Inputs reach shell steps only through a validating step. - `.github/workflows/release.yml`: `publish_canary` hands off to `canary-finalize.yml` and drops to `contents: read`; its tag push, Docker dispatch and smoke job move to `canary-finalize.yml`. - Tests and `doc/RELEASING.md`. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/release-lib.test.mjs scripts/verify-release-registry-state.test.mjs scripts/verify-canary-provenance.test.mjs`: 846 pass. `scripts/__tests__/release-verify-workflow.test.mjs`: 73 pass. - New tests: numbering by master position, rerun moves past a published number, shallow refusal and dry-run fallback; newer/older/foreign dist-tags; provenance from another commit, workflow, branch, repository or package is rejected; finalize wiring, permissions and input handling. - Real npm data: `verify-canary-provenance.mjs 2026.1116.0-canary.7 b4fa1c7f... paperclipai @paperclipai/shared @paperclipai/server` passes; the same version with another commit fails. - `confirm-canary-publication.sh 2026.1116.0-canary.7 b4fa1c7f...` from a checkout of that commit ran all steps against npm in 61 s and ended with "canary/v2026.1116.0-canary.7 already points at b4fa1c7f...", pushing nothing. - `release.sh canary --dry-run --defer-registry-confirmation` and `release.sh nightly --defer-registry-confirmation` both stop with "only applies to a published canary". - `actionlint` reports no new findings compared with master. - Not verified before merge: a full publish through `canary-finalize.yml`. It can run only from master. The first canary after merge is the check. ## Risks - Canary numbers jump from `canary.7` to about `canary.8144` (master has 8,143 commits). This is valid semver and still higher than every earlier canary; users install through the `canary` dist-tag. - The `canary/v` tag, Docker image and smoke now finish after the release run ends. Nightly selects only tagged canaries, as before. If finalize fails, that canary is on npm without a tag, so nightly skips it, as it skips any canary whose old in-run verification failed. - A canary can now publish while the previous one is still being confirmed. Publishing itself is still serialized, so the `canary` dist-tag only moves forward. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
427 lines
12 KiB
JavaScript
427 lines
12 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import test from "node:test";
|
|
|
|
import {
|
|
collectInternalDependencyProblems,
|
|
createManifestLookupKey,
|
|
fetchRegistryJson,
|
|
isCanaryVersion,
|
|
isPrereleaseVersion,
|
|
verifyPackageRegistryProblems,
|
|
verifyPackageRegistryState,
|
|
compareCanaryVersions,
|
|
} from "./verify-release-registry-state.mjs";
|
|
|
|
test("isCanaryVersion matches release canaries", () => {
|
|
assert.equal(isCanaryVersion("2026.427.0-canary.3"), true);
|
|
assert.equal(isCanaryVersion("2026.427.0"), false);
|
|
});
|
|
|
|
test("isPrereleaseVersion matches canary and nightly versions", () => {
|
|
assert.equal(isPrereleaseVersion("2026.427.0-canary.3"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0-nightly.0"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0-beta.2"), true);
|
|
assert.equal(isPrereleaseVersion("2026.427.0"), false);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems flags missing internal versions", () => {
|
|
const manifest = {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
e2b: "^2.19.0",
|
|
},
|
|
};
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, packageDocsByName),
|
|
["dependencies requires @paperclipai/plugin-sdk@2026.425.0-canary.5, but npm does not expose that version"],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems accepts version-specific manifests when the root document is stale", () => {
|
|
const manifest = {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
},
|
|
};
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/plugin-sdk", "2026.425.0-canary.5"),
|
|
{ name: "@paperclipai/plugin-sdk", version: "2026.425.0-canary.5" },
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, packageDocsByName, packageManifestsByKey),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems ignores peer dependency range specifiers", () => {
|
|
const manifest = {
|
|
peerDependencies: {
|
|
"@paperclipai/server": "^2026.430.0-canary.0",
|
|
},
|
|
};
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, new Map()),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("collectInternalDependencyProblems reports unfetched transitive dependency metadata neutrally", () => {
|
|
const manifest = {
|
|
optionalDependencies: {
|
|
"@paperclipai/browser": "2026.430.0-canary.0",
|
|
},
|
|
};
|
|
|
|
assert.deepEqual(
|
|
collectInternalDependencyProblems(manifest, new Map()),
|
|
[
|
|
"optionalDependencies requires @paperclipai/browser@2026.430.0-canary.0, but npm publication metadata was not fetched for that dependency",
|
|
],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState tolerates a stale root versions map when dist-tags and direct manifests are correct", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/ui",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.430.0-canary.0",
|
|
latest: "2026.430.0",
|
|
},
|
|
versions: {},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/shared",
|
|
{
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/ui", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/ui",
|
|
version: "2026.430.0-canary.0",
|
|
dependencies: {
|
|
"@paperclipai/shared": "2026.430.0-canary.0",
|
|
},
|
|
},
|
|
],
|
|
[
|
|
createManifestLookupKey("@paperclipai/shared", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/shared",
|
|
version: "2026.430.0-canary.0",
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/ui",
|
|
packageDoc: packageDocsByName.get("@paperclipai/ui"),
|
|
packageDocsByName,
|
|
packageManifestsByKey,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState fails when canary latest is left in place by default", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-canary.5",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.425.0-canary.5": {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.425.0-canary.5",
|
|
},
|
|
},
|
|
"2026.427.0-canary.3": {
|
|
dependencies: {
|
|
"@paperclipai/plugin-sdk": "2026.427.0-canary.3",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[
|
|
"@paperclipai/plugin-e2b: latest dist-tag still resolves to prerelease 2026.425.0-canary.5; if that state is intentional, rerun the verification script directly with --allow-canary-latest",
|
|
"@paperclipai/plugin-e2b@2026.425.0-canary.5 via latest: dependencies requires @paperclipai/plugin-sdk@2026.425.0-canary.5, but npm does not expose that version",
|
|
],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryProblems marks canary latest drift as non-retriable", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-canary.5",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
const problems = verifyPackageRegistryProblems({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: false,
|
|
});
|
|
|
|
assert.equal(problems[0]?.retriable, false);
|
|
assert.match(problems[0]?.message ?? "", /latest dist-tag still resolves to prerelease/);
|
|
});
|
|
|
|
test("verifyPackageRegistryProblems accepts the nightly channel and flags nightly latest drift", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-e2b",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.425.0-nightly.1",
|
|
nightly: "2026.427.0-nightly.0",
|
|
},
|
|
versions: {
|
|
"2026.427.0-nightly.0": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
const problems = verifyPackageRegistryProblems({
|
|
packageName: "@paperclipai/plugin-e2b",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-e2b"),
|
|
packageDocsByName,
|
|
channel: "nightly",
|
|
distTag: "nightly",
|
|
targetVersion: "2026.427.0-nightly.0",
|
|
allowCanaryLatest: false,
|
|
});
|
|
|
|
assert.equal(problems[0]?.retriable, false);
|
|
assert.match(problems[0]?.message ?? "", /latest dist-tag still resolves to prerelease 2026\.425\.0-nightly\.1/);
|
|
});
|
|
|
|
test("verifyPackageRegistryState allows intentional canary latest but still checks dependencies", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"paperclipai",
|
|
{
|
|
"dist-tags": {
|
|
latest: "2026.427.0-canary.3",
|
|
canary: "2026.427.0-canary.3",
|
|
},
|
|
versions: {
|
|
"2026.427.0-canary.3": {
|
|
dependencies: {
|
|
"@paperclipai/server": "2026.427.0-canary.3",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
[
|
|
"@paperclipai/server",
|
|
{
|
|
versions: {
|
|
"2026.427.0-canary.3": {},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "paperclipai",
|
|
packageDoc: packageDocsByName.get("paperclipai"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.427.0-canary.3",
|
|
allowCanaryLatest: true,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState still fails when the dist-tag is stale", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/ui",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.429.0-canary.2",
|
|
},
|
|
versions: {},
|
|
},
|
|
],
|
|
]);
|
|
const packageManifestsByKey = new Map([
|
|
[
|
|
createManifestLookupKey("@paperclipai/ui", "2026.430.0-canary.0"),
|
|
{
|
|
name: "@paperclipai/ui",
|
|
version: "2026.430.0-canary.0",
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/ui",
|
|
packageDoc: packageDocsByName.get("@paperclipai/ui"),
|
|
packageDocsByName,
|
|
packageManifestsByKey,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
["@paperclipai/ui: dist-tag canary resolves to 2026.429.0-canary.2, expected 2026.430.0-canary.0"],
|
|
);
|
|
});
|
|
|
|
test("verifyPackageRegistryState ignores internal peer dependency ranges", () => {
|
|
const packageDocsByName = new Map([
|
|
[
|
|
"@paperclipai/plugin-sdk",
|
|
{
|
|
"dist-tags": {
|
|
canary: "2026.430.0-canary.0",
|
|
},
|
|
versions: {
|
|
"2026.430.0-canary.0": {
|
|
peerDependencies: {
|
|
"@paperclipai/server": "^2026.430.0-canary.0",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
],
|
|
]);
|
|
|
|
assert.deepEqual(
|
|
verifyPackageRegistryState({
|
|
packageName: "@paperclipai/plugin-sdk",
|
|
packageDoc: packageDocsByName.get("@paperclipai/plugin-sdk"),
|
|
packageDocsByName,
|
|
channel: "canary",
|
|
distTag: "canary",
|
|
targetVersion: "2026.430.0-canary.0",
|
|
allowCanaryLatest: false,
|
|
}),
|
|
[],
|
|
);
|
|
});
|
|
|
|
test("fetchRegistryJson times out hung requests", async () => {
|
|
const originalFetch = globalThis.fetch;
|
|
|
|
globalThis.fetch = (_url, { signal }) =>
|
|
new Promise((_resolve, reject) => {
|
|
signal.addEventListener(
|
|
"abort",
|
|
() => reject(new DOMException("The operation was aborted.", "AbortError")),
|
|
{ once: true },
|
|
);
|
|
});
|
|
|
|
try {
|
|
await assert.rejects(
|
|
fetchRegistryJson(new URL("https://registry.npmjs.org/@paperclipai%2Fui"), { timeoutMs: 1 }),
|
|
/timed out/,
|
|
);
|
|
} finally {
|
|
globalThis.fetch = originalFetch;
|
|
}
|
|
});
|
|
|
|
test("a finalizing canary accepts a dist-tag that a newer canary already moved, and nothing older or foreign", () => {
|
|
const check = (canaryTag, allowNewerCanary, channel = "canary") => {
|
|
const packageDoc = { "dist-tags": { latest: "2026.1110.0", canary: canaryTag }, versions: { "2026.1116.0-canary.21800": {} } };
|
|
return verifyPackageRegistryProblems({
|
|
packageName: "@paperclipai/shared", packageDoc, packageDocsByName: new Map([["@paperclipai/shared", packageDoc]]),
|
|
channel, distTag: "canary", targetVersion: "2026.1116.0-canary.21800", allowCanaryLatest: false, allowNewerCanary,
|
|
}).map((problem) => problem.message);
|
|
};
|
|
assert.deepEqual(check("2026.1116.0-canary.21800", true), []);
|
|
assert.deepEqual(check("2026.1116.0-canary.21805", true), []);
|
|
assert.deepEqual(check("2026.1117.0-canary.21803", true), []);
|
|
assert.match(check("2026.1116.0-canary.21805", false)[0], /resolves to 2026\.1116\.0-canary\.21805, expected/);
|
|
assert.match(check("2026.1116.0-canary.21799", true)[0], /resolves to 2026\.1116\.0-canary\.21799, expected/);
|
|
assert.match(check("2026.1116.0-nightly.3", true)[0], /resolves to 2026\.1116\.0-nightly\.3, expected/);
|
|
assert.match(check(undefined, true)[0], /resolves to <missing>/);
|
|
});
|
|
|
|
test("canary versions compare by base version, then canary number", () => {
|
|
assert.equal(compareCanaryVersions("2026.1116.0-canary.9", "2026.1116.0-canary.21800"), -1);
|
|
assert.equal(compareCanaryVersions("2026.1117.0-canary.1", "2026.1116.0-canary.21800"), 1);
|
|
assert.equal(compareCanaryVersions("2026.1116.0-canary.7", "2026.1116.0-canary.7"), 0);
|
|
assert.equal(compareCanaryVersions("2026.1116.0-nightly.7", "2026.1116.0-canary.7"), null);
|
|
});
|