Files
PaperClipAI/scripts/verify-release-registry-state.mjs
T
DottaandClaude Opus 5.5 d16c06308d perf(release): number canaries by master position and confirm them outside the release lock (#21917)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every master push runs a canary, and only one canary at a time may
run the release workflow
> - After #21554 and #21555 a canary held that lock for 35.5 min, and
13-16 min of it was waiting for npm to show the new packages
> - The wait was required: the next canary computed `canary.N` from
npm's version list, so starting before npm showed the previous canary
would reuse its number
> - This pull request numbers a canary by its commit's position on
master and moves the registry confirmation, the git tag, the Docker
dispatch and the onboarding smoke to a separate `canary-finalize.yml`
run
> - The benefit is that the release lock is held for about 20-25 min per
canary instead of 35, with the same checks before a canary is tagged

## Linked Issues or Issue Description

**What existing behavior does this improve?**
Canary throughput. In run 43715037497 (2026-11-15) `publish_canary` took
23.7 min, and 13.3 min of that was "Waiting for registry visibility"
(13:08:50 to 13:22:11). The next canary could not start until that run
ended.

**Current behavior**
`release.sh canary` sets `N` to (highest published `-canary.N` for
today) + 1, then publishes, waits for npm to show every package, checks
dist-tags, installs `paperclipai@canary`, and creates
`canary/v<version>`. `publish_canary` pushes the tag, dispatches
`docker.yml`, and `smoke_canary_onboarding` runs in the same run.

**Proposed behavior**
`N` = `git rev-list --count HEAD` (full history required), or the
registry number + 1 when that is higher (a rerun of the same commit).
`release.sh canary --defer-registry-confirmation` ends after npm accepts
every publish. `publish_canary` dispatches `canary-finalize.yml`, which
does the confirmation, tag, Docker dispatch and smoke outside the
release lock. Nightly, beta and stable numbering and flow are unchanged.

Refs #21554, #21555

## What Changed

- `scripts/release-lib.sh`: `next_canary_version_from_source`. A
publishing canary refuses a shallow checkout; a dry run (the PR Canary
Dry Run job) falls back to the registry number.
- `scripts/release.sh`: canary uses the new number; new canary-only
`--defer-registry-confirmation` flag skips the registry wait, the
dist-tag check and the local tag, and writes the version to
`PAPERCLIP_RELEASE_VERSION_FILE`.
- `scripts/confirm-canary-publication.sh` (new): waits for registry
visibility, checks provenance, checks dist-tags with
`--allow-newer-canary`, installs `paperclipai@<version>`, then creates
and pushes `canary/v<version>`. A tag already on the same commit is
accepted, so reruns are safe.
- `scripts/verify-canary-provenance.mjs` (new): every package's npm SLSA
provenance must name `.github/workflows/release.yml` on
`refs/heads/master` in `paperclipai/paperclip` at the source commit. A
dispatch with invented inputs cannot tag anything.
- `scripts/verify-release-registry-state.mjs`: `--allow-newer-canary`
accepts a `canary` dist-tag that a newer canary already moved; older,
foreign or missing tags still fail.
- `.github/workflows/canary-finalize.yml` (new, `workflow_dispatch`,
master only): `confirm_canary` (contents: write, actions: write) and
`smoke_canary_onboarding` (downloads the publisher's lockfile artifact
by run id). Inputs reach shell steps only through a validating step.
- `.github/workflows/release.yml`: `publish_canary` hands off to
`canary-finalize.yml` and drops to `contents: read`; its tag push,
Docker dispatch and smoke job move to `canary-finalize.yml`.
- Tests and `doc/RELEASING.md`.

## Verification

- `node --test .github/scripts/tests/*.test.mjs
scripts/release-lib.test.mjs
scripts/verify-release-registry-state.test.mjs
scripts/verify-canary-provenance.test.mjs`: 846 pass.
`scripts/__tests__/release-verify-workflow.test.mjs`: 73 pass.
- New tests: numbering by master position, rerun moves past a published
number, shallow refusal and dry-run fallback; newer/older/foreign
dist-tags; provenance from another commit, workflow, branch, repository
or package is rejected; finalize wiring, permissions and input handling.
- Real npm data: `verify-canary-provenance.mjs 2026.1116.0-canary.7
b4fa1c7f... paperclipai @paperclipai/shared @paperclipai/server` passes;
the same version with another commit fails.
- `confirm-canary-publication.sh 2026.1116.0-canary.7 b4fa1c7f...` from
a checkout of that commit ran all steps against npm in 61 s and ended
with "canary/v2026.1116.0-canary.7 already points at b4fa1c7f...",
pushing nothing.
- `release.sh canary --dry-run --defer-registry-confirmation` and
`release.sh nightly --defer-registry-confirmation` both stop with "only
applies to a published canary".
- `actionlint` reports no new findings compared with master.
- Not verified before merge: a full publish through
`canary-finalize.yml`. It can run only from master. The first canary
after merge is the check.

## Risks

- Canary numbers jump from `canary.7` to about `canary.8144` (master has
8,143 commits). This is valid semver and still higher than every earlier
canary; users install through the `canary` dist-tag.
- The `canary/v` tag, Docker image and smoke now finish after the
release run ends. Nightly selects only tagged canaries, as before. If
finalize fails, that canary is on npm without a tag, so nightly skips
it, as it skips any canary whose old in-run verification failed.
- A canary can now publish while the previous one is still being
confirmed. Publishing itself is still serialized, so the `canary`
dist-tag only moves forward.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:37:20 -06:00

501 lines
15 KiB
JavaScript

#!/usr/bin/env node
import { pathToFileURL } from "node:url";
const CANARY_VERSION_RE = /-canary\.\d+$/;
const PRERELEASE_VERSION_RE = /-(?:canary|nightly|beta)\.\d+$/;
// Channels that publish prerelease versions and must never move `latest`.
const PRERELEASE_CHANNELS = new Set(["canary", "nightly", "beta"]);
const EXIT_RETRIABLE_FAILURE = 1;
const EXIT_NON_RETRIABLE_FAILURE = 2;
export function isCanaryVersion(version) {
return CANARY_VERSION_RE.test(version);
}
export function isPrereleaseVersion(version) {
return PRERELEASE_VERSION_RE.test(version);
}
function createExitError(message, exitCode = EXIT_RETRIABLE_FAILURE) {
return Object.assign(new Error(message), { exitCode });
}
function createProblem(message, { retriable = true } = {}) {
return { message, retriable };
}
function usage() {
process.stderr.write(
[
"Usage:",
" node scripts/verify-release-registry-state.mjs --channel <canary|nightly|beta|stable> --dist-tag <tag> --target-version <version> --package <name> [--package <name> ...] [--allow-canary-latest] [--allow-newer-canary]",
"",
].join("\n"),
);
}
function parseArgs(argv) {
const options = {
channel: "",
distTag: "",
targetVersion: "",
allowCanaryLatest: false,
allowNewerCanary: false,
packages: [],
};
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
switch (arg) {
case "--channel":
options.channel = argv[index + 1] ?? "";
index += 1;
break;
case "--dist-tag":
options.distTag = argv[index + 1] ?? "";
index += 1;
break;
case "--target-version":
options.targetVersion = argv[index + 1] ?? "";
index += 1;
break;
case "--package":
options.packages.push(argv[index + 1] ?? "");
index += 1;
break;
case "--allow-canary-latest":
options.allowCanaryLatest = true;
break;
case "--allow-newer-canary":
options.allowNewerCanary = true;
break;
case "-h":
case "--help":
usage();
process.exit(0);
default:
throw createExitError(`unexpected argument: ${arg}`, EXIT_NON_RETRIABLE_FAILURE);
}
}
if (!PRERELEASE_CHANNELS.has(options.channel) && options.channel !== "stable") {
throw createExitError("--channel must be canary, nightly, beta, or stable", EXIT_NON_RETRIABLE_FAILURE);
}
if (!options.distTag) {
throw createExitError("--dist-tag is required", EXIT_NON_RETRIABLE_FAILURE);
}
if (!options.targetVersion) {
throw createExitError("--target-version is required", EXIT_NON_RETRIABLE_FAILURE);
}
if (options.packages.length === 0 || options.packages.some((name) => !name)) {
throw createExitError("at least one non-empty --package value is required", EXIT_NON_RETRIABLE_FAILURE);
}
if (options.allowCanaryLatest && options.channel !== "canary") {
throw createExitError("--allow-canary-latest only applies to canary releases", EXIT_NON_RETRIABLE_FAILURE);
}
if (options.allowNewerCanary && options.channel !== "canary") {
throw createExitError("--allow-newer-canary only applies to canary releases", EXIT_NON_RETRIABLE_FAILURE);
}
return options;
}
function createRegistryUrl(packageName, version = "") {
const registry = process.env.npm_config_registry ?? process.env.NPM_CONFIG_REGISTRY ?? "https://registry.npmjs.org/";
const baseUrl = registry.endsWith("/") ? registry : `${registry}/`;
const encodedPackage = encodeURIComponent(packageName);
if (!version) {
return new URL(encodedPackage, baseUrl);
}
return new URL(`${encodedPackage}/${encodeURIComponent(version)}`, baseUrl);
}
export async function fetchRegistryJson(url, { allowMissing = false, timeoutMs = 30_000 } = {}) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
let response;
try {
response = await fetch(url, {
signal: controller.signal,
headers: {
accept: "application/vnd.npm.install-v1+json, application/json;q=0.9",
},
});
} catch (error) {
if (error instanceof Error && error.name === "AbortError") {
throw new Error(`npm registry request timed out for ${url} after ${timeoutMs}ms`);
}
throw error;
} finally {
clearTimeout(timeout);
}
if (response.status === 404 && allowMissing) {
return null;
}
if (!response.ok) {
throw new Error(`npm registry request failed for ${url}: ${response.status} ${response.statusText}`);
}
return response.json();
}
async function fetchPackageDocument(packageName, { allowMissing = false } = {}) {
return fetchRegistryJson(createRegistryUrl(packageName), { allowMissing });
}
async function fetchPackageManifest(packageName, version, { allowMissing = false } = {}) {
return fetchRegistryJson(createRegistryUrl(packageName, version), { allowMissing });
}
export function createManifestLookupKey(packageName, version) {
return `${packageName}@${version}`;
}
function isRangeVersionSpecifier(version) {
return /[\^~*xX><| ]/.test(version);
}
function resolvePublishedManifest(packageName, version, packageDoc, packageManifestsByKey = new Map()) {
const directManifest = packageManifestsByKey.get(createManifestLookupKey(packageName, version));
if (directManifest) {
return directManifest;
}
if (directManifest === null) {
return null;
}
return packageDoc?.versions?.[version] ?? null;
}
function collectInternalDependencyProblemEntries(
manifest,
packageDocsByName,
packageManifestsByKey = new Map(),
) {
const problems = [];
const sections = [
["dependencies", manifest.dependencies ?? {}],
["optionalDependencies", manifest.optionalDependencies ?? {}],
["peerDependencies", manifest.peerDependencies ?? {}],
];
for (const [sectionName, deps] of sections) {
for (const [dependencyName, dependencyVersion] of Object.entries(deps)) {
if (!dependencyName.startsWith("@paperclipai/")) {
continue;
}
if (typeof dependencyVersion !== "string" || !dependencyVersion) {
problems.push(
createProblem(
`${sectionName} declares ${dependencyName} with a non-string version: ${JSON.stringify(dependencyVersion)}`,
),
);
continue;
}
// Peer dependency ranges express compatibility, not a manifest that can be fetched directly.
if (sectionName === "peerDependencies" && isRangeVersionSpecifier(dependencyVersion)) {
continue;
}
const dependencyManifest = resolvePublishedManifest(
dependencyName,
dependencyVersion,
packageDocsByName.get(dependencyName),
packageManifestsByKey,
);
const dependencyLookupKey = createManifestLookupKey(dependencyName, dependencyVersion);
if (!dependencyManifest) {
const dependencyDoc = packageDocsByName.get(dependencyName);
if (!dependencyDoc && !packageManifestsByKey.has(dependencyLookupKey)) {
problems.push(
createProblem(
`${sectionName} requires ${dependencyName}@${dependencyVersion}, but npm publication metadata was not fetched for that dependency`,
),
);
continue;
}
problems.push(
createProblem(
`${sectionName} requires ${dependencyName}@${dependencyVersion}, but npm does not expose that version`,
),
);
}
}
}
return problems;
}
export function collectInternalDependencyProblems(
manifest,
packageDocsByName,
packageManifestsByKey = new Map(),
) {
return collectInternalDependencyProblemEntries(
manifest,
packageDocsByName,
packageManifestsByKey,
).map((problem) => problem.message);
}
function requireManifest(packageName, version, packageDoc, packageManifestsByKey, problems) {
const manifest = resolvePublishedManifest(packageName, version, packageDoc, packageManifestsByKey);
if (!manifest) {
if (problems) {
problems.push(createProblem(`${packageName}: npm registry is missing manifest data for ${version}`));
}
return null;
}
return manifest;
}
export function compareCanaryVersions(left, right) {
const parse = (version) => {
const match = /^(\d+)\.(\d+)\.(\d+)-canary\.(\d+)$/.exec(version ?? "");
return match ? match.slice(1).map(Number) : null;
};
const a = parse(left);
const b = parse(right);
if (!a || !b) return null;
for (let index = 0; index < a.length; index += 1) {
if (a[index] !== b[index]) return a[index] < b[index] ? -1 : 1;
}
return 0;
}
export function verifyPackageRegistryProblems({
packageName,
packageDoc,
packageDocsByName,
packageManifestsByKey = new Map(),
channel,
distTag,
targetVersion,
allowCanaryLatest,
allowNewerCanary = false,
}) {
const problems = [];
const distTags = packageDoc["dist-tags"] ?? {};
const taggedVersion = distTags[distTag];
const taggedNewerCanary = allowNewerCanary && channel === "canary" && compareCanaryVersions(taggedVersion, targetVersion) === 1;
if (taggedVersion !== targetVersion && !taggedNewerCanary) {
problems.push(
createProblem(
`${packageName}: dist-tag ${distTag} resolves to ${taggedVersion ?? "<missing>"}, expected ${targetVersion}`,
),
);
}
const targetManifest = requireManifest(packageName, targetVersion, packageDoc, packageManifestsByKey, problems);
if (targetManifest) {
for (const problem of collectInternalDependencyProblemEntries(
targetManifest,
packageDocsByName,
packageManifestsByKey,
)) {
problems.push(createProblem(`${packageName}@${targetVersion}: ${problem.message}`, problem));
}
}
if (PRERELEASE_CHANNELS.has(channel)) {
const latestVersion = distTags.latest;
if (latestVersion && isPrereleaseVersion(latestVersion) && !allowCanaryLatest) {
problems.push(
createProblem(
`${packageName}: latest dist-tag still resolves to prerelease ${latestVersion}; if that state is intentional, rerun the verification script directly with --allow-canary-latest`,
{ retriable: false },
),
);
}
if (latestVersion && isPrereleaseVersion(latestVersion)) {
const latestManifest = requireManifest(
packageName,
latestVersion,
packageDoc,
packageManifestsByKey,
problems,
);
if (latestManifest) {
for (const problem of collectInternalDependencyProblemEntries(
latestManifest,
packageDocsByName,
packageManifestsByKey,
)) {
problems.push(createProblem(`${packageName}@${latestVersion} via latest: ${problem.message}`, problem));
}
}
}
}
return problems;
}
export function verifyPackageRegistryState(options) {
return verifyPackageRegistryProblems(options).map((problem) => problem.message);
}
function collectInternalDependencyVersions(manifest) {
const dependencyVersions = [];
for (const [sectionName, deps] of [
["dependencies", manifest.dependencies ?? {}],
["optionalDependencies", manifest.optionalDependencies ?? {}],
["peerDependencies", manifest.peerDependencies ?? {}],
]) {
for (const [dependencyName, dependencyVersion] of Object.entries(deps)) {
if (!dependencyName.startsWith("@paperclipai/")) {
continue;
}
if (typeof dependencyVersion !== "string" || !dependencyVersion) {
continue;
}
if (sectionName === "peerDependencies" && isRangeVersionSpecifier(dependencyVersion)) {
continue;
}
dependencyVersions.push({
packageName: dependencyName,
version: dependencyVersion,
});
}
}
return dependencyVersions;
}
async function main() {
const options = parseArgs(process.argv.slice(2));
const packageNames = [...new Set(options.packages)];
const packageDocsByName = new Map();
const packageManifestsByKey = new Map();
await Promise.all(
packageNames.map(async (packageName) => {
packageDocsByName.set(packageName, await fetchPackageDocument(packageName));
}),
);
const versionsToFetchByPackage = new Map();
for (const packageName of packageNames) {
const packageDoc = packageDocsByName.get(packageName);
const versionsToFetch = new Set([options.targetVersion]);
const latestVersion = packageDoc?.["dist-tags"]?.latest;
if (latestVersion && isCanaryVersion(latestVersion)) {
versionsToFetch.add(latestVersion);
}
versionsToFetchByPackage.set(packageName, versionsToFetch);
}
await Promise.all(
[...versionsToFetchByPackage.entries()].flatMap(([packageName, versionsToFetch]) =>
[...versionsToFetch].map(async (version) => {
packageManifestsByKey.set(
createManifestLookupKey(packageName, version),
await fetchPackageManifest(packageName, version, { allowMissing: true }),
);
}),
),
);
const dependencyVersionsByKey = new Map();
for (const [packageName, versionsToFetch] of versionsToFetchByPackage.entries()) {
for (const version of versionsToFetch) {
const manifest = resolvePublishedManifest(
packageName,
version,
packageDocsByName.get(packageName),
packageManifestsByKey,
);
if (!manifest) {
continue;
}
for (const dependencyVersion of collectInternalDependencyVersions(manifest)) {
dependencyVersionsByKey.set(
createManifestLookupKey(dependencyVersion.packageName, dependencyVersion.version),
dependencyVersion,
);
}
}
}
await Promise.all(
[...dependencyVersionsByKey.values()].map(async ({ packageName, version }) => {
const lookupKey = createManifestLookupKey(packageName, version);
if (packageManifestsByKey.has(lookupKey)) {
return;
}
packageManifestsByKey.set(
lookupKey,
await fetchPackageManifest(packageName, version, { allowMissing: true }),
);
}),
);
const problems = [];
for (const packageName of packageNames) {
process.stdout.write(` Verifying ${packageName} on dist-tag ${options.distTag}\n`);
const packageProblems = verifyPackageRegistryProblems({
packageName,
packageDoc: packageDocsByName.get(packageName),
packageDocsByName,
packageManifestsByKey,
channel: options.channel,
distTag: options.distTag,
targetVersion: options.targetVersion,
allowCanaryLatest: options.allowCanaryLatest,
allowNewerCanary: options.allowNewerCanary,
});
if (packageProblems.length === 0) {
process.stdout.write(` ✓ dist-tag and published internal dependencies are consistent\n`);
continue;
}
for (const problem of packageProblems) {
process.stderr.write(` ✗ ${problem.message}\n`);
problems.push(problem);
}
}
if (problems.length > 0) {
const exitCode = problems.some((problem) => !problem.retriable)
? EXIT_NON_RETRIABLE_FAILURE
: EXIT_RETRIABLE_FAILURE;
throw createExitError(`npm registry verification failed for ${problems.length} problem(s)`, exitCode);
}
}
const isDirectRun = process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href;
if (isDirectRun) {
main().catch((error) => {
process.stderr.write(`Error: ${error.message}\n`);
process.exit(error.exitCode ?? EXIT_RETRIABLE_FAILURE);
});
}