mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
perf(release): number canaries by master position and confirm them outside the release lock (#21917)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every master push runs a canary, and only one canary at a time may run the release workflow > - After #21554 and #21555 a canary held that lock for 35.5 min, and 13-16 min of it was waiting for npm to show the new packages > - The wait was required: the next canary computed `canary.N` from npm's version list, so starting before npm showed the previous canary would reuse its number > - This pull request numbers a canary by its commit's position on master and moves the registry confirmation, the git tag, the Docker dispatch and the onboarding smoke to a separate `canary-finalize.yml` run > - The benefit is that the release lock is held for about 20-25 min per canary instead of 35, with the same checks before a canary is tagged ## Linked Issues or Issue Description **What existing behavior does this improve?** Canary throughput. In run 43715037497 (2026-11-15) `publish_canary` took 23.7 min, and 13.3 min of that was "Waiting for registry visibility" (13:08:50 to 13:22:11). The next canary could not start until that run ended. **Current behavior** `release.sh canary` sets `N` to (highest published `-canary.N` for today) + 1, then publishes, waits for npm to show every package, checks dist-tags, installs `paperclipai@canary`, and creates `canary/v<version>`. `publish_canary` pushes the tag, dispatches `docker.yml`, and `smoke_canary_onboarding` runs in the same run. **Proposed behavior** `N` = `git rev-list --count HEAD` (full history required), or the registry number + 1 when that is higher (a rerun of the same commit). `release.sh canary --defer-registry-confirmation` ends after npm accepts every publish. `publish_canary` dispatches `canary-finalize.yml`, which does the confirmation, tag, Docker dispatch and smoke outside the release lock. Nightly, beta and stable numbering and flow are unchanged. Refs #21554, #21555 ## What Changed - `scripts/release-lib.sh`: `next_canary_version_from_source`. A publishing canary refuses a shallow checkout; a dry run (the PR Canary Dry Run job) falls back to the registry number. - `scripts/release.sh`: canary uses the new number; new canary-only `--defer-registry-confirmation` flag skips the registry wait, the dist-tag check and the local tag, and writes the version to `PAPERCLIP_RELEASE_VERSION_FILE`. - `scripts/confirm-canary-publication.sh` (new): waits for registry visibility, checks provenance, checks dist-tags with `--allow-newer-canary`, installs `paperclipai@<version>`, then creates and pushes `canary/v<version>`. A tag already on the same commit is accepted, so reruns are safe. - `scripts/verify-canary-provenance.mjs` (new): every package's npm SLSA provenance must name `.github/workflows/release.yml` on `refs/heads/master` in `paperclipai/paperclip` at the source commit. A dispatch with invented inputs cannot tag anything. - `scripts/verify-release-registry-state.mjs`: `--allow-newer-canary` accepts a `canary` dist-tag that a newer canary already moved; older, foreign or missing tags still fail. - `.github/workflows/canary-finalize.yml` (new, `workflow_dispatch`, master only): `confirm_canary` (contents: write, actions: write) and `smoke_canary_onboarding` (downloads the publisher's lockfile artifact by run id). Inputs reach shell steps only through a validating step. - `.github/workflows/release.yml`: `publish_canary` hands off to `canary-finalize.yml` and drops to `contents: read`; its tag push, Docker dispatch and smoke job move to `canary-finalize.yml`. - Tests and `doc/RELEASING.md`. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/release-lib.test.mjs scripts/verify-release-registry-state.test.mjs scripts/verify-canary-provenance.test.mjs`: 846 pass. `scripts/__tests__/release-verify-workflow.test.mjs`: 73 pass. - New tests: numbering by master position, rerun moves past a published number, shallow refusal and dry-run fallback; newer/older/foreign dist-tags; provenance from another commit, workflow, branch, repository or package is rejected; finalize wiring, permissions and input handling. - Real npm data: `verify-canary-provenance.mjs 2026.1116.0-canary.7 b4fa1c7f... paperclipai @paperclipai/shared @paperclipai/server` passes; the same version with another commit fails. - `confirm-canary-publication.sh 2026.1116.0-canary.7 b4fa1c7f...` from a checkout of that commit ran all steps against npm in 61 s and ended with "canary/v2026.1116.0-canary.7 already points at b4fa1c7f...", pushing nothing. - `release.sh canary --dry-run --defer-registry-confirmation` and `release.sh nightly --defer-registry-confirmation` both stop with "only applies to a published canary". - `actionlint` reports no new findings compared with master. - Not verified before merge: a full publish through `canary-finalize.yml`. It can run only from master. The first canary after merge is the check. ## Risks - Canary numbers jump from `canary.7` to about `canary.8144` (master has 8,143 commits). This is valid semver and still higher than every earlier canary; users install through the `canary` dist-tag. - The `canary/v` tag, Docker image and smoke now finish after the release run ends. Nightly selects only tagged canaries, as before. If finalize fails, that canary is on npm without a tag, so nightly skips it, as it skips any canary whose old in-run verification failed. - A canary can now publish while the previous one is still being confirmed. Publishing itself is still serialized, so the `canary` dist-tag only moves forward. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
93111b175b
commit
d16c06308d
15 files changed
+742
-179
No files matched your search
@@ -0,0 +1,151 @@
|
||||
name: Canary finalize
|
||||
run-name: Canary finalize ${{ inputs.version }}
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: Canary version that publish_canary published
|
||||
required: true
|
||||
type: string
|
||||
source_sha:
|
||||
description: Full source commit SHA of that canary
|
||||
required: true
|
||||
type: string
|
||||
release_run_id:
|
||||
description: Release workflow run that published it
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: canary-finalize-${{ inputs.version }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
NPM_PUBLISH_VERIFY_ATTEMPTS: "180"
|
||||
NPM_PUBLISH_VERIFY_DELAY_SECONDS: "10"
|
||||
|
||||
jobs:
|
||||
confirm_canary:
|
||||
name: Confirm canary on npm and tag it
|
||||
if: github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: write
|
||||
actions: write
|
||||
outputs:
|
||||
version: ${{ steps.inputs.outputs.version }}
|
||||
source_sha: ${{ steps.inputs.outputs.source_sha }}
|
||||
release_run_id: ${{ steps.inputs.outputs.release_run_id }}
|
||||
steps:
|
||||
- name: Validate inputs
|
||||
id: inputs
|
||||
env:
|
||||
VERSION: ${{ inputs.version }}
|
||||
SOURCE_SHA: ${{ inputs.source_sha }}
|
||||
RELEASE_RUN_ID: ${{ inputs.release_run_id }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
[[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+-canary\.[0-9]+$ ]] || { echo "Error: not a canary version" >&2; exit 1; }
|
||||
[[ "$SOURCE_SHA" =~ ^[0-9a-f]{40}$ ]] || { echo "Error: not a full commit SHA" >&2; exit 1; }
|
||||
[[ "$RELEASE_RUN_ID" =~ ^[1-9][0-9]*$ ]] || { echo "Error: not a run id" >&2; exit 1; }
|
||||
{
|
||||
echo "version=$VERSION"
|
||||
echo "source_sha=$SOURCE_SHA"
|
||||
echo "release_run_id=$RELEASE_RUN_ID"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
ref: ${{ steps.inputs.outputs.source_sha }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Configure git author
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Confirm the canary and push its tag
|
||||
env:
|
||||
VERSION: ${{ steps.inputs.outputs.version }}
|
||||
SOURCE_SHA: ${{ steps.inputs.outputs.source_sha }}
|
||||
PUBLISH_REMOTE: origin
|
||||
run: ./scripts/confirm-canary-publication.sh "$VERSION" "$SOURCE_SHA"
|
||||
|
||||
- name: Build Docker images for the canary tag
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
VERSION: ${{ steps.inputs.outputs.version }}
|
||||
RELEASE_RUN_ID: ${{ steps.inputs.outputs.release_run_id }}
|
||||
run: |
|
||||
{
|
||||
echo "## Canary published"
|
||||
echo ""
|
||||
echo "- Published canary: \`$VERSION\` ([release run](https://github.com/$GITHUB_REPOSITORY/actions/runs/$RELEASE_RUN_ID))"
|
||||
echo "- Docker build dispatched at \`canary/v$VERSION\`"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
gh workflow run docker.yml --ref "refs/tags/canary/v$VERSION" --repo "$GITHUB_REPOSITORY"
|
||||
|
||||
smoke_canary_onboarding:
|
||||
needs: confirm_canary
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
env:
|
||||
PAPERCLIPAI_VERSION: ${{ needs.confirm_canary.outputs.version }}
|
||||
PAPERCLIP_PLAYWRIGHT_CHANNEL: chrome
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
ref: ${{ needs.confirm_canary.outputs.source_sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
||||
with:
|
||||
version: 9.15.4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Restore canary smoke lockfile
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
name: canary-smoke-lockfile-${{ needs.confirm_canary.outputs.source_sha }}
|
||||
run-id: ${{ needs.confirm_canary.outputs.release_run_id }}
|
||||
github-token: ${{ github.token }}
|
||||
|
||||
- name: Install test dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Show browser version
|
||||
run: google-chrome --version
|
||||
|
||||
- name: Smoke exact published canary through onboarding
|
||||
env:
|
||||
PAPERCLIP_CANARY_SMOKE_SERVER_LOG: ${{ runner.temp }}/canary-onboarding-server.log
|
||||
run: pnpm run test:canary-onboarding-smoke
|
||||
|
||||
- name: Upload failed canary onboarding diagnostics
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: canary-onboarding-smoke-${{ needs.confirm_canary.outputs.version }}
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
path: |
|
||||
${{ runner.temp }}/canary-onboarding-server.log
|
||||
tests/canary-onboarding/playwright-report/
|
||||
tests/canary-onboarding/test-results/
|
||||
@@ -9,8 +9,8 @@ on:
|
||||
- "nightly/v*"
|
||||
- "beta/v*"
|
||||
# Release workflows push lane tags with GITHUB_TOKEN, and GitHub suppresses
|
||||
# push-triggered runs for those, so release.yml dispatches this workflow at
|
||||
# the new tag ref instead. The tag mapping below keys off github.ref either
|
||||
# push-triggered runs for those, so release.yml (canary-finalize.yml for a
|
||||
# canary) dispatches this workflow at the new tag ref instead. The tag mapping below keys off github.ref either
|
||||
# way.
|
||||
workflow_dispatch:
|
||||
|
||||
|
||||
@@ -249,7 +249,10 @@ jobs:
|
||||
run: node "$CI_NODE_TEST" ./scripts/__tests__/e2e-shard.test.mjs
|
||||
|
||||
- name: Test release verify workflow wiring
|
||||
run: node "$CI_NODE_TEST" ./scripts/__tests__/release-verify-workflow.test.mjs ./scripts/__tests__/eval-schedule-monitor.test.mjs ./scripts/__tests__/prepare-runner-release-image.test.mjs ./scripts/cloud-source-verification.test.mjs ./scripts/cloud-readiness-runner-loss.test.mjs ./scripts/standard-image-contract.test.mjs ./scripts/reuse-canary-image.test.mjs
|
||||
run: |
|
||||
tests=(./scripts/__tests__/release-verify-workflow.test.mjs ./scripts/__tests__/eval-schedule-monitor.test.mjs ./scripts/__tests__/prepare-runner-release-image.test.mjs ./scripts/cloud-source-verification.test.mjs ./scripts/cloud-readiness-runner-loss.test.mjs ./scripts/standard-image-contract.test.mjs ./scripts/reuse-canary-image.test.mjs)
|
||||
if [[ -f ./scripts/verify-canary-provenance.test.mjs ]]; then tests+=(./scripts/verify-canary-provenance.test.mjs); fi
|
||||
node "$CI_NODE_TEST" "${tests[@]}"
|
||||
|
||||
- name: Test merge-queue workflow wiring
|
||||
# This workflow loads from master and tests the caller's checkout. A
|
||||
|
||||
@@ -396,9 +396,9 @@ jobs:
|
||||
outputs:
|
||||
canary_version: ${{ steps.canary_tag.outputs.version }}
|
||||
permissions:
|
||||
contents: write
|
||||
contents: read
|
||||
id-token: write
|
||||
# For the explicit docker.yml dispatch below.
|
||||
# For the canary-finalize.yml dispatch below.
|
||||
actions: write
|
||||
|
||||
steps:
|
||||
@@ -488,7 +488,8 @@ jobs:
|
||||
- name: Publish canary
|
||||
env:
|
||||
GITHUB_ACTIONS: "true"
|
||||
run: ./scripts/release.sh canary --skip-verify
|
||||
PAPERCLIP_RELEASE_VERSION_FILE: ${{ runner.temp }}/canary-version
|
||||
run: ./scripts/release.sh canary --skip-verify --defer-registry-confirmation
|
||||
|
||||
- name: Dump npm debug logs
|
||||
if: failure()
|
||||
@@ -500,100 +501,25 @@ jobs:
|
||||
-e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig'
|
||||
done
|
||||
|
||||
- name: Push canary tag
|
||||
- name: Hand off npm confirmation, the tag and the image to canary-finalize
|
||||
id: canary_tag
|
||||
run: |
|
||||
tag="$(git tag --points-at HEAD | grep '^canary/v' | head -1)"
|
||||
if [ -z "$tag" ]; then
|
||||
echo "Error: no canary tag points at HEAD after release." >&2
|
||||
exit 1
|
||||
fi
|
||||
git push origin "refs/tags/${tag}"
|
||||
echo "version=${tag#canary/v}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Canary images previously relied on the master-push docker.yml run,
|
||||
# whose single pending concurrency slot gets superseded by every
|
||||
# newer push — on a busy day no canary image publishes at all (five
|
||||
# consecutive canaries shipped npm packages with no cloud image on
|
||||
# 2026-09-06, starving downstream managed deploys for ~18 hours).
|
||||
# Tag pushes made with GITHUB_TOKEN do not fire docker.yml's
|
||||
# triggers, so dispatch the image build at the canary tag
|
||||
# explicitly, exactly like the nightly and beta lanes: the run keys
|
||||
# its concurrency off the tag ref, so no master push can supersede
|
||||
# it, and docker.yml's `type=sha` mapping publishes the
|
||||
# standard sha-<short> images either way.
|
||||
- name: Build Docker images for the canary tag
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
VERSION_FILE: ${{ runner.temp }}/canary-version
|
||||
SOURCE_SHA: ${{ github.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version="$(cat "$VERSION_FILE")"
|
||||
[[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+-canary\.[0-9]+$ ]] || { echo "Error: release.sh wrote no canary version" >&2; exit 1; }
|
||||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||
gh workflow run canary-finalize.yml --ref master --repo "$GITHUB_REPOSITORY" \
|
||||
-f version="$version" -f source_sha="$SOURCE_SHA" -f release_run_id="$GITHUB_RUN_ID"
|
||||
{
|
||||
echo "## Canary published"
|
||||
echo "## Canary published to npm"
|
||||
echo ""
|
||||
echo "- Published canary: \`${{ steps.canary_tag.outputs.version }}\`"
|
||||
echo "- Docker build dispatched at \`canary/v${{ steps.canary_tag.outputs.version }}\`"
|
||||
echo "- Version: \`$version\`"
|
||||
echo "- npm confirmation, \`canary/v$version\`, the Docker build and the onboarding smoke run in Canary finalize."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
gh workflow run docker.yml --ref "refs/tags/canary/v${{ steps.canary_tag.outputs.version }}" --repo "$GITHUB_REPOSITORY"
|
||||
|
||||
# The package is already public when this gate runs. A red result leaves the
|
||||
# immutable canary in npm, but makes the release workflow visibly fail before
|
||||
# anyone mistakes an installable package for an onboardable one.
|
||||
smoke_canary_onboarding:
|
||||
needs: publish_canary
|
||||
if: ${{ !cancelled() && needs.publish_canary.result == 'success' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
PAPERCLIPAI_VERSION: ${{ needs.publish_canary.outputs.canary_version }}
|
||||
PAPERCLIP_PLAYWRIGHT_CHANNEL: chrome
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
||||
with:
|
||||
version: 9.15.4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
# Default download scope is this workflow run, including job reruns.
|
||||
# A missing artifact fails instead of resolving a different dependency set.
|
||||
- name: Restore canary smoke lockfile
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
name: canary-smoke-lockfile-${{ github.sha }}
|
||||
|
||||
- name: Install test dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Show browser version
|
||||
run: google-chrome --version
|
||||
|
||||
- name: Smoke exact published canary through onboarding
|
||||
env:
|
||||
PAPERCLIP_CANARY_SMOKE_SERVER_LOG: ${{ runner.temp }}/canary-onboarding-server.log
|
||||
run: pnpm run test:canary-onboarding-smoke
|
||||
|
||||
- name: Upload failed canary onboarding diagnostics
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: canary-onboarding-smoke-${{ needs.publish_canary.outputs.canary_version }}
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
path: |
|
||||
${{ runner.temp }}/canary-onboarding-server.log
|
||||
tests/canary-onboarding/playwright-report/
|
||||
tests/canary-onboarding/test-results/
|
||||
|
||||
# ----- Nightly lane -----------------------------------------------------
|
||||
# Once a night (or on a forced nightly dispatch), promote the newest master
|
||||
|
||||
+18
-10
@@ -93,25 +93,33 @@ internal traceability tag.
|
||||
|
||||
### Canary
|
||||
|
||||
Every push to `master` runs the canary path inside [`.github/workflows/release.yml`](../.github/workflows/release.yml).
|
||||
Every push to `master` runs the canary path inside [`.github/workflows/release.yml`](../.github/workflows/release.yml). Only one canary at a time runs that path. It ends as soon as npm has accepted every package, and [`.github/workflows/canary-finalize.yml`](../.github/workflows/canary-finalize.yml) finishes the canary outside that lock, so the next canary can publish while the previous one is still being confirmed.
|
||||
|
||||
It:
|
||||
`release.yml` (`publish_canary`):
|
||||
|
||||
- verifies the pushed commit
|
||||
- computes the canary version for the current UTC date
|
||||
- computes the canary version `YYYY.MDD.P-canary.N` for the current UTC date. `N` is 100 × the commit's position on master (`git rev-list --count HEAD`) plus the run attempt minus 1. Each commit owns its own range of 100, so neither a rerun nor an overlapping canary can take another commit's number. A number already published in that range (the same commit published again outside a GitHub rerun) is skipped, without leaving the range. This needs the full history; a publishing canary refuses a shallow checkout, and a dry run falls back to the next registry number. Nightly, beta and stable numbering are unchanged.
|
||||
- publishes workspace packages dependency-first under npm dist-tag `canary`
|
||||
- waits for each package version to become registry-visible before continuing
|
||||
- publishes the user-facing `paperclipai` package last, so `paperclipai@canary` does not advance before the full package set exists
|
||||
- verifies that `canary` resolves to the just-published version and that published internal dependencies exist on npm
|
||||
- installs `paperclipai@canary` into a clean temporary prefix as the final npm gate
|
||||
- starts the exact published canary through a separate fresh npm install for the onboarding smoke; npm `ETARGET` and verified canary tarball availability failures retry, up to three attempts within two minutes, before onboarding runs once under the existing five-minute startup deadline
|
||||
- dispatches `canary-finalize.yml` with the version, the source commit and its own run id (`release.sh canary --defer-registry-confirmation`)
|
||||
|
||||
`canary-finalize.yml`, which runs [`scripts/confirm-canary-publication.sh`](../scripts/confirm-canary-publication.sh):
|
||||
|
||||
- waits for each package version to become registry-visible
|
||||
- requires the source commit to be on master and the canary number to come from it (`N / 100` equals the commit's master position), so a dispatch with invented inputs cannot tag another commit
|
||||
- checks npm provenance ([`scripts/verify-canary-provenance.mjs`](../scripts/verify-canary-provenance.mjs)): any package whose provenance names something other than `.github/workflows/release.yml` on `refs/heads/master` at the source commit fails the check. A package with no provenance is accepted, because that is the supported retry after a duplicate transparency-log entry, only the trusted release workflow can publish, and the number check above already ties the version to the commit. Brief npm errors are retried.
|
||||
- verifies that `canary` resolves to this version or a newer canary, and that published internal dependencies exist on npm
|
||||
- installs the exact `paperclipai@<version>` into a clean temporary prefix
|
||||
- fails by default if npm leaves `latest` pointing at a canary; use `--allow-canary-latest` only when that state is intentional
|
||||
- creates a git tag `canary/vYYYY.MDD.P-canary.N`
|
||||
- creates and pushes the git tag `canary/vYYYY.MDD.P-canary.N` (a tag that already points at the same commit is accepted, so a rerun is safe). Nightly still selects only tagged canaries.
|
||||
- dispatches `docker.yml` at the tag
|
||||
- starts the exact published canary through a separate fresh npm install for the onboarding smoke; npm `ETARGET` and verified canary tarball availability failures retry, up to three attempts within two minutes, before onboarding runs once under the existing five-minute startup deadline. The smoke installs from the publisher run's lockfile artifact.
|
||||
|
||||
#### Busy master and lost runners
|
||||
|
||||
Master often receives a commit every few minutes, faster than one canary
|
||||
publishes (about 60-90 minutes). The release workflow keeps one running canary
|
||||
publishes (about 20-25 minutes inside `release.yml`, then its confirmation in
|
||||
`canary-finalize.yml`). The release workflow keeps one running canary
|
||||
and one pending canary. A newer push replaces the pending one, so the canary
|
||||
that runs next is always the newest master commit. Commits in between are not
|
||||
published on their own; they ship inside the next canary.
|
||||
@@ -146,7 +154,7 @@ runs `build:binary`.
|
||||
|
||||
After publication, the onboarding smoke checks out the same source SHA and
|
||||
installs its test dependencies from the publisher's resolved lockfile with
|
||||
`--frozen-lockfile`. The same-run artifact is captured before tracked files are
|
||||
`--frozen-lockfile`. The publisher's artifact is captured before tracked files are
|
||||
restored or release versions are rewritten. This lets a source commit use its
|
||||
current patches and manifests while the separate lockfile-refresh PR is still
|
||||
pending, without resolving new dependencies in the smoke job. The smoke still installs and tests
|
||||
|
||||
@@ -46,6 +46,7 @@ list_public_package_info() {
|
||||
package_publish_tool() { printf 'pnpm\\n'; }
|
||||
next_stable_version() { printf '2026.710.0\\n'; }
|
||||
next_prerelease_version() { printf '2026.710.0-%s.0\\n' "$1"; }
|
||||
next_canary_version_from_source() { next_prerelease_version canary "$3"; }
|
||||
release_notes_file() { printf '%s/releases/v%s.md\\n' "$REPO_ROOT" "$1"; }
|
||||
stable_tag_name() { printf 'v%s\\n' "$1"; }
|
||||
prerelease_tag_name() { printf '%s/v%s\\n' "$1" "$2"; }
|
||||
|
||||
@@ -334,76 +334,63 @@ test("post-publish beta smoke survives the skipped candidate-verification ancest
|
||||
);
|
||||
});
|
||||
|
||||
test("published canaries are gated by the exact-version onboarding browser smoke", () => {
|
||||
const releaseWorkflow = readWorkflow("release.yml");
|
||||
function canaryJobs() {
|
||||
const release = readWorkflow("release.yml");
|
||||
const finalize = readWorkflow("canary-finalize.yml");
|
||||
return {
|
||||
release,
|
||||
finalize,
|
||||
publish: release.split(" publish_canary:\n")[1].split(" # ----- Nightly lane")[0],
|
||||
confirm: finalize.split(" confirm_canary:\n")[1].split(" smoke_canary_onboarding:\n")[0],
|
||||
smoke: finalize.split(" smoke_canary_onboarding:\n")[1],
|
||||
};
|
||||
}
|
||||
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/publish_canary:[\s\S]*?outputs:\n\s+canary_version: \$\{\{ steps\.canary_tag\.outputs\.version \}\}/,
|
||||
);
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/smoke_canary_onboarding:\n\s+needs: publish_canary\n\s+if: \$\{\{ !cancelled\(\) && needs\.publish_canary\.result == 'success' \}\}/,
|
||||
);
|
||||
const smoke = releaseWorkflow.split(" smoke_canary_onboarding:\n")[1].split(/\n [a-z_]+:\n/)[0];
|
||||
const condition = smoke.match(/^ if: \$\{\{ (.+) \}\}$/m)?.[1];
|
||||
assert.match(condition, /^!cancelled\(\) && needs\.publish_canary\.result == 'success'$/);
|
||||
for (const [result, cancelled, expected] of [
|
||||
["success", false, true], ["failure", false, false],
|
||||
["cancelled", false, false], ["skipped", false, false], ["success", true, false],
|
||||
]) {
|
||||
const needs = { publish_canary: { result }, release_runner_source: { result: "success" },
|
||||
select_nightly: { result: "skipped" }, select_beta: { result: "skipped" }, preflight_stable: { result: "skipped" } };
|
||||
assert.equal(runInNewContext(condition, { cancelled: () => cancelled, needs }), expected,
|
||||
`Canary smoke: publisher=${result}, cancelled=${cancelled}`);
|
||||
}
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/PAPERCLIPAI_VERSION: \$\{\{ needs\.publish_canary\.outputs\.canary_version \}\}/,
|
||||
);
|
||||
assert.match(releaseWorkflow, /test:canary-onboarding-smoke/);
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/smoke_canary_onboarding:[\s\S]*?uses: actions\/checkout@[0-9a-f]{40} # v7[\s\S]*?uses: pnpm\/action-setup@[0-9a-f]{40} # v6[\s\S]*?uses: actions\/setup-node@[0-9a-f]{40} # v7/,
|
||||
);
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/smoke_canary_onboarding:[\s\S]*?Install test dependencies\n\s+run: pnpm install --frozen-lockfile/,
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
releaseWorkflow.match(
|
||||
/smoke_canary_onboarding:[\s\S]*?(?=\n # ----- Nightly lane)/,
|
||||
)?.[0] ?? "",
|
||||
/cache: pnpm/,
|
||||
);
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/name: Smoke exact published canary through onboarding\n\s+env:\n\s+PAPERCLIP_CANARY_SMOKE_SERVER_LOG: \$\{\{ runner\.temp \}\}\/canary-onboarding-server\.log/,
|
||||
);
|
||||
assert.match(
|
||||
releaseWorkflow,
|
||||
/smoke_canary_onboarding:[\s\S]*?uses: actions\/upload-artifact@[0-9a-f]{40} # v7/,
|
||||
);
|
||||
assert.match(releaseWorkflow, /canary-onboarding-server\.log/);
|
||||
assert.match(releaseWorkflow, /tests\/canary-onboarding\/playwright-report/);
|
||||
test("published canaries are confirmed, tagged and onboarding-smoked outside the release lock", () => {
|
||||
const { release, finalize, publish, confirm, smoke } = canaryJobs();
|
||||
assert.doesNotMatch(release, /smoke_canary_onboarding:|confirm-canary-publication/);
|
||||
assert.match(publish, /outputs:\n\s+canary_version: \$\{\{ steps\.canary_tag\.outputs\.version \}\}/);
|
||||
assert.match(publish, /run: \.\/scripts\/release\.sh canary --skip-verify --defer-registry-confirmation/);
|
||||
assert.match(publish, /gh workflow run canary-finalize\.yml --ref master --repo "\$GITHUB_REPOSITORY" \\\n\s+-f version="\$version" -f source_sha="\$SOURCE_SHA" -f release_run_id="\$GITHUB_RUN_ID"/);
|
||||
assert.match(publish, /SOURCE_SHA: \$\{\{ github\.sha \}\}/);
|
||||
assert.match(publish, /permissions:\n\s+contents: read\n\s+id-token: write/);
|
||||
assert.match(finalize, /^on:\n workflow_dispatch:\n/m);
|
||||
assert.match(finalize, /^permissions: \{\}$/m);
|
||||
assert.match(confirm, /if: github\.repository == 'paperclipai\/paperclip' && github\.repository_id == '1170821064' && github\.ref == 'refs\/heads\/master'/);
|
||||
assert.match(confirm, /permissions:\n\s+contents: write\n\s+actions: write/);
|
||||
assert.match(confirm, /run: \.\/scripts\/confirm-canary-publication\.sh "\$VERSION" "\$SOURCE_SHA"/);
|
||||
assert.match(confirm, /ref: \$\{\{ steps\.inputs\.outputs\.source_sha \}\}\n\s+fetch-depth: 0/, "the master-position check needs full history");
|
||||
assert.match(confirm, /gh workflow run docker\.yml --ref "refs\/tags\/canary\/v\$VERSION"/);
|
||||
const [validation, afterValidation] = confirm.split(" - uses: actions/checkout@");
|
||||
assert.match(validation, /name: Validate inputs/);
|
||||
assert.equal((validation.match(/\$\{\{ inputs\./g) ?? []).length, 3);
|
||||
assert.doesNotMatch(afterValidation, /\$\{\{ inputs\./, "later steps read only the validated step outputs");
|
||||
assert.doesNotMatch(smoke, /\$\{\{ inputs\./);
|
||||
assert.match(smoke, /^ needs: confirm_canary$/m);
|
||||
assert.doesNotMatch(smoke, /^ if: /m);
|
||||
assert.match(smoke, /PAPERCLIPAI_VERSION: \$\{\{ needs\.confirm_canary\.outputs\.version \}\}/);
|
||||
assert.match(smoke, /uses: actions\/checkout@[0-9a-f]{40} # v7[\s\S]*?uses: pnpm\/action-setup@[0-9a-f]{40} # v6[\s\S]*?uses: actions\/setup-node@[0-9a-f]{40} # v7/);
|
||||
assert.match(smoke, /Install test dependencies\n\s+run: pnpm install --frozen-lockfile/);
|
||||
assert.doesNotMatch(smoke, /cache: pnpm/);
|
||||
assert.match(smoke, /run: pnpm run test:canary-onboarding-smoke/);
|
||||
assert.match(smoke, /name: Smoke exact published canary through onboarding\n\s+env:\n\s+PAPERCLIP_CANARY_SMOKE_SERVER_LOG: \$\{\{ runner\.temp \}\}\/canary-onboarding-server\.log/);
|
||||
assert.match(smoke, /uses: actions\/upload-artifact@[0-9a-f]{40} # v7/);
|
||||
assert.match(smoke, /tests\/canary-onboarding\/playwright-report/);
|
||||
assert.match(finalize, /NPM_PUBLISH_VERIFY_ATTEMPTS: "180"\n NPM_PUBLISH_VERIFY_DELAY_SECONDS: "10"/);
|
||||
});
|
||||
|
||||
test("canary smoke consumes its publisher's source-bound lockfile before frozen installation", () => {
|
||||
const workflow = readWorkflow("release.yml");
|
||||
const publish = workflow.split(" publish_canary:\n")[1].split(" smoke_canary_onboarding:\n")[0];
|
||||
const smoke = workflow.split(" smoke_canary_onboarding:\n")[1].split(" # ----- Nightly lane")[0];
|
||||
for (const job of [publish, smoke]) {
|
||||
assert.match(job, /name: Checkout repository\n\s+uses: actions\/checkout@[^\n]+\n\s+with:\n\s+ref: \$\{\{ github\.sha \}\}/);
|
||||
assert.match(job, /name: canary-smoke-lockfile-\$\{\{ github\.sha \}\}/);
|
||||
assert.match(job, /version: 9\.15\.4/);
|
||||
}
|
||||
const { publish, smoke } = canaryJobs();
|
||||
assert.match(publish, /name: Checkout repository\n\s+uses: actions\/checkout@[^\n]+\n\s+with:\n\s+ref: \$\{\{ github\.sha \}\}/);
|
||||
assert.match(smoke, /name: Checkout repository\n\s+uses: actions\/checkout@[^\n]+\n\s+with:\n\s+ref: \$\{\{ needs\.confirm_canary\.outputs\.source_sha \}\}/);
|
||||
for (const job of [publish, smoke]) assert.match(job, /version: 9\.15\.4/);
|
||||
assert.match(publish, /name: Save canary smoke lockfile\n\s+uses: actions\/upload-artifact@[0-9a-f]{40} # v4\n\s+with:\n\s+name: canary-smoke-lockfile-\$\{\{ github\.sha \}\}\n\s+path: pnpm-lock\.yaml\n\s+if-no-files-found: error\n\s+overwrite: true\n\s+retention-days: 14/);
|
||||
assert.ok(publish.indexOf("name: Install dependencies") < publish.indexOf("name: Save canary smoke lockfile"));
|
||||
assert.ok(publish.indexOf("name: Save canary smoke lockfile") < publish.indexOf("name: Restore tracked install-time changes"));
|
||||
assert.ok(publish.indexOf("name: Restore tracked install-time changes") < publish.indexOf("name: Publish canary"));
|
||||
assert.match(smoke, /name: Restore canary smoke lockfile\n\s+uses: actions\/download-artifact@[0-9a-f]{40} # v4\n\s+with:\n\s+name: canary-smoke-lockfile-\$\{\{ github\.sha \}\}/);
|
||||
assert.match(smoke, /name: Restore canary smoke lockfile\n\s+uses: actions\/download-artifact@[0-9a-f]{40} # v4\n\s+with:\n\s+name: canary-smoke-lockfile-\$\{\{ needs\.confirm_canary\.outputs\.source_sha \}\}\n\s+run-id: \$\{\{ needs\.confirm_canary\.outputs\.release_run_id \}\}\n\s+github-token: \$\{\{ github\.token \}\}/);
|
||||
assert.ok(smoke.indexOf("name: Restore canary smoke lockfile") < smoke.indexOf("name: Install test dependencies"));
|
||||
assert.doesNotMatch(smoke, /run-id:|github-token:|repository:|continue-on-error|--no-frozen-lockfile/);
|
||||
assert.doesNotMatch(smoke, /repository:|continue-on-error|--no-frozen-lockfile/);
|
||||
});
|
||||
|
||||
for (const drift of ["patch", "manifest"]) {
|
||||
@@ -429,9 +416,7 @@ for (const drift of ["patch", "manifest"]) {
|
||||
"--- a/index.js", "+++ b/index.js", "@@ -1 +1 @@",
|
||||
'-module.exports = "original";', `+module.exports = "${value}";`, "",
|
||||
].join("\n"));
|
||||
const workflow = readWorkflow("release.yml");
|
||||
const publish = workflow.split(" publish_canary:\n")[1].split(" smoke_canary_onboarding:\n")[0];
|
||||
const smoke = workflow.split(" smoke_canary_onboarding:\n")[1].split(" # ----- Nightly lane")[0];
|
||||
const { publish, smoke } = canaryJobs();
|
||||
const installArgs = (name) => {
|
||||
const job = name === "Install dependencies" ? publish : smoke;
|
||||
const command = job.match(new RegExp(`name: ${name}\\n\\s+run: (pnpm install[^\\n]+)`))?.[1];
|
||||
|
||||
Executable
+65
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
# shellcheck source=./release-lib.sh
|
||||
. "$REPO_ROOT/scripts/release-lib.sh"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/confirm-canary-publication.sh <canary-version> <source-sha>
|
||||
|
||||
Confirms a canary that ./scripts/release.sh canary --defer-registry-confirmation
|
||||
published, then creates and pushes its canary/v<version> tag at <source-sha>.
|
||||
Run it from a checkout of <source-sha>. It waits for every public package
|
||||
version to become registry-visible, requires npm provenance from the master
|
||||
release workflow at <source-sha>, checks dist-tags and internal dependencies
|
||||
(the canary dist-tag may already name a newer canary), and installs the exact
|
||||
version into a clean prefix. A tag that already points at <source-sha> is
|
||||
accepted, so a rerun is safe.
|
||||
EOF
|
||||
}
|
||||
|
||||
[ $# -eq 2 ] || { usage; exit 1; }
|
||||
version="$1"
|
||||
source_sha="$2"
|
||||
|
||||
[[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+-canary\.[0-9]+$ ]] || release_fail "not a canary version: $version"
|
||||
[[ "$source_sha" =~ ^[0-9a-f]{40}$ ]] || release_fail "not a full commit SHA: $source_sha"
|
||||
[ "$(git -C "$REPO_ROOT" rev-parse HEAD)" = "$source_sha" ] || release_fail "checkout is not $source_sha"
|
||||
require_canary_number_from_master_commit "$version" "$source_sha"
|
||||
|
||||
tag_name="$(canary_tag_name "$version")"
|
||||
PUBLISH_REMOTE="$(resolve_release_remote)"
|
||||
|
||||
PACKAGE_INFO=""
|
||||
PACKAGE_NAMES=()
|
||||
while IFS=$'\t' read -r pkg_dir pkg_name _pkg_version; do
|
||||
[ -n "$pkg_name" ] || continue
|
||||
PACKAGE_NAMES+=("$pkg_name")
|
||||
PACKAGE_INFO+="${pkg_dir}"$'\t'"${pkg_name}"$'\t'"${version}"$'\n'
|
||||
done < <(list_public_package_info)
|
||||
[ "${#PACKAGE_NAMES[@]}" -gt 0 ] || release_fail "no public packages were found in the workspace."
|
||||
|
||||
release_info "==> Waiting for registry visibility of ${#PACKAGE_NAMES[@]} packages at ${version}..."
|
||||
wait_for_npm_package_versions "${NPM_PUBLISH_VERIFY_ATTEMPTS:-12}" "${NPM_PUBLISH_VERIFY_DELAY_SECONDS:-5}" "$PACKAGE_INFO" \
|
||||
|| release_fail "npm did not expose every package at ${version}"
|
||||
|
||||
release_info "==> Checking npm provenance..."
|
||||
node "$REPO_ROOT/scripts/verify-canary-provenance.mjs" "$version" "$source_sha" "${PACKAGE_NAMES[@]}" \
|
||||
|| release_fail "npm provenance does not tie ${version} to the master release workflow at ${source_sha}"
|
||||
|
||||
release_info "==> Checking dist-tags and internal dependencies..."
|
||||
verify_args=(--channel canary --dist-tag canary --target-version "$version" --allow-newer-canary)
|
||||
for pkg_name in "${PACKAGE_NAMES[@]}"; do verify_args+=(--package "$pkg_name"); done
|
||||
if wait_for_release_registry_state "${NPM_REGISTRY_STATE_VERIFY_ATTEMPTS:-12}" "${NPM_REGISTRY_STATE_VERIFY_DELAY_SECONDS:-5}" "${verify_args[@]}"; then
|
||||
:
|
||||
else
|
||||
release_fail "npm dist-tags or registry metadata are wrong or never converged for ${version}"
|
||||
fi
|
||||
|
||||
release_info "==> Installing paperclipai@${version} into a clean prefix..."
|
||||
verify_npm_installable "paperclipai@${version}" "$version" || release_fail "paperclipai@${version} did not install cleanly"
|
||||
|
||||
push_release_tag_at "$PUBLISH_REMOTE" "$tag_name" "$source_sha"
|
||||
@@ -270,6 +270,89 @@ next_canary_version() {
|
||||
next_prerelease_version canary "$stable_version" "$@"
|
||||
}
|
||||
|
||||
canary_source_position() {
|
||||
local repo_root="$1"
|
||||
if [ "$(git -C "$repo_root" rev-parse --is-shallow-repository)" = "true" ]; then
|
||||
return 1
|
||||
fi
|
||||
git -C "$repo_root" rev-list --count HEAD
|
||||
}
|
||||
|
||||
next_canary_version_from_source() {
|
||||
local repo_root="$1"
|
||||
local allow_registry_fallback="$2"
|
||||
local stable_version="$3"
|
||||
local run_attempt="$4"
|
||||
shift 4
|
||||
local position
|
||||
|
||||
if ! [[ "$run_attempt" =~ ^[1-9][0-9]?$ ]]; then
|
||||
release_fail "canary numbering needs a run attempt from 1 to 99, got '${run_attempt}'."
|
||||
fi
|
||||
if ! position="$(canary_source_position "$repo_root")"; then
|
||||
if [ "$allow_registry_fallback" = true ]; then
|
||||
release_warn "shallow checkout: canary number falls back to the next registry number."
|
||||
next_prerelease_version canary "$stable_version" "$@"
|
||||
printf '\n'
|
||||
return 0
|
||||
fi
|
||||
release_fail "canary numbering needs the full master history; this checkout is shallow."
|
||||
fi
|
||||
local candidate=$((position * 100 + run_attempt - 1))
|
||||
local range_end=$((position * 100 + 99))
|
||||
local published
|
||||
published="$(node -e '
|
||||
const [file, stable, low, high] = process.argv.slice(1);
|
||||
let cache = {};
|
||||
try { cache = JSON.parse(require("node:fs").readFileSync(file, "utf8")); } catch {}
|
||||
const pattern = new RegExp(`^${stable.replace(/\./g, "\\.")}-canary\\.(\\d+)$`);
|
||||
const numbers = new Set();
|
||||
for (const versions of Object.values(cache)) for (const version of Array.isArray(versions) ? versions : []) {
|
||||
const number = Number(pattern.exec(version)?.[1]);
|
||||
if (number >= Number(low) && number <= Number(high)) numbers.add(number);
|
||||
}
|
||||
process.stdout.write([...numbers].join(" "));
|
||||
' "${RELEASE_PACKAGE_VERSIONS_FILE:-}" "$stable_version" "$candidate" "$range_end")"
|
||||
while [[ " $published " == *" $candidate "* ]]; do
|
||||
candidate=$((candidate + 1))
|
||||
done
|
||||
[ "$candidate" -le "$range_end" ] || release_fail "every canary number for master position ${position} is already published."
|
||||
printf '%s-canary.%s\n' "$stable_version" "$candidate"
|
||||
}
|
||||
|
||||
require_canary_number_from_master_commit() {
|
||||
local version="$1"
|
||||
local source_sha="$2"
|
||||
local master_ref="${3:-origin/master}"
|
||||
local number="${version##*-canary.}"
|
||||
local position
|
||||
|
||||
git -C "$REPO_ROOT" merge-base --is-ancestor "$source_sha" "$master_ref" \
|
||||
|| release_fail "${source_sha} is not on ${master_ref}"
|
||||
position="$(canary_source_position "$REPO_ROOT")" || release_fail "canary confirmation needs the full master history; this checkout is shallow."
|
||||
[ "$((number / 100))" = "$position" ] \
|
||||
|| release_fail "${version} was not numbered from ${source_sha} (master position ${position})"
|
||||
release_info " ✓ ${version} was numbered from ${source_sha} (master position ${position})"
|
||||
}
|
||||
|
||||
push_release_tag_at() {
|
||||
local remote="$1"
|
||||
local tag_name="$2"
|
||||
local source_sha="$3"
|
||||
local existing_sha
|
||||
|
||||
existing_sha="$(git -C "$REPO_ROOT" ls-remote --tags "$remote" "refs/tags/${tag_name}^{}" | awk '{print $1}')"
|
||||
[ -n "$existing_sha" ] || existing_sha="$(git -C "$REPO_ROOT" ls-remote --tags "$remote" "refs/tags/${tag_name}" | awk '{print $1}')"
|
||||
if [ -n "$existing_sha" ]; then
|
||||
[ "$existing_sha" = "$source_sha" ] || release_fail "tag ${tag_name} already exists on ${existing_sha}, not ${source_sha}"
|
||||
release_info " ✓ ${tag_name} already points at ${source_sha}"
|
||||
return 0
|
||||
fi
|
||||
git -C "$REPO_ROOT" tag "$tag_name" "$source_sha"
|
||||
git -C "$REPO_ROOT" push "$remote" "refs/tags/${tag_name}"
|
||||
release_info " ✓ Pushed ${tag_name} at ${source_sha}"
|
||||
}
|
||||
|
||||
release_notes_file() {
|
||||
printf '%s/releases/v%s.md\n' "$REPO_ROOT" "$1"
|
||||
}
|
||||
|
||||
@@ -406,3 +406,94 @@ test("every publish job budgets for build time and four delayed packages", () =>
|
||||
assert.ok(timeoutMinutes * 60 > requiredSeconds, `${job} must leave time beyond build and polling`);
|
||||
}
|
||||
});
|
||||
|
||||
function canaryNumberFixture({ commits, publishedVersions, shallow = false }) {
|
||||
const dir = mkdtempSync(join(tmpdir(), "paperclip-canary-number-"));
|
||||
const source = join(dir, "source");
|
||||
mkdirSync(source);
|
||||
const git = (cwd, ...args) => execFileSync("git", args, { cwd, encoding: "utf8", env: { ...process.env, GIT_AUTHOR_NAME: "t", GIT_AUTHOR_EMAIL: "t@example.test", GIT_COMMITTER_NAME: "t", GIT_COMMITTER_EMAIL: "t@example.test" } });
|
||||
git(source, "init", "-q", "-b", "master");
|
||||
for (let index = 0; index < commits; index += 1) git(source, "commit", "-q", "--allow-empty", "-m", `c${index}`);
|
||||
let repo = source;
|
||||
if (shallow) {
|
||||
repo = join(dir, "shallow");
|
||||
git(dir, "clone", "-q", "--depth", "1", `file://${source}`, repo);
|
||||
}
|
||||
const versionsFile = join(dir, "versions.json");
|
||||
writeFileSync(versionsFile, JSON.stringify({ "@paperclipai/shared": publishedVersions, paperclipai: publishedVersions }));
|
||||
return { dir, repo, versionsFile, git };
|
||||
}
|
||||
|
||||
function nextCanary(fixture, { allowFallback = false, attempt = "1" } = {}) {
|
||||
const script = `source "${repoRoot}/scripts/release-lib.sh"; next_canary_version_from_source "$1" "$2" 2026.1116.0 "$3" @paperclipai/shared paperclipai`;
|
||||
return execFileSync("bash", ["-c", script, "next-canary", fixture.repo, String(allowFallback), attempt], {
|
||||
encoding: "utf8", env: { ...process.env, RELEASE_PACKAGE_VERSIONS_FILE: fixture.versionsFile }, stdio: ["ignore", "pipe", "pipe"],
|
||||
}).trim();
|
||||
}
|
||||
|
||||
test("a canary is numbered from its master position and run attempt, ignoring other commits' numbers", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 25, publishedVersions: ["2026.1116.0-canary.2600", "2026.1116.0-canary.9999"] });
|
||||
assert.equal(nextCanary(fixture), "2026.1116.0-canary.2500");
|
||||
assert.equal(nextCanary(fixture, { attempt: "2" }), "2026.1116.0-canary.2501");
|
||||
assert.equal(nextCanary(fixture, { attempt: "99" }), "2026.1116.0-canary.2598");
|
||||
});
|
||||
|
||||
test("publishing the same commit again outside a rerun moves to the next free number in its own range", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 25, publishedVersions: ["2026.1116.0-canary.2500", "2026.1116.0-canary.2501", "2026.1116.0-canary.2600", "2026.1115.0-canary.2502"] });
|
||||
assert.equal(nextCanary(fixture), "2026.1116.0-canary.2502");
|
||||
assert.equal(nextCanary(fixture, { attempt: "3" }), "2026.1116.0-canary.2502");
|
||||
assert.equal(nextCanary(fixture, { attempt: "4" }), "2026.1116.0-canary.2503");
|
||||
const full = canaryNumberFixture({ commits: 25, publishedVersions: Array.from({ length: 100 }, (_, index) => `2026.1116.0-canary.${2500 + index}`) });
|
||||
assert.throws(() => nextCanary(full), /every canary number for master position 25 is already published/);
|
||||
});
|
||||
|
||||
test("reruns of one commit never reach the next commit's numbers", () => {
|
||||
const earlier = canaryNumberFixture({ commits: 25, publishedVersions: [] });
|
||||
const later = canaryNumberFixture({ commits: 26, publishedVersions: [] });
|
||||
const earlierNumbers = new Set(Array.from({ length: 99 }, (_, index) => nextCanary(earlier, { attempt: String(index + 1) })));
|
||||
assert.equal(earlierNumbers.size, 99);
|
||||
assert.equal(earlierNumbers.has(nextCanary(later)), false);
|
||||
});
|
||||
|
||||
test("an invalid run attempt is refused", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 3, publishedVersions: [] });
|
||||
for (const attempt of ["0", "100", "", "1a"]) assert.throws(() => nextCanary(fixture, { attempt }), /run attempt from 1 to 99/);
|
||||
});
|
||||
|
||||
test("a publishing canary refuses a shallow checkout, and a dry run falls back to the registry number", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 5, publishedVersions: ["2026.1116.0-canary.7"], shallow: true });
|
||||
assert.throws(() => nextCanary(fixture), /needs the full master history/);
|
||||
assert.equal(nextCanary(fixture, { allowFallback: true }), "2026.1116.0-canary.8");
|
||||
});
|
||||
|
||||
test("a canary tag is pushed once, accepted again on the same commit, and refused on another commit", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 2, publishedVersions: [] });
|
||||
const remote = join(fixture.dir, "remote.git");
|
||||
fixture.git(fixture.dir, "init", "-q", "--bare", remote);
|
||||
fixture.git(fixture.repo, "remote", "add", "origin", remote);
|
||||
const [head, parent] = fixture.git(fixture.repo, "rev-list", "-2", "HEAD").trim().split("\n");
|
||||
const pushTag = (sha) => execFileSync("bash", ["-c", `REPO_ROOT="$1"; source "${repoRoot}/scripts/release-lib.sh"; push_release_tag_at origin canary/v2026.1116.0-canary.200 "$2"`, "push-tag", fixture.repo, sha],
|
||||
{ encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
|
||||
assert.match(pushTag(head), /Pushed canary\/v2026\.1116\.0-canary\.200/);
|
||||
assert.equal(fixture.git(fixture.dir, "--git-dir", remote, "rev-parse", "canary/v2026.1116.0-canary.200").trim(), head);
|
||||
assert.match(pushTag(head), /already points at/);
|
||||
fixture.git(fixture.repo, "tag", "-d", "canary/v2026.1116.0-canary.200");
|
||||
assert.throws(() => pushTag(parent), /already exists on [0-9a-f]{40}, not/);
|
||||
assert.equal(fixture.git(fixture.dir, "--git-dir", remote, "rev-parse", "canary/v2026.1116.0-canary.200").trim(), head);
|
||||
});
|
||||
|
||||
test("a canary version is accepted only for the master commit whose position numbered it", () => {
|
||||
const fixture = canaryNumberFixture({ commits: 25, publishedVersions: [] });
|
||||
const head = fixture.git(fixture.repo, "rev-parse", "HEAD").trim();
|
||||
fixture.git(fixture.repo, "update-ref", "refs/remotes/origin/master", head);
|
||||
const confirm = (version, sha) => execFileSync("bash", ["-c", `REPO_ROOT="$1"; source "${repoRoot}/scripts/release-lib.sh"; require_canary_number_from_master_commit "$2" "$3"`, "confirm", fixture.repo, version, sha],
|
||||
{ encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
|
||||
assert.match(confirm("2026.1116.0-canary.2500", head), /numbered from .* \(master position 25\)/);
|
||||
assert.match(confirm("2026.1116.0-canary.2503", head), /master position 25/);
|
||||
assert.throws(() => confirm("2026.1116.0-canary.2600", head), /was not numbered from/);
|
||||
assert.throws(() => confirm("2026.1116.0-canary.7", head), /was not numbered from/);
|
||||
fixture.git(fixture.repo, "checkout", "-q", "-b", "side");
|
||||
fixture.git(fixture.repo, "commit", "-q", "--allow-empty", "-m", "side");
|
||||
const side = fixture.git(fixture.repo, "rev-parse", "HEAD").trim();
|
||||
assert.throws(() => confirm("2026.1116.0-canary.2600", side), /is not on origin\/master/);
|
||||
});
|
||||
+35
-10
@@ -12,6 +12,7 @@ dry_run=false
|
||||
skip_verify=false
|
||||
print_version_only=false
|
||||
from_candidate=false
|
||||
defer_registry_confirmation=false
|
||||
notes_file_override=
|
||||
tag_name=""
|
||||
|
||||
@@ -20,7 +21,7 @@ cleanup_on_exit=false
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/release.sh <canary|nightly|beta|stable> [--date YYYY-MM-DD] [--dry-run] [--skip-verify] [--print-version] [--notes-file PATH]
|
||||
./scripts/release.sh <canary|nightly|beta|stable> [--date YYYY-MM-DD] [--dry-run] [--skip-verify] [--print-version] [--notes-file PATH] [--defer-registry-confirmation]
|
||||
|
||||
Examples:
|
||||
./scripts/release.sh canary
|
||||
@@ -36,7 +37,15 @@ Notes:
|
||||
- Stable versions use YYYY.MDD.P, where M is the UTC month, DD is the
|
||||
zero-padded UTC day, and P is the same-day stable patch slot.
|
||||
- Canary releases publish YYYY.MDD.P-canary.N under the npm dist-tag
|
||||
"canary" and create the git tag canary/vYYYY.MDD.P-canary.N.
|
||||
"canary" and create the git tag canary/vYYYY.MDD.P-canary.N. N is
|
||||
100 x the source commit's position on master (git rev-list --count HEAD)
|
||||
plus the run attempt minus 1 (GITHUB_RUN_ATTEMPT, default 1). Each commit
|
||||
owns its own range, so neither a rerun nor an overlapping canary can take
|
||||
another commit's number, and no registry lookup is involved.
|
||||
- --defer-registry-confirmation (published canary only) stops after npm
|
||||
accepts every publish. It skips the registry-visibility wait, the
|
||||
dist-tag check and the git tag; scripts/confirm-canary-publication.sh
|
||||
does those later, outside the release lock.
|
||||
- Nightly releases republish a commit that already shipped a canary (HEAD
|
||||
must carry a canary/v* tag) as YYYY.MDD.P-nightly.N under the npm
|
||||
dist-tag "nightly", with the git tag nightly/vYYYY.MDD.P-nightly.N.
|
||||
@@ -118,6 +127,7 @@ while [ $# -gt 0 ]; do
|
||||
--skip-verify) skip_verify=true ;;
|
||||
--print-version) print_version_only=true ;;
|
||||
--from-candidate) from_candidate=true ;;
|
||||
--defer-registry-confirmation) defer_registry_confirmation=true ;;
|
||||
--notes-file)
|
||||
shift
|
||||
[ $# -gt 0 ] || release_fail "--notes-file requires a path."
|
||||
@@ -147,6 +157,10 @@ if [ -n "$notes_file_override" ] && [ "$channel" != "stable" ]; then
|
||||
release_fail "--notes-file only applies to the stable channel."
|
||||
fi
|
||||
|
||||
if [ "$defer_registry_confirmation" = true ] && { [ "$channel" != "canary" ] || [ "$dry_run" = true ]; }; then
|
||||
release_fail "--defer-registry-confirmation only applies to a published canary."
|
||||
fi
|
||||
|
||||
PUBLISH_REMOTE="$(resolve_release_remote)"
|
||||
fetch_release_remote "$PUBLISH_REMOTE"
|
||||
|
||||
@@ -178,7 +192,7 @@ DIST_TAG="latest"
|
||||
|
||||
if [ "$channel" = "canary" ]; then
|
||||
require_on_master_branch
|
||||
TARGET_PUBLISH_VERSION="$(next_prerelease_version canary "$TARGET_STABLE_VERSION" "${PUBLIC_PACKAGE_NAMES[@]}")"
|
||||
TARGET_PUBLISH_VERSION="$(next_canary_version_from_source "$REPO_ROOT" "$dry_run" "$TARGET_STABLE_VERSION" "${GITHUB_RUN_ATTEMPT:-1}" "${PUBLIC_PACKAGE_NAMES[@]}")"
|
||||
DIST_TAG="canary"
|
||||
tag_name="$(prerelease_tag_name canary "$TARGET_PUBLISH_VERSION")"
|
||||
elif [ "$channel" = "nightly" ]; then
|
||||
@@ -422,19 +436,23 @@ else
|
||||
release_info " ✓ npm accepted the publish"
|
||||
done <<< "$VERSIONED_PACKAGE_INFO"
|
||||
release_info " ✓ npm accepted every package publish under dist-tag $DIST_TAG"
|
||||
release_info " Waiting for registry visibility of all packages concurrently..."
|
||||
if ! wait_for_npm_package_versions \
|
||||
"$VERIFY_ATTEMPTS" \
|
||||
"$VERIFY_DELAY_SECONDS" \
|
||||
"$VERSIONED_PACKAGE_INFO"; then
|
||||
release_fail "stopping release: npm did not expose every published version under dist-tag $DIST_TAG"
|
||||
if [ "$defer_registry_confirmation" = false ]; then
|
||||
release_info " Waiting for registry visibility of all packages concurrently..."
|
||||
if ! wait_for_npm_package_versions \
|
||||
"$VERIFY_ATTEMPTS" \
|
||||
"$VERIFY_DELAY_SECONDS" \
|
||||
"$VERSIONED_PACKAGE_INFO"; then
|
||||
release_fail "stopping release: npm did not expose every published version under dist-tag $DIST_TAG"
|
||||
fi
|
||||
release_info " ✓ Published the full package set under dist-tag $DIST_TAG"
|
||||
fi
|
||||
release_info " ✓ Published the full package set under dist-tag $DIST_TAG"
|
||||
fi
|
||||
|
||||
release_info ""
|
||||
if [ "$dry_run" = true ]; then
|
||||
release_info "==> Step 6/7: Skipping npm verification in dry-run mode..."
|
||||
elif [ "$defer_registry_confirmation" = true ]; then
|
||||
release_info "==> Step 6/7: Registry confirmation deferred to scripts/confirm-canary-publication.sh"
|
||||
else
|
||||
release_info "==> Step 6/7: Confirming npm package availability and dist-tag integrity..."
|
||||
REGISTRY_STATE_VERIFY_ATTEMPTS="${NPM_REGISTRY_STATE_VERIFY_ATTEMPTS:-12}"
|
||||
@@ -476,6 +494,11 @@ fi
|
||||
release_info ""
|
||||
if [ "$dry_run" = true ]; then
|
||||
release_info "==> Step 7/7: Dry run complete..."
|
||||
elif [ "$defer_registry_confirmation" = true ]; then
|
||||
release_info "==> Step 7/7: Tag deferred until npm confirms ${TARGET_PUBLISH_VERSION}"
|
||||
if [ -n "${PAPERCLIP_RELEASE_VERSION_FILE:-}" ]; then
|
||||
printf '%s\n' "$TARGET_PUBLISH_VERSION" > "$PAPERCLIP_RELEASE_VERSION_FILE"
|
||||
fi
|
||||
else
|
||||
release_info "==> Step 7/7: Creating git tag..."
|
||||
git -C "$REPO_ROOT" tag "$tag_name" "$CURRENT_SHA"
|
||||
@@ -485,6 +508,8 @@ fi
|
||||
release_info ""
|
||||
if [ "$dry_run" = true ]; then
|
||||
release_info "Dry run complete for $channel ${TARGET_PUBLISH_VERSION}."
|
||||
elif [ "$defer_registry_confirmation" = true ]; then
|
||||
release_info "Published $channel ${TARGET_PUBLISH_VERSION}; npm confirmation and the git tag follow in canary-finalize."
|
||||
else
|
||||
case "$channel" in
|
||||
canary|nightly|beta)
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
#!/usr/bin/env node
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
export const RELEASE_WORKFLOW = Object.freeze({
|
||||
repository: "https://github.com/paperclipai/paperclip",
|
||||
path: ".github/workflows/release.yml",
|
||||
ref: "refs/heads/master",
|
||||
});
|
||||
|
||||
const CANARY_VERSION = /^\d+\.\d+\.\d+-canary\.\d+$/;
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
|
||||
function slsaStatements(attestations) {
|
||||
return (attestations?.attestations ?? [])
|
||||
.filter((entry) => typeof entry?.predicateType === "string" && entry.predicateType.startsWith("https://slsa.dev/provenance/"))
|
||||
.map((entry) => {
|
||||
try {
|
||||
return JSON.parse(Buffer.from(entry.bundle?.dsseEnvelope?.payload ?? "", "base64").toString("utf8"));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
export function packageProvenance(attestations, { packageName, version, sourceSha }) {
|
||||
const subject = `pkg:npm/${packageName.replace(/^@/, "%40")}@${version}`;
|
||||
const statements = slsaStatements(attestations);
|
||||
if (statements.length === 0) return "none";
|
||||
const matches = statements.some((statement) => {
|
||||
const workflow = statement.predicate?.buildDefinition?.externalParameters?.workflow ?? {};
|
||||
const commit = statement.predicate?.buildDefinition?.resolvedDependencies?.[0]?.digest?.gitCommit;
|
||||
return statement.subject?.some((entry) => entry?.name === subject)
|
||||
&& workflow.repository === RELEASE_WORKFLOW.repository
|
||||
&& workflow.path === RELEASE_WORKFLOW.path
|
||||
&& workflow.ref === RELEASE_WORKFLOW.ref
|
||||
&& commit === sourceSha;
|
||||
});
|
||||
return matches ? "match" : "mismatch";
|
||||
}
|
||||
|
||||
const TRANSIENT = new Set([408, 425, 429, 500, 502, 503, 504]);
|
||||
|
||||
async function fetchAttestations(url, { fetcher, attempts, sleep }) {
|
||||
let lastError;
|
||||
for (let attempt = 1; attempt <= attempts; attempt += 1) {
|
||||
try {
|
||||
const response = await fetcher(url, { signal: AbortSignal.timeout(30_000) });
|
||||
if (response.status === 404) return null;
|
||||
if (response.ok) return await response.json();
|
||||
if (!TRANSIENT.has(response.status)) throw Object.assign(new Error(`HTTP ${response.status}`), { permanent: true });
|
||||
lastError = new Error(`HTTP ${response.status}`);
|
||||
} catch (error) {
|
||||
if (error.permanent) throw error;
|
||||
lastError = error;
|
||||
}
|
||||
if (attempt < attempts) await sleep(Math.min(30_000, 1_000 * 2 ** (attempt - 1)));
|
||||
}
|
||||
throw lastError;
|
||||
}
|
||||
|
||||
export async function verifyCanaryProvenance({ packages, version, sourceSha, fetcher = fetch, attempts = 5,
|
||||
sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)) }) {
|
||||
if (!CANARY_VERSION.test(version ?? "")) throw new Error(`not a canary version: ${version}`);
|
||||
if (!SHA.test(sourceSha ?? "")) throw new Error(`not a full commit SHA: ${sourceSha}`);
|
||||
if (!packages?.length) throw new Error("at least one package is required");
|
||||
const problems = [];
|
||||
const withoutProvenance = [];
|
||||
let matched = 0;
|
||||
for (const packageName of packages) {
|
||||
const url = `https://registry.npmjs.org/-/npm/v1/attestations/${encodeURIComponent(packageName).replace(/^%40/, "@")}@${version}`;
|
||||
let attestations;
|
||||
try {
|
||||
attestations = await fetchAttestations(url, { fetcher, attempts, sleep });
|
||||
} catch (error) {
|
||||
problems.push(`${packageName}@${version}: attestations unavailable (${error.message})`);
|
||||
continue;
|
||||
}
|
||||
const result = attestations ? packageProvenance(attestations, { packageName, version, sourceSha }) : "none";
|
||||
if (result === "match") matched += 1;
|
||||
else if (result === "none") withoutProvenance.push(packageName);
|
||||
else problems.push(`${packageName}@${version}: provenance does not name ${RELEASE_WORKFLOW.path} on ${RELEASE_WORKFLOW.ref} at ${sourceSha}`);
|
||||
}
|
||||
return { problems, matched, withoutProvenance };
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
const [version, sourceSha, ...packages] = process.argv.slice(2);
|
||||
try {
|
||||
const { problems, matched, withoutProvenance } = await verifyCanaryProvenance({ packages, version, sourceSha });
|
||||
if (problems.length) {
|
||||
for (const problem of problems) console.error(problem);
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
console.log(`Provenance for ${matched} of ${packages.length} packages names ${RELEASE_WORKFLOW.path} at ${sourceSha}.`);
|
||||
if (withoutProvenance.length) console.log(`Published without provenance (npm transparency-log retry): ${withoutProvenance.join(", ")}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(error.message);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { packageProvenance, verifyCanaryProvenance } from "./verify-canary-provenance.mjs";
|
||||
|
||||
const sha = "b".repeat(40);
|
||||
const version = "2026.1116.0-canary.2180000";
|
||||
|
||||
function attestation({ name = "%40paperclipai/shared", commit = sha, path = ".github/workflows/release.yml", ref = "refs/heads/master",
|
||||
repository = "https://github.com/paperclipai/paperclip", predicateType = "https://slsa.dev/provenance/v1" } = {}) {
|
||||
const statement = { subject: [{ name: `pkg:npm/${name}@${version}` }],
|
||||
predicate: { buildDefinition: { externalParameters: { workflow: { repository, path, ref } }, resolvedDependencies: [{ digest: { gitCommit: commit } }] } } };
|
||||
return { attestations: [{ predicateType, bundle: { dsseEnvelope: { payload: Buffer.from(JSON.stringify(statement)).toString("base64") } } }] };
|
||||
}
|
||||
|
||||
const check = (doc, packageName = "@paperclipai/shared") => packageProvenance(doc, { packageName, version, sourceSha: sha });
|
||||
const respond = (body, status = 200) => new Response(JSON.stringify(body), { status });
|
||||
const noSleep = async () => {};
|
||||
const registry = (byPackage) => async (url) => {
|
||||
const name = decodeURIComponent(new URL(url).pathname.split("/attestations/")[1]).replace(/@[^@]+$/, "");
|
||||
const entry = byPackage[name];
|
||||
return typeof entry === "function" ? entry() : entry ?? respond({}, 404);
|
||||
};
|
||||
|
||||
test("provenance from master's release workflow at the source commit matches, scoped or not", () => {
|
||||
assert.equal(check(attestation()), "match");
|
||||
assert.equal(check(attestation({ name: "paperclipai" }), "paperclipai"), "match");
|
||||
});
|
||||
|
||||
test("provenance from another commit, workflow, branch, repository or package is a mismatch", () => {
|
||||
for (const overrides of [{ commit: "c".repeat(40) }, { path: ".github/workflows/pr.yml" }, { ref: "refs/heads/feature" },
|
||||
{ repository: "https://github.com/someone/paperclip" }, { name: "%40paperclipai/server" }]) {
|
||||
assert.equal(check(attestation(overrides)), "mismatch", JSON.stringify(overrides));
|
||||
}
|
||||
});
|
||||
|
||||
test("a package published through the transparency-log retry has no provenance, which is allowed beside a match", async () => {
|
||||
assert.equal(check({ attestations: [] }), "none");
|
||||
assert.equal(check(attestation({ predicateType: "https://github.com/npm/attestation/tree/main/specs/publish/v0.1" })), "none");
|
||||
const result = await verifyCanaryProvenance({ packages: ["@paperclipai/shared", "paperclipai"], version, sourceSha: sha, sleep: noSleep,
|
||||
fetcher: registry({ paperclipai: respond(attestation({ name: "paperclipai" })) }) });
|
||||
assert.deepEqual(result, { problems: [], matched: 1, withoutProvenance: ["@paperclipai/shared"] });
|
||||
});
|
||||
|
||||
test("packages that all took the retry report no match and no problem, and any mismatch is a problem", async () => {
|
||||
const none = await verifyCanaryProvenance({ packages: ["@paperclipai/shared", "paperclipai"], version, sourceSha: sha, sleep: noSleep, fetcher: registry({}) });
|
||||
assert.deepEqual(none, { problems: [], matched: 0, withoutProvenance: ["@paperclipai/shared", "paperclipai"] });
|
||||
const mixed = await verifyCanaryProvenance({ packages: ["@paperclipai/shared", "paperclipai"], version, sourceSha: sha, sleep: noSleep,
|
||||
fetcher: registry({ "@paperclipai/shared": respond(attestation()), paperclipai: respond(attestation({ name: "paperclipai", commit: "c".repeat(40) })) }) });
|
||||
assert.equal(mixed.problems.length, 1);
|
||||
assert.match(mixed.problems[0], /^paperclipai@.*provenance does not name/);
|
||||
});
|
||||
|
||||
test("a brief npm failure is retried, and a lasting or permanent one is a problem", async () => {
|
||||
let calls = 0;
|
||||
const flaky = await verifyCanaryProvenance({ packages: ["@paperclipai/shared"], version, sourceSha: sha, sleep: noSleep,
|
||||
fetcher: registry({ "@paperclipai/shared": () => (++calls < 3 ? respond({}, 503) : respond(attestation())) }) });
|
||||
assert.deepEqual(flaky.problems, []);
|
||||
assert.equal(calls, 3);
|
||||
const lasting = await verifyCanaryProvenance({ packages: ["@paperclipai/shared"], version, sourceSha: sha, sleep: noSleep, attempts: 3,
|
||||
fetcher: async () => { throw new Error("socket hang up"); } });
|
||||
assert.match(lasting.problems[0], /attestations unavailable \(socket hang up\)/);
|
||||
let forbidden = 0;
|
||||
const permanent = await verifyCanaryProvenance({ packages: ["@paperclipai/shared"], version, sourceSha: sha, sleep: noSleep,
|
||||
fetcher: async () => { forbidden += 1; return respond({}, 403); } });
|
||||
assert.match(permanent.problems[0], /HTTP 403/);
|
||||
assert.equal(forbidden, 1);
|
||||
});
|
||||
|
||||
test("inputs are validated before any request", async () => {
|
||||
const fetcher = async () => { throw new Error("must not fetch"); };
|
||||
await assert.rejects(verifyCanaryProvenance({ packages: ["paperclipai"], version: "2026.1116.0-nightly.1", sourceSha: sha, fetcher }), /not a canary version/);
|
||||
await assert.rejects(verifyCanaryProvenance({ packages: ["paperclipai"], version, sourceSha: "abc", fetcher }), /not a full commit SHA/);
|
||||
});
|
||||
@@ -29,7 +29,7 @@ function usage() {
|
||||
process.stderr.write(
|
||||
[
|
||||
"Usage:",
|
||||
" node scripts/verify-release-registry-state.mjs --channel <canary|nightly|beta|stable> --dist-tag <tag> --target-version <version> --package <name> [--package <name> ...] [--allow-canary-latest]",
|
||||
" node scripts/verify-release-registry-state.mjs --channel <canary|nightly|beta|stable> --dist-tag <tag> --target-version <version> --package <name> [--package <name> ...] [--allow-canary-latest] [--allow-newer-canary]",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
@@ -41,6 +41,7 @@ function parseArgs(argv) {
|
||||
distTag: "",
|
||||
targetVersion: "",
|
||||
allowCanaryLatest: false,
|
||||
allowNewerCanary: false,
|
||||
packages: [],
|
||||
};
|
||||
|
||||
@@ -67,6 +68,9 @@ function parseArgs(argv) {
|
||||
case "--allow-canary-latest":
|
||||
options.allowCanaryLatest = true;
|
||||
break;
|
||||
case "--allow-newer-canary":
|
||||
options.allowNewerCanary = true;
|
||||
break;
|
||||
case "-h":
|
||||
case "--help":
|
||||
usage();
|
||||
@@ -96,6 +100,10 @@ function parseArgs(argv) {
|
||||
throw createExitError("--allow-canary-latest only applies to canary releases", EXIT_NON_RETRIABLE_FAILURE);
|
||||
}
|
||||
|
||||
if (options.allowNewerCanary && options.channel !== "canary") {
|
||||
throw createExitError("--allow-newer-canary only applies to canary releases", EXIT_NON_RETRIABLE_FAILURE);
|
||||
}
|
||||
|
||||
return options;
|
||||
}
|
||||
|
||||
@@ -258,6 +266,20 @@ function requireManifest(packageName, version, packageDoc, packageManifestsByKey
|
||||
return manifest;
|
||||
}
|
||||
|
||||
export function compareCanaryVersions(left, right) {
|
||||
const parse = (version) => {
|
||||
const match = /^(\d+)\.(\d+)\.(\d+)-canary\.(\d+)$/.exec(version ?? "");
|
||||
return match ? match.slice(1).map(Number) : null;
|
||||
};
|
||||
const a = parse(left);
|
||||
const b = parse(right);
|
||||
if (!a || !b) return null;
|
||||
for (let index = 0; index < a.length; index += 1) {
|
||||
if (a[index] !== b[index]) return a[index] < b[index] ? -1 : 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
export function verifyPackageRegistryProblems({
|
||||
packageName,
|
||||
packageDoc,
|
||||
@@ -267,12 +289,14 @@ export function verifyPackageRegistryProblems({
|
||||
distTag,
|
||||
targetVersion,
|
||||
allowCanaryLatest,
|
||||
allowNewerCanary = false,
|
||||
}) {
|
||||
const problems = [];
|
||||
const distTags = packageDoc["dist-tags"] ?? {};
|
||||
const taggedVersion = distTags[distTag];
|
||||
const taggedNewerCanary = allowNewerCanary && channel === "canary" && compareCanaryVersions(taggedVersion, targetVersion) === 1;
|
||||
|
||||
if (taggedVersion !== targetVersion) {
|
||||
if (taggedVersion !== targetVersion && !taggedNewerCanary) {
|
||||
problems.push(
|
||||
createProblem(
|
||||
`${packageName}: dist-tag ${distTag} resolves to ${taggedVersion ?? "<missing>"}, expected ${targetVersion}`,
|
||||
@@ -444,6 +468,7 @@ async function main() {
|
||||
distTag: options.distTag,
|
||||
targetVersion: options.targetVersion,
|
||||
allowCanaryLatest: options.allowCanaryLatest,
|
||||
allowNewerCanary: options.allowNewerCanary,
|
||||
});
|
||||
|
||||
if (packageProblems.length === 0) {
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
isPrereleaseVersion,
|
||||
verifyPackageRegistryProblems,
|
||||
verifyPackageRegistryState,
|
||||
compareCanaryVersions,
|
||||
} from "./verify-release-registry-state.mjs";
|
||||
|
||||
test("isCanaryVersion matches release canaries", () => {
|
||||
@@ -399,3 +400,27 @@ test("fetchRegistryJson times out hung requests", async () => {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("a finalizing canary accepts a dist-tag that a newer canary already moved, and nothing older or foreign", () => {
|
||||
const check = (canaryTag, allowNewerCanary, channel = "canary") => {
|
||||
const packageDoc = { "dist-tags": { latest: "2026.1110.0", canary: canaryTag }, versions: { "2026.1116.0-canary.21800": {} } };
|
||||
return verifyPackageRegistryProblems({
|
||||
packageName: "@paperclipai/shared", packageDoc, packageDocsByName: new Map([["@paperclipai/shared", packageDoc]]),
|
||||
channel, distTag: "canary", targetVersion: "2026.1116.0-canary.21800", allowCanaryLatest: false, allowNewerCanary,
|
||||
}).map((problem) => problem.message);
|
||||
};
|
||||
assert.deepEqual(check("2026.1116.0-canary.21800", true), []);
|
||||
assert.deepEqual(check("2026.1116.0-canary.21805", true), []);
|
||||
assert.deepEqual(check("2026.1117.0-canary.21803", true), []);
|
||||
assert.match(check("2026.1116.0-canary.21805", false)[0], /resolves to 2026\.1116\.0-canary\.21805, expected/);
|
||||
assert.match(check("2026.1116.0-canary.21799", true)[0], /resolves to 2026\.1116\.0-canary\.21799, expected/);
|
||||
assert.match(check("2026.1116.0-nightly.3", true)[0], /resolves to 2026\.1116\.0-nightly\.3, expected/);
|
||||
assert.match(check(undefined, true)[0], /resolves to <missing>/);
|
||||
});
|
||||
|
||||
test("canary versions compare by base version, then canary number", () => {
|
||||
assert.equal(compareCanaryVersions("2026.1116.0-canary.9", "2026.1116.0-canary.21800"), -1);
|
||||
assert.equal(compareCanaryVersions("2026.1117.0-canary.1", "2026.1116.0-canary.21800"), 1);
|
||||
assert.equal(compareCanaryVersions("2026.1116.0-canary.7", "2026.1116.0-canary.7"), 0);
|
||||
assert.equal(compareCanaryVersions("2026.1116.0-nightly.7", "2026.1116.0-canary.7"), null);
|
||||
});
|
||||
Reference in new issue
Block a user