Files
PaperClipAI/.github/workflows/release.yml
T
Devin Foley f9173782cd feat(release): add smoke-gated nightly channel and lane-separated Docker tags (#11006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem publishes the `paperclipai` npm package set
and the Docker images on two lanes: canary on every master push, and
stable on manual promotion
> - There is no middle ground between those lanes. Users must track
every merge or wait weeks for a stable. Docker `:latest` also tracks
master, so Docker users have no stable image at all
> - A calm prerelease lane needs to exist, and it must never ship a
build that failed its checks
> - This pull request adds the nightly channel: a scheduled job that
selects the newest master commit with a green canary publish, runs the
full release smoke suite against that exact published canary, and only
then republishes it as the nightly. It also separates Docker tags by
lane, so `:latest` finally means stable
> - The benefit is that users can follow prereleases at a nightly
cadence with a smoke-tested guarantee, and Docker users get real
`:canary`, `:nightly`, and stable image tags

## Linked Issues or Issue Description

**Subsystem affected**

Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

The project publishes only `canary` (every master push) and `latest`
(manual stable). Users who want prereleases without per-merge churn have
no option. Docker has a second problem: master builds overwrite
`:latest`, and CI-published stables never produced Docker images,
because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger
in `docker.yml`. No stable-versioned image exists in ghcr today.

**Proposed solution**

Add a `nightly` channel. A scheduled job selects the newest
canary-tagged master commit, smoke-tests that exact published canary,
and republishes the same commit as `YYYY.MDD.P-nightly.N` under the
`nightly` dist-tag. Separate Docker tags by lane (`:canary` for master,
`:nightly` for nightly tags, `:latest` plus version tags for stable tags
only), and have the release jobs dispatch `docker.yml` at the new tag so
lane images actually build.

**Alternatives considered**

Moving the `nightly` dist-tag to the existing canary version without a
republish. Rejected: the version string would say `canary` while the
user is on nightly, which breaks at-a-glance lane identification in bug
reports and `--version` output.

## What Changed

- `scripts/release-lib.sh`: channel-parameterized
`next_prerelease_version` and `prerelease_tag_name` helpers (canary
helpers delegate to them), a `require_channel_tag_at_head` guard, and
the no-provenance retry for Sigstore transparency-log duplicates now
covers the `nightly` dist-tag as well as `canary`
- `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry
a `canary/v*` tag, publishes the full public package set as
`YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source
commit `nightly/vYYYY.MDD.P-nightly.N`
- `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) —
select candidate, smoke it via `release-smoke.yml`, publish on green
under the existing `npm-canary` environment, push the tag, dispatch
`docker.yml`. New `channel` dispatch input (default `stable`, so
existing stable dispatches are unchanged) with `nightly_source_version`
and `dry_run` support for forced runs. The stable path now also
dispatches `docker.yml` at the new `v*` tag
- `.github/workflows/docker.yml`: lane tag mapping for both image jobs —
master pushes publish `:canary` and no longer move `:latest`;
`nightly/v*` tags publish `:nightly`; only stable `v*` tags publish
`:latest` and the versioned tags. New `workflow_dispatch` trigger for
the release-job dispatches. Build-version stamping uses the exact
nightly version on nightly tag builds
- `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch
choice list
- `doc/CHANNELS.md` (new): user-facing guide to the channels
- `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping
table, and a nightly failure playbook
- `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses
`npm-canary` and needs no npm trusted-publisher changes
- Tests: channel-parameterized version helper coverage in
`scripts/release-registry-versions.test.mjs`, and nightly flow coverage
(publish identity, notes not required, canary-tag guard) in
`scripts/__tests__/release-dry-run-notes.test.mjs`

## Verification

- `node --test` on the release script suites: 68 pass, including 6 new
tests. The only failure, `acpx-patch-packaging.test.mjs`, needs
installed `node_modules` and fails identically on a pristine checkout of
master in the same environment
- `bash -n` on both shell scripts and YAML parse of all three workflows
- Live fail-path check: `./scripts/release.sh nightly --print-version`
from a master tip with no canary tag fails with `HEAD has no canary/v*
tag`
- Live success-path check: the same command from the
`canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0`
- Live selection check: the candidate-selection shell logic run against
the real repository selects the commit of `canary/v2026.806.0-canary.7`,
which matches the current npm `canary` dist-tag exactly
- After merge: dispatch `release.yml` with `channel: nightly` and
`dry_run: true` to preview, then a real forced run to validate end to
end before the first scheduled run

## Risks

- Docker `:latest` changes meaning from "latest master build" to "latest
stable release". This is deliberate and will be announced. Users who
want the old behavior pull `:canary`. Until the first stable release
after this change, `:latest` stays at its current (master-built) image
- The nightly is a rebuild of the same source commit, not the
byte-identical canary artifact that was smoked. The lockfile pins
dependencies, and the publish path's registry-visibility and
clean-prefix install gates still run on the nightly artifacts
- All npm publishing must stay inside `release.yml` because npm trusted
publishing pins that workflow file per package. The nightly jobs were
added to `release.yml` for exactly that reason; this constraint is now
documented in `RELEASING.md`
- The stable-lane Docker dispatch fails gracefully (a warning with
manual instructions) when the source ref predates `docker.yml`'s
`workflow_dispatch` trigger

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 12:16:55 -07:00

466 lines
16 KiB
YAML

name: Release
on:
push:
branches:
- master
schedule:
# Nightly cut at 09:00 UTC, after the workday's merges have settled.
- cron: "0 9 * * *"
workflow_dispatch:
inputs:
channel:
description: Release channel to publish
required: true
type: choice
options:
- stable
- nightly
default: stable
source_ref:
description: (stable) Commit SHA, branch, or tag to publish as stable
required: true
type: string
default: master
stable_date:
description: Enter a UTC date in YYYY-MM-DD format, for example 2026-03-18. Do not enter a version string. The workflow will resolve that date to a stable version such as 2026.318.0, then 2026.318.1 for the next same-day stable.
required: false
type: string
source_version:
description: (nightly) Explicit canary version to promote, for example 2026.806.0-canary.7. Leave empty to select the newest canary on master.
required: false
type: string
dry_run:
description: Preview the release without publishing
required: true
type: boolean
default: false
concurrency:
group: release-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: false
jobs:
verify_canary:
if: github.event_name == 'push'
uses: ./.github/workflows/release-verify.yml
with:
ref: ${{ github.sha }}
publish_canary:
if: github.event_name == 'push'
needs: verify_canary
runs-on: ubuntu-latest
timeout-minutes: 45
environment: npm-canary
permissions:
contents: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Validate release package manifest
run: node ./scripts/release-package-map.mjs check
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Restore tracked install-time changes
run: git checkout -- pnpm-lock.yaml
- name: Configure git author
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: Publish canary
env:
GITHUB_ACTIONS: "true"
run: ./scripts/release.sh canary --skip-verify
- name: Dump npm debug logs
if: failure()
run: |
shopt -s nullglob
for f in "$HOME"/.npm/_logs/*.log; do
echo "===== $f ====="
tail -n 300 "$f" | sed -E \
-e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig'
done
- name: Push canary tag
run: |
tag="$(git tag --points-at HEAD | grep '^canary/v' | head -1)"
if [ -z "$tag" ]; then
echo "Error: no canary tag points at HEAD after release." >&2
exit 1
fi
git push origin "refs/tags/${tag}"
# ----- Nightly lane -----------------------------------------------------
# Once a night (or on a forced nightly dispatch), promote the newest master
# commit that already shipped a green canary: smoke-test that exact
# published canary first, then republish the same commit under the nightly
# identity. The candidate commit already passed release-verify during its
# canary publish, so the nightly publish skips re-verification.
select_nightly:
if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.channel == 'nightly')
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
outputs:
proceed: ${{ steps.select.outputs.proceed }}
sha: ${{ steps.select.outputs.sha }}
canary_version: ${{ steps.select.outputs.canary_version }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
ref: master
fetch-depth: 0
- name: Select nightly candidate
id: select
env:
EXPLICIT_CANARY_VERSION: ${{ github.event_name == 'workflow_dispatch' && inputs.source_version || '' }}
run: |
set -euo pipefail
git fetch origin --tags --prune --quiet
skip() {
echo "proceed=false" >> "$GITHUB_OUTPUT"
{
echo "## Nightly skipped"
echo ""
echo "$1"
} >> "$GITHUB_STEP_SUMMARY"
echo "Nightly skipped: $1"
}
if [ -n "${EXPLICIT_CANARY_VERSION:-}" ]; then
tag="canary/v${EXPLICIT_CANARY_VERSION}"
sha="$(git rev-list -n 1 "$tag" 2>/dev/null || true)"
if [ -z "$sha" ]; then
echo "Error: tag $tag does not exist." >&2
exit 1
fi
else
# Newest canary-tagged commit on master. Canary tags are pushed
# only after a successful canary publish, so tag presence is the
# green-publish signal. Walk master newest-first and stop at the
# first commit that carries a canary tag.
sha="$(grep -m1 -F \
-f <(git for-each-ref 'refs/tags/canary/v*' --format='%(objectname)') \
<(git rev-list origin/master -n 500) || true)"
if [ -z "$sha" ]; then
skip "No canary/v* tag found on the last 500 commits of master."
exit 0
fi
tag="$(git tag --points-at "$sha" | grep '^canary/v' | sort -V | tail -1)"
fi
canary_version="${tag#canary/v}"
existing_nightly="$(git tag --points-at "$sha" | grep '^nightly/v' | head -1 || true)"
if [ -n "$existing_nightly" ]; then
skip "Candidate \`$sha\` (canary \`$canary_version\`) already shipped as \`$existing_nightly\`."
exit 0
fi
echo "proceed=true" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "canary_version=$canary_version" >> "$GITHUB_OUTPUT"
{
echo "## Nightly candidate"
echo ""
echo "- Source SHA: \`$sha\`"
echo "- Source canary: \`$canary_version\`"
} >> "$GITHUB_STEP_SUMMARY"
# Gate the promotion on the release smoke suite, run against the exact
# published canary artifact that would become tonight's nightly. Red smoke
# means no nightly tonight. Skipped for dry-run dispatches.
smoke_nightly:
needs: select_nightly
if: needs.select_nightly.outputs.proceed == 'true' && !(github.event_name == 'workflow_dispatch' && inputs.dry_run)
uses: ./.github/workflows/release-smoke.yml
with:
paperclip_version: ${{ needs.select_nightly.outputs.canary_version }}
artifact_name: nightly-release-smoke
publish_nightly:
needs: [select_nightly, smoke_nightly]
# Publish when smoke passed, or when smoke was deliberately skipped by a
# dry-run dispatch (the publish itself is a dry-run in that case).
if: >-
!cancelled() &&
needs.select_nightly.outputs.proceed == 'true' &&
(needs.smoke_nightly.result == 'success' ||
(needs.smoke_nightly.result == 'skipped' && github.event_name == 'workflow_dispatch' && inputs.dry_run))
runs-on: ubuntu-latest
timeout-minutes: 45
environment: npm-canary
# The workflow-level concurrency group is per event, so a forced dispatch
# nightly could otherwise overlap the scheduled one and race it to the
# same next -nightly.N version. Serialize actual nightly publishes across
# events here; release.sh additionally refuses to double-publish a commit
# that already carries a nightly tag.
concurrency:
group: release-publish-nightly
cancel-in-progress: false
permissions:
contents: write
id-token: write
actions: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
ref: ${{ needs.select_nightly.outputs.sha }}
fetch-depth: 0
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Validate release package manifest
run: node ./scripts/release-package-map.mjs check
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Restore tracked install-time changes
run: git checkout -- pnpm-lock.yaml
- name: Configure git author
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: Publish nightly
env:
GITHUB_ACTIONS: "true"
run: |
args=(nightly --skip-verify)
if [ "${{ github.event_name == 'workflow_dispatch' && inputs.dry_run }}" = "true" ]; then
args+=(--dry-run)
fi
./scripts/release.sh "${args[@]}"
- name: Dump npm debug logs
if: failure()
run: |
shopt -s nullglob
for f in "$HOME"/.npm/_logs/*.log; do
echo "===== $f ====="
tail -n 300 "$f" | sed -E \
-e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig'
done
- name: Push nightly tag
if: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }}
run: |
tag="$(git tag --points-at HEAD | grep '^nightly/v' | head -1)"
if [ -z "$tag" ]; then
echo "Error: no nightly tag points at HEAD after release." >&2
exit 1
fi
git push origin "refs/tags/${tag}"
# Tag pushes made with GITHUB_TOKEN do not fire docker.yml's tag
# trigger (GitHub suppresses workflow runs caused by GITHUB_TOKEN
# pushes), so dispatch the image build at the new tag explicitly.
- name: Build Docker images for the nightly tag
if: ${{ !(github.event_name == 'workflow_dispatch' && inputs.dry_run) }}
env:
GH_TOKEN: ${{ github.token }}
run: |
tag="$(git tag --points-at HEAD | grep '^nightly/v' | head -1)"
{
echo "## Nightly published"
echo ""
echo "- Source SHA: \`${{ needs.select_nightly.outputs.sha }}\`"
echo "- Source canary: \`${{ needs.select_nightly.outputs.canary_version }}\`"
echo "- Published nightly: \`${tag#nightly/v}\`"
echo "- Docker build dispatched at \`${tag}\`"
} >> "$GITHUB_STEP_SUMMARY"
gh workflow run docker.yml --ref "refs/tags/${tag}" --repo "$GITHUB_REPOSITORY"
# ----- Stable lane ------------------------------------------------------
verify_stable:
if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable'
uses: ./.github/workflows/release-verify.yml
with:
ref: ${{ inputs.source_ref }}
preview_stable:
if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' && inputs.dry_run
needs: verify_stable
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0
ref: ${{ inputs.source_ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Validate release package manifest
run: node ./scripts/release-package-map.mjs check
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Dry-run stable release
env:
GITHUB_ACTIONS: "true"
run: |
args=(stable --skip-verify --dry-run)
if [ -n "${{ inputs.stable_date }}" ]; then
args+=(--date "${{ inputs.stable_date }}")
fi
./scripts/release.sh "${args[@]}"
publish_stable:
if: github.event_name == 'workflow_dispatch' && inputs.channel == 'stable' && !inputs.dry_run
needs: verify_stable
runs-on: ubuntu-latest
timeout-minutes: 45
environment: npm-stable
permissions:
contents: write
id-token: write
actions: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0
ref: ${{ inputs.source_ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Restore tracked install-time changes
run: git checkout -- pnpm-lock.yaml
- name: Configure git author
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: Publish stable
env:
GITHUB_ACTIONS: "true"
run: |
args=(stable --skip-verify)
if [ -n "${{ inputs.stable_date }}" ]; then
args+=(--date "${{ inputs.stable_date }}")
fi
./scripts/release.sh "${args[@]}"
- name: Dump npm debug logs
if: failure()
run: |
shopt -s nullglob
for f in "$HOME"/.npm/_logs/*.log; do
echo "===== $f ====="
tail -n 300 "$f" | sed -E \
-e 's#((authorization|_authToken|_auth|node_auth_token|npm_token)"?[[:space:]]*[:=][[:space:]]*"?)(Bearer[[:space:]]+)?[^",[:space:]]+#\1***REDACTED***#Ig'
done
- name: Push stable tag
run: |
tag="$(git tag --points-at HEAD | grep '^v' | head -1)"
if [ -z "$tag" ]; then
echo "Error: no stable tag points at HEAD after release." >&2
exit 1
fi
git push origin "refs/tags/${tag}"
# Tag pushes made with GITHUB_TOKEN do not fire docker.yml's tag
# trigger (GitHub suppresses workflow runs caused by GITHUB_TOKEN
# pushes), so dispatch the image build at the new tag explicitly. This
# is what moves Docker `:latest` and publishes the versioned stable
# image tags.
- name: Build Docker images for the stable tag
env:
GH_TOKEN: ${{ github.token }}
run: |
tag="$(git tag --points-at HEAD | grep '^v' | head -1)"
if gh workflow run docker.yml --ref "refs/tags/${tag}" --repo "$GITHUB_REPOSITORY"; then
echo "Dispatched docker.yml at ${tag}."
else
# Older source commits may predate docker.yml's workflow_dispatch
# trigger; the dispatch then fails while the npm release is
# already complete and correct.
echo "::warning::Could not dispatch docker.yml at ${tag}. Run docker.yml manually at that tag to publish the stable images."
fi
- name: Create GitHub Release
env:
GH_TOKEN: ${{ github.token }}
PUBLISH_REMOTE: origin
run: |
version="$(git tag --points-at HEAD | grep '^v' | head -1 | sed 's/^v//')"
if [ -z "$version" ]; then
echo "Error: no v* tag points at HEAD after stable release." >&2
exit 1
fi
./scripts/create-github-release.sh "$version"