## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release subsystem publishes the `paperclipai` npm package set and the Docker images on two lanes: canary on every master push, and stable on manual promotion > - There is no middle ground between those lanes. Users must track every merge or wait weeks for a stable. Docker `:latest` also tracks master, so Docker users have no stable image at all > - A calm prerelease lane needs to exist, and it must never ship a build that failed its checks > - This pull request adds the nightly channel: a scheduled job that selects the newest master commit with a green canary publish, runs the full release smoke suite against that exact published canary, and only then republishes it as the nightly. It also separates Docker tags by lane, so `:latest` finally means stable > - The benefit is that users can follow prereleases at a nightly cadence with a smoke-tested guarantee, and Docker users get real `:canary`, `:nightly`, and stable image tags ## Linked Issues or Issue Description **Subsystem affected** Release automation: `scripts/release.sh`, `scripts/release-lib.sh`, `.github/workflows/release.yml`, `.github/workflows/docker.yml`, `.github/workflows/release-smoke.yml`. **Problem or motivation** The project publishes only `canary` (every master push) and `latest` (manual stable). Users who want prereleases without per-merge churn have no option. Docker has a second problem: master builds overwrite `:latest`, and CI-published stables never produced Docker images, because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger in `docker.yml`. No stable-versioned image exists in ghcr today. **Proposed solution** Add a `nightly` channel. A scheduled job selects the newest canary-tagged master commit, smoke-tests that exact published canary, and republishes the same commit as `YYYY.MDD.P-nightly.N` under the `nightly` dist-tag. Separate Docker tags by lane (`:canary` for master, `:nightly` for nightly tags, `:latest` plus version tags for stable tags only), and have the release jobs dispatch `docker.yml` at the new tag so lane images actually build. **Alternatives considered** Moving the `nightly` dist-tag to the existing canary version without a republish. Rejected: the version string would say `canary` while the user is on nightly, which breaks at-a-glance lane identification in bug reports and `--version` output. ## What Changed - `scripts/release-lib.sh`: channel-parameterized `next_prerelease_version` and `prerelease_tag_name` helpers (canary helpers delegate to them), a `require_channel_tag_at_head` guard, and the no-provenance retry for Sigstore transparency-log duplicates now covers the `nightly` dist-tag as well as `canary` - `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry a `canary/v*` tag, publishes the full public package set as `YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source commit `nightly/vYYYY.MDD.P-nightly.N` - `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) — select candidate, smoke it via `release-smoke.yml`, publish on green under the existing `npm-canary` environment, push the tag, dispatch `docker.yml`. New `channel` dispatch input (default `stable`, so existing stable dispatches are unchanged) with `nightly_source_version` and `dry_run` support for forced runs. The stable path now also dispatches `docker.yml` at the new `v*` tag - `.github/workflows/docker.yml`: lane tag mapping for both image jobs — master pushes publish `:canary` and no longer move `:latest`; `nightly/v*` tags publish `:nightly`; only stable `v*` tags publish `:latest` and the versioned tags. New `workflow_dispatch` trigger for the release-job dispatches. Build-version stamping uses the exact nightly version on nightly tag builds - `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch choice list - `doc/CHANNELS.md` (new): user-facing guide to the channels - `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping table, and a nightly failure playbook - `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses `npm-canary` and needs no npm trusted-publisher changes - Tests: channel-parameterized version helper coverage in `scripts/release-registry-versions.test.mjs`, and nightly flow coverage (publish identity, notes not required, canary-tag guard) in `scripts/__tests__/release-dry-run-notes.test.mjs` ## Verification - `node --test` on the release script suites: 68 pass, including 6 new tests. The only failure, `acpx-patch-packaging.test.mjs`, needs installed `node_modules` and fails identically on a pristine checkout of master in the same environment - `bash -n` on both shell scripts and YAML parse of all three workflows - Live fail-path check: `./scripts/release.sh nightly --print-version` from a master tip with no canary tag fails with `HEAD has no canary/v* tag` - Live success-path check: the same command from the `canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0` - Live selection check: the candidate-selection shell logic run against the real repository selects the commit of `canary/v2026.806.0-canary.7`, which matches the current npm `canary` dist-tag exactly - After merge: dispatch `release.yml` with `channel: nightly` and `dry_run: true` to preview, then a real forced run to validate end to end before the first scheduled run ## Risks - Docker `:latest` changes meaning from "latest master build" to "latest stable release". This is deliberate and will be announced. Users who want the old behavior pull `:canary`. Until the first stable release after this change, `:latest` stays at its current (master-built) image - The nightly is a rebuild of the same source commit, not the byte-identical canary artifact that was smoked. The lockfile pins dependencies, and the publish path's registry-visibility and clean-prefix install gates still run on the nightly artifacts - All npm publishing must stay inside `release.yml` because npm trusted publishing pins that workflow file per package. The nightly jobs were added to `release.yml` for exactly that reason; this constraint is now documented in `RELEASING.md` - The stable-lane Docker dispatch fails gracefully (a warning with manual instructions) when the source ref predates `docker.yml`'s `workflow_dispatch` trigger ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use (repository exploration, local test execution, live registry and git verification). All code, tests, and docs in this PR were model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge
11 KiB
Releasing Paperclip
Maintainer runbook for shipping Paperclip across npm, GitHub, and the website-facing changelog surface.
The release model is now commit-driven:
- Every push to
masterpublishes a canary automatically. - Once a night, the newest master commit with a green canary publish is smoke-tested and republished as the nightly.
- Stable releases are manually promoted from a chosen tested commit or canary tag.
- Stable release notes live in
releases/vYYYY.MDD.P.md. - Only stable releases get GitHub Releases.
The user-facing guide to the channels is CHANNELS.md.
Versioning Model
Paperclip uses calendar versions that still fit semver syntax:
- stable:
YYYY.MDD.P - canary:
YYYY.MDD.P-canary.N - nightly:
YYYY.MDD.P-nightly.N
Examples:
- first stable on March 18, 2026:
2026.318.0 - second stable on March 18, 2026:
2026.318.1 - fourth canary for the
2026.318.1line:2026.318.1-canary.3 - first nightly cut on March 18, 2026:
2026.318.1-nightly.0
A nightly republishes the exact source commit of an existing canary; its version dates the nightly cut (the scheduled run's UTC date), not the source canary.
Important constraints:
- the middle numeric slot is
MDD, whereMis the UTC month andDDis the zero-padded UTC day - use
2026.303.0for March 3, not2026.33.0 - do not use leading zeroes such as
2026.0318.0 - do not use four numeric segments such as
2026.3.18.1 - the semver-safe canary form is
2026.318.0-canary.1
Release Surfaces
Every stable release has four separate surfaces:
- Verification — the exact git SHA passes typecheck, tests, and build
- npm —
paperclipaiand public workspace packages are published - GitHub — the stable release gets a git tag and GitHub Release
- Website / announcements — the stable changelog is published externally and announced
A stable release is done only when all four surfaces are handled.
Canaries and nightlies only cover the first two surfaces plus an internal traceability tag.
Core Invariants
- canaries publish from
master - nightlies republish a commit that already shipped a canary (the commit must
carry a
canary/v*tag), and only after the release smoke suite passes against that exact published canary - stables publish from an explicitly chosen source ref
- tags point at the original source commit, not a generated release commit
- stable notes are always
releases/vYYYY.MDD.P.md - canaries and nightlies never create GitHub Releases
- canaries and nightlies never require changelog generation
- Docker
:latestmoves only on stable releases; master builds publish:canaryand nightly builds publish:nightly
TL;DR
Canary
Every push to master runs the canary path inside .github/workflows/release.yml.
It:
- verifies the pushed commit
- computes the canary version for the current UTC date
- publishes workspace packages dependency-first under npm dist-tag
canary - waits for each package version to become registry-visible before continuing
- publishes the user-facing
paperclipaipackage last, sopaperclipai@canarydoes not advance before the full package set exists - verifies that
canaryresolves to the just-published version and that published internal dependencies exist on npm - installs
paperclipai@canaryinto a clean temporary prefix as the final npm gate - fails by default if npm leaves
latestpointing at a canary; use--allow-canary-latestonly when that state is intentional - creates a git tag
canary/vYYYY.MDD.P-canary.N
Users install canaries with:
npx paperclipai@canary onboard
# or
npx paperclipai@canary onboard --data-dir "$(mktemp -d /tmp/paperclip-canary.XXXXXX)"
Nightly
A scheduled job in .github/workflows/release.yml
runs once a night at 09:00 UTC.
It:
- selects the newest commit on
masterthat carries acanary/v*tag (the tag is pushed only after a successful canary publish, so it is the green-publish signal) - skips with a job-summary reason when there is no new candidate or the candidate already shipped as a nightly
- runs the release smoke suite (
release-smoke.yml) against that exact published canary version — red smoke means no nightly tonight - republishes the same source commit as
YYYY.MDD.P-nightly.Nunder the npm dist-tagnightly(the commit was already verified by its canary run, so verification is not repeated) - creates and pushes the git tag
nightly/vYYYY.MDD.P-nightly.N - dispatches
docker.ymlat that tag to publish the:nightlyimages
To force a nightly outside the schedule (recovery, or promoting a specific
canary), dispatch release.yml with channel: nightly. Leave
source_version empty for automatic selection, or set it to an exact
canary version. dry_run: true previews the publish and skips smoke, the tag
push, and the Docker dispatch.
Users install nightlies with:
npx paperclipai@nightly onboard
Stable
Use .github/workflows/release.yml from the Actions tab with the manual workflow_dispatch inputs.
Inputs:
channelstable(the default) for a stable release;nightlyforces a nightly run (see above)
source_ref- commit SHA, branch, or tag
stable_date- optional UTC date override in
YYYY-MM-DD - enter a date like
2026-03-18, not a version like2026.318.0
- optional UTC date override in
dry_run- preview only when true
Before running stable:
- pick the canary commit or tag you trust
- resolve the target stable version with
./scripts/release.sh stable --date "$(date +%F)" --print-version - create or update
releases/vYYYY.MDD.P.mdon that source ref - run the stable workflow from that source ref
Example:
source_ref:masterstable_date:2026-03-18- resulting stable version:
2026.318.0
The workflow:
- re-verifies the exact source ref
- computes the next stable patch slot for the chosen UTC date
- publishes
YYYY.MDD.Punder npm dist-taglatest - creates git tag
vYYYY.MDD.P - dispatches
docker.ymlat that tag to publish:latestand the versioned stable images - creates or updates the GitHub Release from
releases/vYYYY.MDD.P.md
Docker Image Tags
docker.yml publishes both the self-hosted
image and the -cloud variant with the same lane mapping:
| Build ref | Tags |
|---|---|
master push |
:canary, :sha-<short> |
nightly/v* tag |
:nightly, :sha-<short> |
v* tag (stable) |
:latest, :YYYY.MDD.P, :YYYY.MDD, :sha-<short> |
Lane tags are pushed by release workflows using GITHUB_TOKEN, and GitHub
suppresses push-triggered workflow runs for those pushes. The release jobs
therefore dispatch docker.yml explicitly at the new tag ref; the tag
mapping keys off github.ref either way.
Local Commands
Preview a canary locally
./scripts/release.sh canary --dry-run
Preview a nightly locally
Requires HEAD to be a commit that already shipped a canary (it must carry a
canary/v* tag):
./scripts/release.sh nightly --dry-run
Preview a stable locally
./scripts/release.sh stable --dry-run
Publish a stable locally
This is mainly for emergency/manual use. The normal path is the GitHub workflow.
./scripts/release.sh stable
git push public-gh refs/tags/vYYYY.MDD.P
PUBLISH_REMOTE=public-gh ./scripts/create-github-release.sh YYYY.MDD.P
Stable Changelog Workflow
Stable changelog files live at:
releases/vYYYY.MDD.P.md
Canaries do not get changelog files.
Recommended local generation flow:
VERSION="$(./scripts/release.sh stable --date 2026-03-18 --print-version)"
claude --print --output-format stream-json --verbose --dangerously-skip-permissions --model claude-opus-4-6 "Use the release-changelog skill to draft or update releases/v${VERSION}.md for Paperclip. Read doc/RELEASING.md and .agents/skills/release-changelog/SKILL.md, then generate the stable changelog for v${VERSION} from commits since the last stable tag. Do not create a canary changelog."
The repo intentionally does not run this through GitHub Actions because:
- canaries are too frequent
- stable notes are the only public narrative surface that needs LLM help
- maintainer LLM tokens should not live in Actions
Smoke Testing
For a canary:
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
For the current stable:
PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
Useful isolated variants:
HOST_PORT=3232 DATA_DIR=./data/release-smoke-canary PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
HOST_PORT=3233 DATA_DIR=./data/release-smoke-stable PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
Automated browser smoke is also available:
gh workflow run release-smoke.yml -f paperclip_version=canary
gh workflow run release-smoke.yml -f paperclip_version=nightly
gh workflow run release-smoke.yml -f paperclip_version=latest
The nightly lane runs this same suite automatically against its candidate before publishing.
Minimum checks:
npx paperclipai@canary onboardinstalls- onboarding completes without crashes
- authenticated login works with the smoke credentials
- the browser lands in onboarding on a fresh instance
- company creation succeeds
- the first CEO agent is created
- the first CEO heartbeat run is triggered
Rollback
Rollback does not unpublish versions.
It only moves the latest dist-tag back to a previous stable:
./scripts/rollback-latest.sh 2026.318.0 --dry-run
./scripts/rollback-latest.sh 2026.318.0
Then fix forward with a new stable patch slot or release date.
Failure Playbooks
If the canary publishes but smoke testing fails
Do not run stable.
Instead:
- fix the issue on
master - merge the fix
- wait for the next automatic canary
- rerun smoke testing
If the nightly skipped or failed
A skipped nightly is working as designed — the job summary names the reason (no new green candidate, candidate already shipped, or red smoke). Nothing was published, so there is nothing to clean up.
To recover after fixing the cause, either wait for the next scheduled run or
force one: dispatch release.yml with channel: nightly (optionally pinning
source_version to a specific canary).
If the nightly published to npm but the tag push or Docker dispatch failed,
push the nightly/v* tag manually and run docker.yml at that tag.
If stable npm publish succeeds but tag push or GitHub release creation fails
This is a partial release. npm is already live.
Do this immediately:
- push the missing tag
- rerun
PUBLISH_REMOTE=public-gh ./scripts/create-github-release.sh YYYY.MDD.P - verify the GitHub Release notes point at
releases/vYYYY.MDD.P.md
Do not republish the same version.
If latest is broken after stable publish
Roll back the dist-tag:
./scripts/rollback-latest.sh YYYY.MDD.P
Then fix forward with a new stable release.