0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-06 21:05:21 +02:00

Files

T

Devin Foley 8f7b8b3fda feat(release): add human-gated beta channel with stable soak enforcement (#11008 )

> Follow-up to #11006 (merged): rebased onto master and ready for
review.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem now publishes canary (every master push),
nightly (scheduled, smoke-gated, added in #11006), and stable (manual)
> - There is still no human-approved release-candidate lane between
nightly and stable, and nothing enforces that a stable actually soaked
anywhere before shipping
> - Betas need a real approval gate, and stables need a soak policy that
is data, not prose
> - This pull request adds the beta channel: a manual promotion of a
chosen nightly behind the `npm-beta` environment gate, re-smoked after
publish, plus a stable preflight that enforces a 3-day beta soak with a
written-justification bypass
> - The benefit is a complete canary → nightly → beta → stable train
where every stable shipped as a beta first, and emergencies leave a
written trace

## Linked Issues or Issue Description

**Subsystem affected**

Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

After #11006 the project has canary and nightly prerelease lanes, but no
release-candidate lane. Stable promotion has no enforced soak: any ref
can ship as stable directly. There is no approval boundary for a
broader-audience prerelease, and no structured way to record why an
emergency release skipped validation.

**Proposed solution**

Add a `beta` channel: a manual dispatch that promotes a chosen nightly's
source commit, publishes behind the `npm-beta` GitHub environment
(required reviewers are the gate), re-smokes the published beta, and
tags `beta/vX`. Enforce in the stable path that the source commit
shipped as a beta at least 3 days earlier (measured from the beta's npm
publish time), with a `skip_soak_justification` input as the recorded
emergency bypass.

**Alternatives considered**

Codifying the soak policy in docs only. Rejected: an unenforced policy
decays; the preflight makes the policy executable while the
justification input keeps the emergency path usable and auditable.

## What Changed

- `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel —
requires HEAD to carry a `nightly/v*` tag, publishes the package set as
`YYYY.MDD.P-beta.N` under dist-tag `beta`, tags
`beta/vYYYY.MDD.P-beta.N`
- `.github/workflows/release.yml`:
- `channel: beta` dispatch path: `select_beta` resolves the newest (or
an explicit `source_version`) nightly and fails loudly on selection
problems; `publish_beta` runs behind the `npm-beta` environment, pushes
the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release
smoke suite against the exact published beta version
- stable path: new `preflight_stable` job enforces the 3-day beta soak
from the beta's npm publish time; `skip_soak_justification` bypasses
with the reason echoed into the job summary; dry runs report without
blocking
- `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both
images, with exact version stamping
- `.github/workflows/release-smoke.yml`: `beta` added to the dispatch
choice list
- Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate,
and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta`
environment setup, including the warning to create the environment
before the first beta dispatch (GitHub auto-creates unprotected
environments on first reference)
- Tests: beta version-counting coverage in
`scripts/release-registry-versions.test.mjs`; beta identity and
nightly-tag guard coverage in
`scripts/__tests__/release-dry-run-notes.test.mjs`

## Verification

- `node --test` on the two touched suites: 17 pass, including the 3 new
beta tests
- `bash -n` on both shell scripts and YAML parse of all three workflows
- After merge, in order: create the `npm-beta` environment, dispatch
`channel: beta` with `dry_run: true` to preview, then a real promotion
of a published nightly through the approval gate, then a stable dry-run
against a young beta to see the soak gate report

## Risks

- If the `npm-beta` environment does not exist when the first beta
dispatch runs, GitHub creates it with no protection rules and the beta
publishes without approval. Mitigated by documentation and by creating
the environment before merge (operator step)
- Until the first beta exists, every stable dispatch requires
`skip_soak_justification`. This is deliberate — the first beta ships
immediately after this merges — but it is a behavior change to the
stable dispatch
- The soak clock reads the beta's npm publish time from the registry; a
registry outage makes the preflight fall back to requiring justification
(fail-closed)

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge

2026-08-10 16:52:59 -07:00

11 KiB

Raw Blame History

Release Automation Setup

This document covers the GitHub and npm setup required for the current Paperclip release model:

automatic canaries from master
manual stable promotion from a chosen source ref
npm trusted publishing via GitHub OIDC
protected release infrastructure in a public repository

Repo-side files that depend on this setup:

.github/workflows/release.yml
.github/CODEOWNERS

Note:

the release workflows intentionally use pnpm install --no-frozen-lockfile
this matches the repo's current policy where pnpm-lock.yaml is refreshed by GitHub automation after manifest changes land on master
the publish jobs then restore pnpm-lock.yaml before running scripts/release.sh, so the release script still sees a clean worktree

1. Merge the Repo Changes First

Before touching GitHub or npm settings, merge the release automation code so the referenced workflow filenames already exist on the default branch.

Required files:

.github/workflows/release.yml
.github/CODEOWNERS

2. Configure npm Trusted Publishing

Do this for every public package that Paperclip publishes.

At minimum that includes:

paperclipai
@paperclipai/server
@paperclipai/ui
public packages under packages/

2.1. In npm, open each package settings page

For each package:

open npm as an owner of the package
go to the package settings / publishing access area
add a trusted publisher for the GitHub repository paperclipai/paperclip

2.2. Add one trusted publisher entry per package

npm currently allows one trusted publisher configuration per package.

Configure:

workflow: .github/workflows/release.yml

Repository:

paperclipai/paperclip

Environment name:

leave the npm trusted-publisher environment field blank

Why:

the single release.yml workflow handles both canary and stable publishing
GitHub environments npm-canary and npm-stable still enforce different approval rules on the GitHub side

2.2.1. Newly added public packages need a bootstrap phase

Trusted publishing is configured on the npm package itself, not at the repo scope. That means a brand-new public package must not be auto-enrolled into CI publishing until its npm package exists and its trusted publisher has been configured.

Repo policy:

add every non-private package to scripts/release-package-manifest.json
set "publishFromCi": true only when CI is expected to publish that package
if the package is not ready for CI publishing yet, keep "publishFromCi": false
complete the package bootstrap before merging any PR that changes a release-enabled new package

Bootstrap sequence for a new package:

publish the package once from a trusted maintainer machine using normal npm auth
open that package on npm and add the paperclipai/paperclip trusted publisher for .github/workflows/release.yml
rerun or dry-run the release flow as needed to confirm CI publishing now works
only then enable "publishFromCi": true

PR CI enforces this by checking changed release-enabled package manifests against npm. That keeps master canary publishing healthy while preserving the no-long-lived-token model for normal CI releases.

2.3. Verify trusted publishing before removing old auth

After the workflows are live:

run a canary publish
confirm npm publish succeeds without any NPM_TOKEN
run a stable dry-run
run one real stable publish

Only after that should you remove old token-based access.

3. Remove Legacy npm Tokens

After trusted publishing works:

revoke any repository or organization NPM_TOKEN secrets used for publish
revoke any personal automation token that used to publish Paperclip
if npm offers a package-level setting to restrict publishing to trusted publishers, enable it

Goal:

no long-lived npm publishing token should remain in GitHub Actions

4. Create GitHub Environments

Create three environments in the GitHub repository:

npm-canary
npm-beta
npm-stable

Path:

GitHub repository
Settings
Environments
New environment

5. Configure `npm-canary`

Recommended settings for npm-canary:

environment name: npm-canary
required reviewers: none
wait timer: none
deployment branches and tags:
- selected branches only
- allow master

Reasoning:

every push to master should be able to publish a canary automatically
no human approval should be required for canaries

The scheduled nightly lane also publishes under npm-canary: it is the same trust level (fully automated, no human gate), its runs execute on master so the branch rule is satisfied, and reusing the environment means the nightly lane required no new environments and no npm trusted-publisher changes (publishing still happens from release.yml, see section 2.2).

5.1. Configure `npm-beta`

Recommended settings for npm-beta:

environment name: npm-beta
required reviewers: at least one maintainer
prevent self-review: enabled when your team size allows it
wait timer: none
deployment branches and tags:
- selected branches only
- allow master

Reasoning:

beta promotions are deliberate human decisions; the required reviewer on this environment is the promotion gate
create this environment before the first channel: beta dispatch. If the workflow runs first, GitHub auto-creates the environment with no protection rules, and that first beta would publish without approval

Like nightly, beta publishing lives in release.yml, so no npm trusted-publisher changes are needed (see section 2.2).

6. Configure `npm-stable`

Recommended settings for npm-stable:

environment name: npm-stable
required reviewers: at least one maintainer other than the person triggering the workflow when possible
prevent self-review: enabled
admin bypass: disabled if your team can tolerate it
wait timer: optional
deployment branches and tags:
- selected branches only
- allow master

Reasoning:

stable publishes should require an explicit human approval gate
the workflow is manual, but the environment should still be the real control point

7. Protect `master`

Open the branch protection settings for master.

Recommended rules:

require pull requests before merging
require status checks to pass before merging
require review from code owners
dismiss stale approvals when new commits are pushed
restrict who can push directly to master

At minimum, make sure workflow and release script changes cannot land without review.

8. Enforce CODEOWNERS Review

This repo now includes .github/CODEOWNERS, but GitHub only enforces it if branch protection requires code owner reviews.

In branch protection for master, enable:

Require review from Code Owners

Then verify the owner entries are correct for your actual maintainer set.

Current file:

.github/CODEOWNERS

If @cryppadotta is not the right reviewer identity in the public repo, change it before enabling enforcement.

9. Protect Release Infrastructure Specifically

These files should always trigger code owner review:

.github/workflows/release.yml
scripts/release.sh
scripts/release-lib.sh
scripts/release-package-map.mjs
scripts/create-github-release.sh
scripts/rollback-latest.sh
doc/RELEASING.md
doc/PUBLISHING.md

If you want stronger controls, add a repository ruleset that explicitly blocks direct pushes to:

.github/workflows/**
scripts/release*

10. Do Not Store a Claude Token in GitHub Actions

Do not add a personal Claude or Anthropic token for automatic changelog generation.

Recommended policy:

stable changelog generation happens locally from a trusted maintainer machine
canaries never generate changelogs

This keeps LLM spending intentional and avoids a high-value token sitting in Actions.

11. Verify the Canary Workflow

After setup:

merge a harmless commit to master
open the Release workflow run triggered by that push
confirm it passes verification
confirm publish succeeds under the npm-canary environment
confirm npm now shows a new canary release
confirm a git tag named canary/vYYYY.MDD.P-canary.N was pushed

Install-path check:

npm install --prefix "$(mktemp -d)" paperclipai@canary --no-audit --no-fund

The release script runs this clean-prefix install after publishing every workspace package dependency-first and publishing paperclipai last. A package that is not yet registry-visible stops the train before the channel entrypoint can advance.

12. Verify the Stable Workflow

After at least one good canary exists:

resolve the target stable version with ./scripts/release.sh stable --date YYYY-MM-DD --print-version
prepare releases/vYYYY.MDD.P.md on the source commit you want to promote
open Actions -> Release
run it with:
- source_ref: the tested commit SHA or canary tag source commit
- stable_date: leave blank or set the intended UTC date like 2026-03-18 do not enter a version like 2026.318.0; the workflow computes that from the date
- dry_run: true
confirm the dry-run succeeds
rerun with dry_run: false
approve the npm-stable environment when prompted
confirm npm latest points to the new stable version
confirm git tag vYYYY.MDD.P exists
confirm the GitHub Release was created

Implementation note:

the GitHub Actions stable workflow calls create-github-release.sh with PUBLISH_REMOTE=origin
local maintainer usage can still pass PUBLISH_REMOTE=public-gh explicitly when needed

13. Suggested Maintainer Policy

Use this policy going forward:

canaries are automatic and cheap
stables are manual and approved
only stables get public notes and announcements
release notes are committed before stable publish
rollback uses npm dist-tag, not unpublish

14. Troubleshooting

Trusted publishing fails with an auth error

Check:

the workflow filename on GitHub exactly matches the filename configured in npm
the package has the trusted publisher entry for the correct repository
the job has id-token: write
the job is running from the expected repository, not a fork

Stable workflow runs but never asks for approval

Check:

the publish job uses environment npm-stable
the environment actually has required reviewers configured
the workflow is running in the canonical repository, not a fork

CODEOWNERS does not trigger

Check:

.github/CODEOWNERS is on the default branch
branch protection on master requires code owner review
the owner identities in the file are valid reviewers with repository access

11 KiB Raw Blame History

Release Automation Setup

1. Merge the Repo Changes First

2. Configure npm Trusted Publishing

2.1. In npm, open each package settings page

2.2. Add one trusted publisher entry per package

2.2.1. Newly added public packages need a bootstrap phase

2.3. Verify trusted publishing before removing old auth

3. Remove Legacy npm Tokens

4. Create GitHub Environments

5. Configure npm-canary

5.1. Configure npm-beta

6. Configure npm-stable

7. Protect master

8. Enforce CODEOWNERS Review

9. Protect Release Infrastructure Specifically

10. Do Not Store a Claude Token in GitHub Actions

11. Verify the Canary Workflow

12. Verify the Stable Workflow

13. Suggested Maintainer Policy

14. Troubleshooting

Trusted publishing fails with an auth error

Stable workflow runs but never asks for approval

CODEOWNERS does not trigger

Related Docs

11 KiB

Raw Blame History

5. Configure `npm-canary`

5.1. Configure `npm-beta`

6. Configure `npm-stable`

7. Protect `master`