Files
PaperClipAI/scripts/__tests__/release-dry-run-notes.test.mjs
T
Devin Foley f9173782cd feat(release): add smoke-gated nightly channel and lane-separated Docker tags (#11006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem publishes the `paperclipai` npm package set
and the Docker images on two lanes: canary on every master push, and
stable on manual promotion
> - There is no middle ground between those lanes. Users must track
every merge or wait weeks for a stable. Docker `:latest` also tracks
master, so Docker users have no stable image at all
> - A calm prerelease lane needs to exist, and it must never ship a
build that failed its checks
> - This pull request adds the nightly channel: a scheduled job that
selects the newest master commit with a green canary publish, runs the
full release smoke suite against that exact published canary, and only
then republishes it as the nightly. It also separates Docker tags by
lane, so `:latest` finally means stable
> - The benefit is that users can follow prereleases at a nightly
cadence with a smoke-tested guarantee, and Docker users get real
`:canary`, `:nightly`, and stable image tags

## Linked Issues or Issue Description

**Subsystem affected**

Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

The project publishes only `canary` (every master push) and `latest`
(manual stable). Users who want prereleases without per-merge churn have
no option. Docker has a second problem: master builds overwrite
`:latest`, and CI-published stables never produced Docker images,
because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger
in `docker.yml`. No stable-versioned image exists in ghcr today.

**Proposed solution**

Add a `nightly` channel. A scheduled job selects the newest
canary-tagged master commit, smoke-tests that exact published canary,
and republishes the same commit as `YYYY.MDD.P-nightly.N` under the
`nightly` dist-tag. Separate Docker tags by lane (`:canary` for master,
`:nightly` for nightly tags, `:latest` plus version tags for stable tags
only), and have the release jobs dispatch `docker.yml` at the new tag so
lane images actually build.

**Alternatives considered**

Moving the `nightly` dist-tag to the existing canary version without a
republish. Rejected: the version string would say `canary` while the
user is on nightly, which breaks at-a-glance lane identification in bug
reports and `--version` output.

## What Changed

- `scripts/release-lib.sh`: channel-parameterized
`next_prerelease_version` and `prerelease_tag_name` helpers (canary
helpers delegate to them), a `require_channel_tag_at_head` guard, and
the no-provenance retry for Sigstore transparency-log duplicates now
covers the `nightly` dist-tag as well as `canary`
- `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry
a `canary/v*` tag, publishes the full public package set as
`YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source
commit `nightly/vYYYY.MDD.P-nightly.N`
- `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) —
select candidate, smoke it via `release-smoke.yml`, publish on green
under the existing `npm-canary` environment, push the tag, dispatch
`docker.yml`. New `channel` dispatch input (default `stable`, so
existing stable dispatches are unchanged) with `nightly_source_version`
and `dry_run` support for forced runs. The stable path now also
dispatches `docker.yml` at the new `v*` tag
- `.github/workflows/docker.yml`: lane tag mapping for both image jobs —
master pushes publish `:canary` and no longer move `:latest`;
`nightly/v*` tags publish `:nightly`; only stable `v*` tags publish
`:latest` and the versioned tags. New `workflow_dispatch` trigger for
the release-job dispatches. Build-version stamping uses the exact
nightly version on nightly tag builds
- `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch
choice list
- `doc/CHANNELS.md` (new): user-facing guide to the channels
- `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping
table, and a nightly failure playbook
- `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses
`npm-canary` and needs no npm trusted-publisher changes
- Tests: channel-parameterized version helper coverage in
`scripts/release-registry-versions.test.mjs`, and nightly flow coverage
(publish identity, notes not required, canary-tag guard) in
`scripts/__tests__/release-dry-run-notes.test.mjs`

## Verification

- `node --test` on the release script suites: 68 pass, including 6 new
tests. The only failure, `acpx-patch-packaging.test.mjs`, needs
installed `node_modules` and fails identically on a pristine checkout of
master in the same environment
- `bash -n` on both shell scripts and YAML parse of all three workflows
- Live fail-path check: `./scripts/release.sh nightly --print-version`
from a master tip with no canary tag fails with `HEAD has no canary/v*
tag`
- Live success-path check: the same command from the
`canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0`
- Live selection check: the candidate-selection shell logic run against
the real repository selects the commit of `canary/v2026.806.0-canary.7`,
which matches the current npm `canary` dist-tag exactly
- After merge: dispatch `release.yml` with `channel: nightly` and
`dry_run: true` to preview, then a real forced run to validate end to
end before the first scheduled run

## Risks

- Docker `:latest` changes meaning from "latest master build" to "latest
stable release". This is deliberate and will be announced. Users who
want the old behavior pull `:canary`. Until the first stable release
after this change, `:latest` stays at its current (master-built) image
- The nightly is a rebuild of the same source commit, not the
byte-identical canary artifact that was smoked. The lockfile pins
dependencies, and the publish path's registry-visibility and
clean-prefix install gates still run on the nightly artifacts
- All npm publishing must stay inside `release.yml` because npm trusted
publishing pins that workflow file per package. The nightly jobs were
added to `release.yml` for exactly that reason; this constraint is now
documented in `RELEASING.md`
- The stable-lane Docker dispatch fails gracefully (a warning with
manual instructions) when the source ref predates `docker.yml`'s
`workflow_dispatch` trigger

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 12:16:55 -07:00

198 lines
6.2 KiB
JavaScript

import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
const repoRoot = new URL("../..", import.meta.url).pathname.replace(/\/$/, "");
function writeExecutable(path, body) {
writeFileSync(path, body, { mode: 0o755 });
}
function createReleaseFixture() {
const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-release-dry-run-"));
const scriptsDir = join(fixtureDir, "scripts");
const binDir = join(fixtureDir, "bin");
const callLog = join(fixtureDir, "calls.log");
mkdirSync(scriptsDir, { recursive: true });
mkdirSync(join(fixtureDir, "releases"));
mkdirSync(binDir);
writeFileSync(callLog, "");
copyFileSync(join(repoRoot, "scripts", "release.sh"), join(scriptsDir, "release.sh"));
chmodSync(join(scriptsDir, "release.sh"), 0o755);
writeFileSync(
join(scriptsDir, "release-lib.sh"),
`#!/usr/bin/env bash
release_info() { echo "$@"; }
release_fail() { echo "Error: $*" >&2; exit 1; }
resolve_release_remote() { printf 'origin\\n'; }
fetch_release_remote() { :; }
git_current_branch() { printf 'master\\n'; }
get_last_stable_tag() { printf 'v2026.709.0\\n'; }
get_current_stable_version() { printf '2026.709.0\\n'; }
utc_date_iso() { printf '2026-07-10\\n'; }
list_public_package_info() { printf 'cli\\tpaperclipai\\t0.0.0\\n'; }
next_stable_version() { printf '2026.710.0\\n'; }
next_prerelease_version() { printf '2026.710.0-%s.0\\n' "$1"; }
release_notes_file() { printf '%s/releases/v%s.md\\n' "$REPO_ROOT" "$1"; }
stable_tag_name() { printf 'v%s\\n' "$1"; }
prerelease_tag_name() { printf '%s/v%s\\n' "$1" "$2"; }
require_channel_tag_at_head() {
if [ "\${FAKE_MISSING_CHANNEL_TAG:-}" = "$1" ]; then
echo "Error: HEAD has no $1/v* tag; this channel only publishes commits that already shipped a $1 release." >&2
exit 1
fi
echo "[fixture] require_channel_tag_at_head $1"
}
require_channel_tag_absent_at_head() {
if [ "\${FAKE_PRESENT_CHANNEL_TAG:-}" = "$1" ]; then
echo "Error: HEAD already shipped as $1/v2026.710.0-$1.0; delete that tag first if you really want to republish this commit on the $1 channel." >&2
exit 1
fi
echo "[fixture] require_channel_tag_absent_at_head $1"
}
require_clean_worktree() { :; }
require_npm_publish_auth() { :; }
git_local_tag_exists() { return 1; }
git_remote_tag_exists() { return 1; }
npm_package_version_exists() { return 1; }
set_public_package_version() { :; }
`,
);
writeExecutable(
join(scriptsDir, "release-registry-versions.mjs"),
`#!/usr/bin/env node
const [mode] = process.argv.slice(2);
if (mode === "fetch") {
process.stdout.write('{"paperclipai":[]}\\n');
process.exit(0);
}
if (mode === "assert-absent") {
process.exit(0);
}
process.exit(2);
`,
);
writeExecutable(
join(binDir, "git"),
`#!/usr/bin/env bash
set -euo pipefail
if [ "$1" = "-C" ]; then
shift 2
fi
printf 'git %s\\n' "$*" >> "$FAKE_CALL_LOG"
case "$1" in
rev-parse)
if [ "\${2:-}" = "HEAD" ]; then
echo abcdef1234567890
exit 0
fi
;;
diff|ls-files)
exit 0
;;
checkout)
exit 0
;;
esac
exit 0
`,
);
writeExecutable(
join(binDir, "pnpm"),
`#!/usr/bin/env bash
set -euo pipefail
printf 'pnpm %s\\n' "$*" >> "$FAKE_CALL_LOG"
if [ "$*" = "build" ]; then
echo "fixture stopped at workspace build"
exit 42
fi
exit 0
`,
);
return { binDir, callLog, fixtureDir, script: join(scriptsDir, "release.sh") };
}
function runRelease(args, extraEnv = {}) {
const fixture = createReleaseFixture();
const result = spawnSync(fixture.script, args, {
cwd: fixture.fixtureDir,
encoding: "utf8",
env: {
...process.env,
PATH: `${fixture.binDir}:${process.env.PATH}`,
FAKE_CALL_LOG: fixture.callLog,
...extraEnv,
},
});
const calls = readFileSync(fixture.callLog, "utf8");
rmSync(fixture.fixtureDir, { recursive: true, force: true });
return {
calls,
output: result.stdout + result.stderr,
status: result.status,
};
}
test("stable dry-run preview does not require a pre-authored release notes file", () => {
const result = runRelease(["stable", "--skip-verify", "--dry-run"]);
assert.equal(result.status, 42);
assert.match(result.output, /==> Release plan/);
assert.match(result.output, /==> Step 2\/7: Building workspace artifacts/);
assert.doesNotMatch(result.output, /stable release notes file is required/);
assert.match(result.calls, /^pnpm build$/m);
});
test("stable publish still requires release notes before publish work starts", () => {
const result = runRelease(["stable", "--skip-verify"]);
assert.equal(result.status, 1);
assert.match(result.output, /stable release notes file is required/);
assert.doesNotMatch(result.output, /==> Step 2\/7: Building workspace artifacts/);
assert.doesNotMatch(result.calls, /^pnpm /m);
});
test("nightly dry-run publishes under the nightly identity without release notes", () => {
const result = runRelease(["nightly", "--skip-verify", "--dry-run"]);
assert.equal(result.status, 42);
assert.match(result.output, /\[fixture\] require_channel_tag_at_head canary/);
assert.match(result.output, /Nightly version: 2026\.710\.0-nightly\.0/);
assert.match(result.output, /Dist-tag: nightly/);
assert.match(result.output, /Git tag: nightly\/v2026\.710\.0-nightly\.0/);
assert.doesNotMatch(result.output, /stable release notes file is required/);
assert.match(result.calls, /^pnpm build$/m);
});
test("nightly refuses commits that never shipped a canary", () => {
const result = runRelease(["nightly", "--skip-verify", "--dry-run"], {
FAKE_MISSING_CHANNEL_TAG: "canary",
});
assert.equal(result.status, 1);
assert.match(result.output, /HEAD has no canary\/v\* tag/);
assert.doesNotMatch(result.calls, /^pnpm /m);
});
test("nightly refuses commits that already shipped as a nightly", () => {
const result = runRelease(["nightly", "--skip-verify", "--dry-run"], {
FAKE_PRESENT_CHANNEL_TAG: "nightly",
});
assert.equal(result.status, 1);
assert.match(result.output, /HEAD already shipped as nightly\/v/);
assert.doesNotMatch(result.calls, /^pnpm /m);
});