mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 20:05:57 +02:00
> Follow-up to #11006 (merged): rebased onto master and ready for review. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release subsystem now publishes canary (every master push), nightly (scheduled, smoke-gated, added in #11006), and stable (manual) > - There is still no human-approved release-candidate lane between nightly and stable, and nothing enforces that a stable actually soaked anywhere before shipping > - Betas need a real approval gate, and stables need a soak policy that is data, not prose > - This pull request adds the beta channel: a manual promotion of a chosen nightly behind the `npm-beta` environment gate, re-smoked after publish, plus a stable preflight that enforces a 3-day beta soak with a written-justification bypass > - The benefit is a complete canary → nightly → beta → stable train where every stable shipped as a beta first, and emergencies leave a written trace ## Linked Issues or Issue Description **Subsystem affected** Release automation: `scripts/release.sh`, `scripts/release-lib.sh`, `.github/workflows/release.yml`, `.github/workflows/docker.yml`, `.github/workflows/release-smoke.yml`. **Problem or motivation** After #11006 the project has canary and nightly prerelease lanes, but no release-candidate lane. Stable promotion has no enforced soak: any ref can ship as stable directly. There is no approval boundary for a broader-audience prerelease, and no structured way to record why an emergency release skipped validation. **Proposed solution** Add a `beta` channel: a manual dispatch that promotes a chosen nightly's source commit, publishes behind the `npm-beta` GitHub environment (required reviewers are the gate), re-smokes the published beta, and tags `beta/vX`. Enforce in the stable path that the source commit shipped as a beta at least 3 days earlier (measured from the beta's npm publish time), with a `skip_soak_justification` input as the recorded emergency bypass. **Alternatives considered** Codifying the soak policy in docs only. Rejected: an unenforced policy decays; the preflight makes the policy executable while the justification input keeps the emergency path usable and auditable. ## What Changed - `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel — requires HEAD to carry a `nightly/v*` tag, publishes the package set as `YYYY.MDD.P-beta.N` under dist-tag `beta`, tags `beta/vYYYY.MDD.P-beta.N` - `.github/workflows/release.yml`: - `channel: beta` dispatch path: `select_beta` resolves the newest (or an explicit `source_version`) nightly and fails loudly on selection problems; `publish_beta` runs behind the `npm-beta` environment, pushes the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release smoke suite against the exact published beta version - stable path: new `preflight_stable` job enforces the 3-day beta soak from the beta's npm publish time; `skip_soak_justification` bypasses with the reason echoed into the job summary; dry runs report without blocking - `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both images, with exact version stamping - `.github/workflows/release-smoke.yml`: `beta` added to the dispatch choice list - Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate, and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta` environment setup, including the warning to create the environment before the first beta dispatch (GitHub auto-creates unprotected environments on first reference) - Tests: beta version-counting coverage in `scripts/release-registry-versions.test.mjs`; beta identity and nightly-tag guard coverage in `scripts/__tests__/release-dry-run-notes.test.mjs` ## Verification - `node --test` on the two touched suites: 17 pass, including the 3 new beta tests - `bash -n` on both shell scripts and YAML parse of all three workflows - After merge, in order: create the `npm-beta` environment, dispatch `channel: beta` with `dry_run: true` to preview, then a real promotion of a published nightly through the approval gate, then a stable dry-run against a young beta to see the soak gate report ## Risks - If the `npm-beta` environment does not exist when the first beta dispatch runs, GitHub creates it with no protection rules and the beta publishes without approval. Mitigated by documentation and by creating the environment before merge (operator step) - Until the first beta exists, every stable dispatch requires `skip_soak_justification`. This is deliberate — the first beta ships immediately after this merges — but it is a behavior change to the stable dispatch - The soak clock reads the beta's npm publish time from the registry; a registry outage makes the preflight fall back to requiring justification (fail-closed) ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use (repository exploration, local test execution, live registry and git verification). All code, tests, and docs in this PR were model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge
259 lines
7.9 KiB
JavaScript
259 lines
7.9 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { execFileSync } from "node:child_process";
|
|
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import test from "node:test";
|
|
|
|
const repoRoot = new URL("..", import.meta.url).pathname.replace(/\/$/, "");
|
|
const scriptPath = join(repoRoot, "scripts", "release-registry-versions.mjs");
|
|
|
|
function writeExecutable(path, body) {
|
|
writeFileSync(path, body, { mode: 0o755 });
|
|
}
|
|
|
|
function makeFixture() {
|
|
const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-release-registry-"));
|
|
const binDir = join(fixtureDir, "bin");
|
|
const callLog = join(fixtureDir, "calls.log");
|
|
mkdirSync(binDir);
|
|
writeFileSync(callLog, "");
|
|
|
|
writeExecutable(
|
|
join(binDir, "npm"),
|
|
`#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
printf 'npm %s\\n' "$*" >> "$FAKE_CALL_LOG"
|
|
target="$2"
|
|
case "$target" in
|
|
"@paperclipai/present@"*)
|
|
printf '%s\\n' "\${target##*@}"
|
|
;;
|
|
"@paperclipai/absent@"*)
|
|
exit 1
|
|
;;
|
|
"@paperclipai/present")
|
|
echo '["1.0.0","2026.707.0","2026.707.1","2026.707.1-canary.4"]'
|
|
;;
|
|
*)
|
|
exit 1
|
|
;;
|
|
esac
|
|
`,
|
|
);
|
|
|
|
return { fixtureDir, binDir, callLog };
|
|
}
|
|
|
|
function runScript(args, { binDir, callLog }, extraEnv = {}) {
|
|
let status = 0;
|
|
let stdout = "";
|
|
let stderr = "";
|
|
try {
|
|
stdout = execFileSync("node", [scriptPath, ...args], {
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
PATH: `${binDir}:${process.env.PATH}`,
|
|
FAKE_CALL_LOG: callLog,
|
|
...extraEnv,
|
|
},
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
});
|
|
} catch (error) {
|
|
status = error.status ?? 1;
|
|
stdout = error.stdout ?? "";
|
|
stderr = error.stderr ?? "";
|
|
}
|
|
return { status, stdout, stderr, calls: readFileSync(callLog, "utf8") };
|
|
}
|
|
|
|
function runReleaseLibHelper(fnCall, { binDir, callLog }, extraEnv = {}) {
|
|
const script = `
|
|
set -euo pipefail
|
|
source "${repoRoot}/scripts/release-lib.sh"
|
|
${fnCall}
|
|
`;
|
|
let status = 0;
|
|
let output = "";
|
|
try {
|
|
output = execFileSync("bash", ["-c", script], {
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
PATH: `${binDir}:${process.env.PATH}`,
|
|
FAKE_CALL_LOG: callLog,
|
|
REPO_ROOT: repoRoot,
|
|
...extraEnv,
|
|
},
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
});
|
|
} catch (error) {
|
|
status = error.status ?? 1;
|
|
output = `${error.stdout ?? ""}${error.stderr ?? ""}`;
|
|
}
|
|
return { status, output, calls: readFileSync(callLog, "utf8") };
|
|
}
|
|
|
|
test("fetch prints a JSON version map and treats missing packages as empty", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(["fetch", "@paperclipai/present", "@paperclipai/missing"], fixture);
|
|
|
|
assert.equal(result.status, 0);
|
|
const map = JSON.parse(result.stdout);
|
|
assert.deepEqual(map["@paperclipai/present"], [
|
|
"1.0.0",
|
|
"2026.707.0",
|
|
"2026.707.1",
|
|
"2026.707.1-canary.4",
|
|
]);
|
|
assert.deepEqual(map["@paperclipai/missing"], []);
|
|
assert.match(result.calls, /^npm view @paperclipai\/present versions --json$/m);
|
|
assert.match(result.calls, /^npm view @paperclipai\/missing versions --json$/m);
|
|
});
|
|
|
|
test("assert-absent succeeds when no package has the version", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(
|
|
["assert-absent", "2026.707.2", "@paperclipai/absent", "@paperclipai/absent"],
|
|
fixture,
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.match(result.calls, /^npm view @paperclipai\/absent@2026\.707\.2 version$/m);
|
|
});
|
|
|
|
test("assert-absent fails and names packages that already have the version", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(
|
|
["assert-absent", "2026.707.2", "@paperclipai/present", "@paperclipai/absent"],
|
|
fixture,
|
|
);
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.stderr, /npm version @paperclipai\/present@2026\.707\.2 already exists\./);
|
|
assert.doesNotMatch(result.stderr, /@paperclipai\/absent@/);
|
|
});
|
|
|
|
test("invalid concurrency fails instead of skipping registry checks", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(["assert-absent", "2026.707.2", "@paperclipai/present"], fixture, {
|
|
RELEASE_REGISTRY_CONCURRENCY: "0",
|
|
});
|
|
|
|
assert.equal(result.status, 2);
|
|
assert.match(result.stderr, /RELEASE_REGISTRY_CONCURRENCY must be a positive integer\./);
|
|
assert.equal(result.calls, "");
|
|
});
|
|
|
|
test("next_stable_version reads RELEASE_PACKAGE_VERSIONS_FILE without calling npm", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.0", "2026.707.1", "2026.707.1-canary.4"],
|
|
"@paperclipai/b": [],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper(
|
|
'next_stable_version 2026-07-07 "@paperclipai/a" "@paperclipai/b"',
|
|
fixture,
|
|
{ RELEASE_PACKAGE_VERSIONS_FILE: versionsFile },
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.2");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_canary_version reads RELEASE_PACKAGE_VERSIONS_FILE without calling npm", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.0", "2026.707.1", "2026.707.1-canary.4"],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper('next_canary_version 2026.707.1 "@paperclipai/a"', fixture, {
|
|
RELEASE_PACKAGE_VERSIONS_FILE: versionsFile,
|
|
});
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.1-canary.5");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_stable_version falls back to npm view without a versions file", () => {
|
|
const fixture = makeFixture();
|
|
const result = runReleaseLibHelper('next_stable_version 2026-07-07 "@paperclipai/present"', fixture);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.2");
|
|
assert.match(result.calls, /^npm view @paperclipai\/present versions --json$/m);
|
|
});
|
|
|
|
test("next_prerelease_version counts per channel so nightly numbering ignores canaries", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.1-canary.4", "2026.707.1-nightly.0", "2026.707.1-nightly.1"],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper(
|
|
'next_prerelease_version nightly 2026.707.1 "@paperclipai/a"',
|
|
fixture,
|
|
{ RELEASE_PACKAGE_VERSIONS_FILE: versionsFile },
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.1-nightly.2");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_prerelease_version counts beta numbering independently of other channels", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.1-canary.4", "2026.707.1-nightly.3", "2026.707.1-beta.0"],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper(
|
|
'next_prerelease_version beta 2026.707.1 "@paperclipai/a"',
|
|
fixture,
|
|
{ RELEASE_PACKAGE_VERSIONS_FILE: versionsFile },
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.1-beta.1");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_prerelease_version rejects unknown channels", () => {
|
|
const fixture = makeFixture();
|
|
const result = runReleaseLibHelper(
|
|
'next_prerelease_version weekly 2026.707.1 "@paperclipai/a"',
|
|
fixture,
|
|
);
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /unknown prerelease channel: weekly/);
|
|
});
|
|
|
|
test("prerelease_tag_name namespaces tags by channel", () => {
|
|
const fixture = makeFixture();
|
|
const result = runReleaseLibHelper("prerelease_tag_name nightly 2026.707.1-nightly.2", fixture);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output.trim(), "nightly/v2026.707.1-nightly.2");
|
|
});
|