mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every master push runs a canary, and only one canary at a time may run the release workflow > - After #21554 and #21555 a canary held that lock for 35.5 min, and 13-16 min of it was waiting for npm to show the new packages > - The wait was required: the next canary computed `canary.N` from npm's version list, so starting before npm showed the previous canary would reuse its number > - This pull request numbers a canary by its commit's position on master and moves the registry confirmation, the git tag, the Docker dispatch and the onboarding smoke to a separate `canary-finalize.yml` run > - The benefit is that the release lock is held for about 20-25 min per canary instead of 35, with the same checks before a canary is tagged ## Linked Issues or Issue Description **What existing behavior does this improve?** Canary throughput. In run 43715037497 (2026-11-15) `publish_canary` took 23.7 min, and 13.3 min of that was "Waiting for registry visibility" (13:08:50 to 13:22:11). The next canary could not start until that run ended. **Current behavior** `release.sh canary` sets `N` to (highest published `-canary.N` for today) + 1, then publishes, waits for npm to show every package, checks dist-tags, installs `paperclipai@canary`, and creates `canary/v<version>`. `publish_canary` pushes the tag, dispatches `docker.yml`, and `smoke_canary_onboarding` runs in the same run. **Proposed behavior** `N` = `git rev-list --count HEAD` (full history required), or the registry number + 1 when that is higher (a rerun of the same commit). `release.sh canary --defer-registry-confirmation` ends after npm accepts every publish. `publish_canary` dispatches `canary-finalize.yml`, which does the confirmation, tag, Docker dispatch and smoke outside the release lock. Nightly, beta and stable numbering and flow are unchanged. Refs #21554, #21555 ## What Changed - `scripts/release-lib.sh`: `next_canary_version_from_source`. A publishing canary refuses a shallow checkout; a dry run (the PR Canary Dry Run job) falls back to the registry number. - `scripts/release.sh`: canary uses the new number; new canary-only `--defer-registry-confirmation` flag skips the registry wait, the dist-tag check and the local tag, and writes the version to `PAPERCLIP_RELEASE_VERSION_FILE`. - `scripts/confirm-canary-publication.sh` (new): waits for registry visibility, checks provenance, checks dist-tags with `--allow-newer-canary`, installs `paperclipai@<version>`, then creates and pushes `canary/v<version>`. A tag already on the same commit is accepted, so reruns are safe. - `scripts/verify-canary-provenance.mjs` (new): every package's npm SLSA provenance must name `.github/workflows/release.yml` on `refs/heads/master` in `paperclipai/paperclip` at the source commit. A dispatch with invented inputs cannot tag anything. - `scripts/verify-release-registry-state.mjs`: `--allow-newer-canary` accepts a `canary` dist-tag that a newer canary already moved; older, foreign or missing tags still fail. - `.github/workflows/canary-finalize.yml` (new, `workflow_dispatch`, master only): `confirm_canary` (contents: write, actions: write) and `smoke_canary_onboarding` (downloads the publisher's lockfile artifact by run id). Inputs reach shell steps only through a validating step. - `.github/workflows/release.yml`: `publish_canary` hands off to `canary-finalize.yml` and drops to `contents: read`; its tag push, Docker dispatch and smoke job move to `canary-finalize.yml`. - Tests and `doc/RELEASING.md`. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/release-lib.test.mjs scripts/verify-release-registry-state.test.mjs scripts/verify-canary-provenance.test.mjs`: 846 pass. `scripts/__tests__/release-verify-workflow.test.mjs`: 73 pass. - New tests: numbering by master position, rerun moves past a published number, shallow refusal and dry-run fallback; newer/older/foreign dist-tags; provenance from another commit, workflow, branch, repository or package is rejected; finalize wiring, permissions and input handling. - Real npm data: `verify-canary-provenance.mjs 2026.1116.0-canary.7 b4fa1c7f... paperclipai @paperclipai/shared @paperclipai/server` passes; the same version with another commit fails. - `confirm-canary-publication.sh 2026.1116.0-canary.7 b4fa1c7f...` from a checkout of that commit ran all steps against npm in 61 s and ended with "canary/v2026.1116.0-canary.7 already points at b4fa1c7f...", pushing nothing. - `release.sh canary --dry-run --defer-registry-confirmation` and `release.sh nightly --defer-registry-confirmation` both stop with "only applies to a published canary". - `actionlint` reports no new findings compared with master. - Not verified before merge: a full publish through `canary-finalize.yml`. It can run only from master. The first canary after merge is the check. ## Risks - Canary numbers jump from `canary.7` to about `canary.8144` (master has 8,143 commits). This is valid semver and still higher than every earlier canary; users install through the `canary` dist-tag. - The `canary/v` tag, Docker image and smoke now finish after the release run ends. Nightly selects only tagged canaries, as before. If finalize fails, that canary is on npm without a tag, so nightly skips it, as it skips any canary whose old in-run verification failed. - A canary can now publish while the previous one is still being confirmed. Publishing itself is still serialized, so the `canary` dist-tag only moves forward. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
396 lines
14 KiB
JavaScript
396 lines
14 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { spawnSync } from "node:child_process";
|
|
import { chmodSync, copyFileSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import test from "node:test";
|
|
|
|
const repoRoot = new URL("../..", import.meta.url).pathname.replace(/\/$/, "");
|
|
|
|
function writeExecutable(path, body) {
|
|
writeFileSync(path, body, { mode: 0o755 });
|
|
}
|
|
|
|
function createReleaseFixture() {
|
|
const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-release-dry-run-"));
|
|
const scriptsDir = join(fixtureDir, "scripts");
|
|
const binDir = join(fixtureDir, "bin");
|
|
const callLog = join(fixtureDir, "calls.log");
|
|
|
|
mkdirSync(scriptsDir, { recursive: true });
|
|
mkdirSync(join(fixtureDir, "releases"));
|
|
mkdirSync(binDir);
|
|
writeFileSync(callLog, "");
|
|
|
|
copyFileSync(join(repoRoot, "scripts", "release.sh"), join(scriptsDir, "release.sh"));
|
|
chmodSync(join(scriptsDir, "release.sh"), 0o755);
|
|
|
|
writeFileSync(
|
|
join(scriptsDir, "release-lib.sh"),
|
|
`#!/usr/bin/env bash
|
|
release_info() { echo "$@"; }
|
|
release_fail() { echo "Error: $*" >&2; exit 1; }
|
|
resolve_release_remote() { printf 'origin\\n'; }
|
|
fetch_release_remote() { :; }
|
|
git_current_branch() { printf 'master\\n'; }
|
|
get_last_stable_tag() { printf 'v2026.709.0\\n'; }
|
|
get_current_stable_version() { printf '2026.709.0\\n'; }
|
|
utc_date_iso() { printf '2026-07-10\\n'; }
|
|
list_public_package_info() {
|
|
if [ -n "\${FAKE_PACKAGE_INFO:-}" ]; then
|
|
printf '%b' "$FAKE_PACKAGE_INFO"
|
|
else
|
|
printf 'cli\\tpaperclipai\\t0.0.0\\n'
|
|
fi
|
|
}
|
|
package_publish_tool() { printf 'pnpm\\n'; }
|
|
next_stable_version() { printf '2026.710.0\\n'; }
|
|
next_prerelease_version() { printf '2026.710.0-%s.0\\n' "$1"; }
|
|
next_canary_version_from_source() { next_prerelease_version canary "$3"; }
|
|
release_notes_file() { printf '%s/releases/v%s.md\\n' "$REPO_ROOT" "$1"; }
|
|
stable_tag_name() { printf 'v%s\\n' "$1"; }
|
|
prerelease_tag_name() { printf '%s/v%s\\n' "$1" "$2"; }
|
|
require_channel_tag_at_head() {
|
|
if [ "\${FAKE_MISSING_CHANNEL_TAG:-}" = "$1" ]; then
|
|
echo "Error: HEAD has no $1/v* tag; this channel only publishes commits that already shipped a $1 release." >&2
|
|
exit 1
|
|
fi
|
|
echo "[fixture] require_channel_tag_at_head $1"
|
|
}
|
|
require_channel_tag_absent_at_head() {
|
|
if [ "\${FAKE_PRESENT_CHANNEL_TAG:-}" = "$1" ]; then
|
|
echo "Error: HEAD already shipped as $1/v2026.710.0-$1.0; delete that tag first if you really want to republish this commit on the $1 channel." >&2
|
|
exit 1
|
|
fi
|
|
echo "[fixture] require_channel_tag_absent_at_head $1"
|
|
}
|
|
require_on_master_branch() { :; }
|
|
require_clean_worktree() { :; }
|
|
require_runner_lockfile_current() {
|
|
if [ "\${FAKE_RUNNER_LOCKFILE_BEHIND:-}" = "1" ]; then
|
|
echo "Error: a beta or stable release needs a pnpm-lock.yaml that matches the Runner's manifests at HEAD." >&2
|
|
exit 1
|
|
fi
|
|
echo "[fixture] require_runner_lockfile_current"
|
|
}
|
|
require_npm_publish_auth() { :; }
|
|
git_local_tag_exists() { return 1; }
|
|
git_remote_tag_exists() { return 1; }
|
|
npm_package_version_exists() { return 1; }
|
|
set_public_package_version() { :; }
|
|
`,
|
|
);
|
|
|
|
writeExecutable(
|
|
join(scriptsDir, "release-registry-versions.mjs"),
|
|
`#!/usr/bin/env node
|
|
const [mode] = process.argv.slice(2);
|
|
if (mode === "fetch") {
|
|
process.stdout.write('{"paperclipai":[]}\\n');
|
|
process.exit(0);
|
|
}
|
|
if (mode === "assert-absent") {
|
|
process.exit(0);
|
|
}
|
|
process.exit(2);
|
|
`,
|
|
);
|
|
|
|
writeExecutable(
|
|
join(binDir, "git"),
|
|
`#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
if [ "$1" = "-C" ]; then
|
|
shift 2
|
|
fi
|
|
printf 'git %s\\n' "$*" >> "$FAKE_CALL_LOG"
|
|
case "$1" in
|
|
rev-parse)
|
|
if [ "\${2:-}" = "HEAD" ]; then
|
|
echo abcdef1234567890
|
|
exit 0
|
|
fi
|
|
;;
|
|
diff|ls-files)
|
|
exit 0
|
|
;;
|
|
checkout)
|
|
exit 0
|
|
;;
|
|
esac
|
|
exit 0
|
|
`,
|
|
);
|
|
|
|
writeExecutable(
|
|
join(binDir, "pnpm"),
|
|
`#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
printf 'pnpm %s\\n' "$*" >> "$FAKE_CALL_LOG"
|
|
if [ "$*" = "build" ]; then
|
|
if [ "\${FAKE_BUILD_OK:-}" = "1" ]; then
|
|
exit 0
|
|
fi
|
|
echo "fixture stopped at workspace build"
|
|
exit 42
|
|
fi
|
|
if [ "$1" = "publish" ]; then
|
|
echo "fixture publish preview in $PWD"
|
|
case "$PWD" in
|
|
*"\${FAKE_FAIL_PUBLISH_DIR:-__none__}")
|
|
echo "fixture preview failure" >&2
|
|
exit 3
|
|
;;
|
|
esac
|
|
fi
|
|
exit 0
|
|
`,
|
|
);
|
|
|
|
return { binDir, callLog, fixtureDir, script: join(scriptsDir, "release.sh") };
|
|
}
|
|
|
|
function runRelease(args, extraEnv = {}, prepare = null) {
|
|
const fixture = createReleaseFixture();
|
|
if (prepare) {
|
|
prepare(fixture);
|
|
}
|
|
const result = spawnSync(fixture.script, args, {
|
|
cwd: fixture.fixtureDir,
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
PATH: `${fixture.binDir}:${process.env.PATH}`,
|
|
FAKE_CALL_LOG: fixture.callLog,
|
|
...extraEnv,
|
|
},
|
|
});
|
|
|
|
const calls = readFileSync(fixture.callLog, "utf8");
|
|
rmSync(fixture.fixtureDir, { recursive: true, force: true });
|
|
|
|
return {
|
|
calls,
|
|
output: result.stdout + result.stderr,
|
|
status: result.status,
|
|
};
|
|
}
|
|
|
|
test("stable dry-run preview does not require a pre-authored release notes file", () => {
|
|
const result = runRelease(["stable", "--skip-verify", "--dry-run"]);
|
|
|
|
assert.equal(result.status, 42);
|
|
assert.match(result.output, /==> Release plan/);
|
|
assert.match(result.output, /==> Step 2\/7: Building workspace artifacts/);
|
|
assert.doesNotMatch(result.output, /stable release notes file is required/);
|
|
assert.match(result.calls, /^pnpm build$/m);
|
|
});
|
|
|
|
test("stable publish still requires release notes before publish work starts", () => {
|
|
const result = runRelease(["stable", "--skip-verify"]);
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /stable release notes file is required/);
|
|
assert.doesNotMatch(result.output, /==> Step 2\/7: Building workspace artifacts/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
test("beta and stable check the lockfile against the Runner's manifests before any build, in a dry run too", () => {
|
|
for (const args of [["stable"], ["beta"], ["beta", "--from-candidate"]]) {
|
|
const current = runRelease([...args, "--skip-verify", "--dry-run"]);
|
|
assert.match(current.output, /\[fixture\] require_runner_lockfile_current/, args.join(" "));
|
|
|
|
const behind = runRelease([...args, "--skip-verify", "--dry-run"], { FAKE_RUNNER_LOCKFILE_BEHIND: "1" });
|
|
assert.equal(behind.status, 1, args.join(" "));
|
|
assert.match(behind.output, /needs a pnpm-lock\.yaml that matches the Runner's manifests at HEAD/);
|
|
assert.doesNotMatch(behind.output, /==> Release plan/);
|
|
assert.doesNotMatch(behind.calls, /^pnpm /m);
|
|
}
|
|
});
|
|
|
|
test("canary and nightly do not check the lockfile, and printing a version does not", () => {
|
|
for (const args of [["canary", "--skip-verify", "--dry-run"], ["nightly", "--skip-verify", "--dry-run"]]) {
|
|
const result = runRelease(args, { FAKE_RUNNER_LOCKFILE_BEHIND: "1" });
|
|
assert.doesNotMatch(result.output, /Runner's manifests/, args[0]);
|
|
assert.match(result.output, /==> Release plan/, args[0]);
|
|
}
|
|
for (const channel of ["stable", "beta"]) {
|
|
const version = runRelease([channel, "--print-version"], { FAKE_RUNNER_LOCKFILE_BEHIND: "1" });
|
|
assert.equal(version.status, 0, channel);
|
|
assert.doesNotMatch(version.output, /Runner's manifests/, channel);
|
|
}
|
|
});
|
|
|
|
test("nightly dry-run publishes under the nightly identity without release notes", () => {
|
|
const result = runRelease(["nightly", "--skip-verify", "--dry-run"]);
|
|
|
|
assert.equal(result.status, 42);
|
|
assert.match(result.output, /\[fixture\] require_channel_tag_at_head canary/);
|
|
assert.match(result.output, /Nightly version: 2026\.710\.0-nightly\.0/);
|
|
assert.match(result.output, /Dist-tag: nightly/);
|
|
assert.match(result.output, /Git tag: nightly\/v2026\.710\.0-nightly\.0/);
|
|
assert.doesNotMatch(result.output, /stable release notes file is required/);
|
|
assert.match(result.calls, /^pnpm build$/m);
|
|
});
|
|
|
|
test("nightly refuses commits that never shipped a canary", () => {
|
|
const result = runRelease(["nightly", "--skip-verify", "--dry-run"], {
|
|
FAKE_MISSING_CHANNEL_TAG: "canary",
|
|
});
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /HEAD has no canary\/v\* tag/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
test("nightly refuses commits that already shipped as a nightly", () => {
|
|
const result = runRelease(["nightly", "--skip-verify", "--dry-run"], {
|
|
FAKE_PRESENT_CHANNEL_TAG: "nightly",
|
|
});
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /HEAD already shipped as nightly\/v/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
test("beta dry-run publishes under the beta identity without release notes", () => {
|
|
const result = runRelease(["beta", "--skip-verify", "--dry-run"]);
|
|
|
|
assert.equal(result.status, 42);
|
|
assert.match(result.output, /\[fixture\] require_channel_tag_at_head nightly/);
|
|
assert.match(result.output, /Beta version: 2026\.710\.0-beta\.0/);
|
|
assert.match(result.output, /Dist-tag: beta/);
|
|
assert.match(result.output, /Git tag: beta\/v2026\.710\.0-beta\.0/);
|
|
assert.doesNotMatch(result.output, /stable release notes file is required/);
|
|
assert.match(result.calls, /^pnpm build$/m);
|
|
});
|
|
|
|
test("beta refuses commits that already shipped as a beta", () => {
|
|
const result = runRelease(["beta", "--skip-verify", "--dry-run"], {
|
|
FAKE_PRESENT_CHANNEL_TAG: "beta",
|
|
});
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /HEAD already shipped as beta\/v/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
test("beta --from-candidate waives the nightly requirement but keeps the duplicate guard", () => {
|
|
const result = runRelease(["beta", "--from-candidate", "--skip-verify", "--dry-run"], {
|
|
FAKE_MISSING_CHANNEL_TAG: "nightly",
|
|
});
|
|
|
|
assert.equal(result.status, 42);
|
|
assert.doesNotMatch(result.output, /require_channel_tag_at_head nightly/);
|
|
assert.match(result.output, /\[fixture\] require_channel_tag_absent_at_head beta/);
|
|
assert.match(result.output, /Beta version: 2026\.710\.0-beta\.0/);
|
|
assert.match(result.calls, /^pnpm build$/m);
|
|
});
|
|
|
|
test("--from-candidate is rejected outside the beta channel", () => {
|
|
const result = runRelease(["nightly", "--from-candidate", "--skip-verify", "--dry-run"]);
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /--from-candidate only applies to the beta channel/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
test("beta refuses commits that never shipped a nightly", () => {
|
|
const result = runRelease(["beta", "--skip-verify", "--dry-run"], {
|
|
FAKE_MISSING_CHANNEL_TAG: "nightly",
|
|
});
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.output, /HEAD has no nightly\/v\* tag/);
|
|
assert.doesNotMatch(result.calls, /^pnpm /m);
|
|
});
|
|
|
|
// --- Step 5 concurrent publish-payload previews (dry-run only) ---
|
|
|
|
const PREVIEW_PACKAGE_COUNT = 10; // more than one batch of 8
|
|
|
|
function preparePreviewFixture(fixture) {
|
|
const { fixtureDir } = fixture;
|
|
// Step 2 copies the skills tree into these package dirs.
|
|
mkdirSync(join(fixtureDir, "skills"), { recursive: true });
|
|
for (const dir of ["server", "packages/adapters/claude-local", "packages/adapters/codex-local"]) {
|
|
mkdirSync(join(fixtureDir, dir), { recursive: true });
|
|
}
|
|
// Step 2 build helpers that live outside the pnpm stub.
|
|
writeExecutable(
|
|
join(fixtureDir, "scripts", "build-standalone-public-packages.mjs"),
|
|
"#!/usr/bin/env node\nprocess.exit(0);\n",
|
|
);
|
|
writeExecutable(join(fixtureDir, "scripts", "prepare-server-ui-dist.sh"), "#!/usr/bin/env bash\nexit 0\n");
|
|
writeExecutable(join(fixtureDir, "scripts", "build-npm.sh"), "#!/usr/bin/env bash\nexit 0\n");
|
|
// Step 4 verifies the CLI package version against the target canary version.
|
|
mkdirSync(join(fixtureDir, "cli"), { recursive: true });
|
|
writeFileSync(
|
|
join(fixtureDir, "cli", "package.json"),
|
|
JSON.stringify({ name: "paperclipai", version: "2026.710.0-canary.0" }),
|
|
);
|
|
// Step 5 cds into each previewed package directory.
|
|
for (let index = 1; index <= PREVIEW_PACKAGE_COUNT; index += 1) {
|
|
mkdirSync(join(fixtureDir, previewPackageDir(index)), { recursive: true });
|
|
}
|
|
}
|
|
|
|
function previewPackageDir(index) {
|
|
return `pkg-${String(index).padStart(2, "0")}`;
|
|
}
|
|
|
|
function previewPackageInfo() {
|
|
return Array.from({ length: PREVIEW_PACKAGE_COUNT }, (_, i) => {
|
|
const dir = previewPackageDir(i + 1);
|
|
return `${dir}\\t@fixture/${dir}\\t0.0.0\\n`;
|
|
}).join("");
|
|
}
|
|
|
|
test("canary dry-run previews every publish payload and replays logs in package order", () => {
|
|
const result = runRelease(
|
|
["canary", "--skip-verify", "--dry-run"],
|
|
{ FAKE_BUILD_OK: "1", FAKE_PACKAGE_INFO: previewPackageInfo() },
|
|
preparePreviewFixture,
|
|
);
|
|
|
|
assert.equal(result.status, 0, result.output);
|
|
const headerOffsets = [];
|
|
for (let index = 1; index <= PREVIEW_PACKAGE_COUNT; index += 1) {
|
|
const offset = result.output.indexOf(`--- ${previewPackageDir(index)} ---`);
|
|
assert.notEqual(offset, -1, `missing preview header for ${previewPackageDir(index)}`);
|
|
headerOffsets.push(offset);
|
|
}
|
|
assert.deepEqual(
|
|
headerOffsets,
|
|
[...headerOffsets].sort((a, b) => a - b),
|
|
"preview logs must replay in package order even though previews run concurrently",
|
|
);
|
|
const previews = result.output.match(/fixture publish preview in /g) ?? [];
|
|
assert.equal(previews.length, PREVIEW_PACKAGE_COUNT, "every package must be previewed exactly once");
|
|
assert.match(result.output, /Would create git tag/);
|
|
});
|
|
|
|
test("canary dry-run fails when any concurrent payload preview fails, after replaying every log", () => {
|
|
const result = runRelease(
|
|
["canary", "--skip-verify", "--dry-run"],
|
|
{
|
|
FAKE_BUILD_OK: "1",
|
|
FAKE_PACKAGE_INFO: previewPackageInfo(),
|
|
FAKE_FAIL_PUBLISH_DIR: previewPackageDir(7),
|
|
},
|
|
preparePreviewFixture,
|
|
);
|
|
|
|
assert.notEqual(result.status, 0, "a failed preview must fail the dry run");
|
|
assert.match(result.output, /one or more publish payload previews failed/);
|
|
assert.match(result.output, /fixture preview failure/);
|
|
for (let index = 1; index <= PREVIEW_PACKAGE_COUNT; index += 1) {
|
|
assert.match(
|
|
result.output,
|
|
new RegExp(`--- ${previewPackageDir(index)} ---`),
|
|
"every package's log must still replay so the failure is diagnosable",
|
|
);
|
|
}
|
|
assert.doesNotMatch(result.output, /Would create git tag/, "a failed preview must not reach tagging");
|
|
});
|