mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 12:07:09 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip publishes canary and stable packages through a shared release script > - GitHub Actions authenticates those publishes through npm trusted publishing and an OIDC identity token > - Bundled-dependency packages recently moved from pnpm publish to a pinned npm CLI to preserve their bundled files > - That pin selected npm 10, which cannot use trusted publishing, so the first bundled package failed with `ENEEDAUTH` > - This pull request keeps bundled-package packing on npm 10 while routing actual publishing through the trusted-publishing-capable npm 11 version > - The benefit is bundled packages keep their required npm packaging behavior while canary and stable releases authenticate successfully ## Linked Issues or Issue Description **What happened** The canary release job failed while publishing `@paperclipai/adapter-utils` with `ENEEDAUTH`. The package has bundled dependencies, so the release helper selected pinned `npm@10.9.7`; the workflow provides OIDC trusted publishing rather than an npm token, and npm 10 cannot use that authentication path. Because this is the first package attempted, the release exited before trying the remaining packages. **Expected behavior** Bundled-dependency packages should publish with an npm CLI that both preserves bundled dependencies and supports GitHub Actions trusted publishing. **Steps to reproduce** Run the canary release workflow from master after PR #9980. The `publish_canary` job reaches `@paperclipai/adapter-utils`, invokes `npx npm@10.9.7 publish`, and fails with `ENEEDAUTH`. **Deployment mode** GitHub Actions canary and stable npm release workflows. Refs #9980. ## What Changed - Kept bundled-package dry-run packing on npm `10.9.7`, which successfully produces the staged tarball. - Routed bundled-package publishing through npm `11.16.0`, which supports GitHub Actions trusted publishing. - Split the pack and publish helpers so future npm changes cannot silently couple the two compatibility requirements. - Updated focused release and ACPX packaging tests to enforce both versions and call paths. ## Verification - `pnpm test:release-registry` — 67 passed locally. - Initial all-npm-11 PR head: Canary Dry Run reproduced an npm-internal crash during bundled `pack`. - Current head `5b3961ed13dd26ed2d6b1096ea23fd91b32e4353`: Canary Dry Run passed with split npm pack/publish helpers. - All PR checks passed, including build, typecheck + release registry, server/workspace suites, both e2e shards, security gates, and Greptile. - Greptile reviewed the current head at 5/5 confidence with no blocking issues. ## Risks - Low risk: the change only separates the npm CLI used for bundled-package packing from the CLI used for publishing. - The versions remain explicitly pinned because npm 11.16.0 currently crashes on the bundled pack payload, while npm 10.9.7 cannot perform trusted publishing. - Focused tests assert both pins and both helper call paths, and the full Canary Dry Run passes on the current head. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with repository, shell, GitHub CLI, and code-execution tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
461 lines
11 KiB
Bash
461 lines
11 KiB
Bash
#!/usr/bin/env bash
|
|
|
|
if [ -z "${REPO_ROOT:-}" ]; then
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
fi
|
|
|
|
release_info() {
|
|
echo "$@"
|
|
}
|
|
|
|
release_warn() {
|
|
echo "Warning: $*" >&2
|
|
}
|
|
|
|
release_fail() {
|
|
echo "Error: $*" >&2
|
|
exit 1
|
|
}
|
|
|
|
git_remote_exists() {
|
|
git -C "$REPO_ROOT" remote get-url "$1" >/dev/null 2>&1
|
|
}
|
|
|
|
github_repo_from_remote() {
|
|
local remote_url
|
|
|
|
remote_url="$(git -C "$REPO_ROOT" remote get-url "$1" 2>/dev/null || true)"
|
|
[ -n "$remote_url" ] || return 1
|
|
|
|
remote_url="${remote_url%.git}"
|
|
remote_url="${remote_url#ssh://}"
|
|
|
|
node - "$remote_url" <<'NODE'
|
|
const remoteUrl = process.argv[2];
|
|
|
|
const patterns = [
|
|
/^https?:\/\/github\.com\/([^/]+\/[^/]+)$/,
|
|
/^git@github\.com:([^/]+\/[^/]+)$/,
|
|
/^[^:]+:([^/]+\/[^/]+)$/
|
|
];
|
|
|
|
for (const pattern of patterns) {
|
|
const match = remoteUrl.match(pattern);
|
|
if (!match) continue;
|
|
process.stdout.write(match[1]);
|
|
process.exit(0);
|
|
}
|
|
|
|
process.exit(1);
|
|
NODE
|
|
}
|
|
|
|
resolve_release_remote() {
|
|
local remote="${RELEASE_REMOTE:-${PUBLISH_REMOTE:-}}"
|
|
|
|
if [ -n "$remote" ]; then
|
|
git_remote_exists "$remote" || release_fail "git remote '$remote' does not exist."
|
|
printf '%s\n' "$remote"
|
|
return
|
|
fi
|
|
|
|
if git_remote_exists public-gh; then
|
|
printf 'public-gh\n'
|
|
return
|
|
fi
|
|
|
|
if git_remote_exists public; then
|
|
printf 'public\n'
|
|
return
|
|
fi
|
|
|
|
if git_remote_exists origin; then
|
|
printf 'origin\n'
|
|
return
|
|
fi
|
|
|
|
release_fail "no git remote found. Configure RELEASE_REMOTE or PUBLISH_REMOTE."
|
|
}
|
|
|
|
fetch_release_remote() {
|
|
git -C "$REPO_ROOT" fetch "$1" --prune --tags
|
|
}
|
|
|
|
git_current_branch() {
|
|
git -C "$REPO_ROOT" symbolic-ref --quiet --short HEAD 2>/dev/null || true
|
|
}
|
|
|
|
git_local_tag_exists() {
|
|
git -C "$REPO_ROOT" show-ref --verify --quiet "refs/tags/$1"
|
|
}
|
|
|
|
git_remote_tag_exists() {
|
|
git -C "$REPO_ROOT" ls-remote --exit-code --tags "$2" "refs/tags/$1" >/dev/null 2>&1
|
|
}
|
|
|
|
get_last_stable_tag() {
|
|
git -C "$REPO_ROOT" tag --list 'v*' --sort=-version:refname | head -1
|
|
}
|
|
|
|
get_current_stable_version() {
|
|
local tag
|
|
tag="$(get_last_stable_tag)"
|
|
if [ -z "$tag" ]; then
|
|
printf '0.0.0\n'
|
|
else
|
|
printf '%s\n' "${tag#v}"
|
|
fi
|
|
}
|
|
|
|
stable_version_slot_for_date() {
|
|
node - "${1:-}" <<'NODE'
|
|
const input = process.argv[2];
|
|
|
|
const date = input ? new Date(`${input}T00:00:00Z`) : new Date();
|
|
if (Number.isNaN(date.getTime())) {
|
|
console.error(`invalid date: ${input}`);
|
|
process.exit(1);
|
|
}
|
|
|
|
const month = String(date.getUTCMonth() + 1);
|
|
const day = String(date.getUTCDate()).padStart(2, '0');
|
|
|
|
process.stdout.write(`${date.getUTCFullYear()}.${month}${day}`);
|
|
NODE
|
|
}
|
|
|
|
utc_date_iso() {
|
|
node <<'NODE'
|
|
const date = new Date();
|
|
const y = date.getUTCFullYear();
|
|
const m = String(date.getUTCMonth() + 1).padStart(2, '0');
|
|
const d = String(date.getUTCDate()).padStart(2, '0');
|
|
process.stdout.write(`${y}-${m}-${d}`);
|
|
NODE
|
|
}
|
|
|
|
next_stable_version() {
|
|
local release_date="$1"
|
|
shift
|
|
|
|
node - "$release_date" "$@" <<'NODE'
|
|
const input = process.argv[2];
|
|
const packageNames = process.argv.slice(3);
|
|
const { execSync } = require("node:child_process");
|
|
const { readFileSync } = require("node:fs");
|
|
|
|
const date = input ? new Date(`${input}T00:00:00Z`) : new Date();
|
|
if (Number.isNaN(date.getTime())) {
|
|
console.error(`invalid date: ${input}`);
|
|
process.exit(1);
|
|
}
|
|
|
|
// Optional pre-fetched version data (see release-registry-versions.mjs).
|
|
// Avoids one serial `npm view` round-trip per package.
|
|
let versionsCache = null;
|
|
if (process.env.RELEASE_PACKAGE_VERSIONS_FILE) {
|
|
try {
|
|
versionsCache = JSON.parse(readFileSync(process.env.RELEASE_PACKAGE_VERSIONS_FILE, "utf8"));
|
|
} catch {
|
|
versionsCache = null;
|
|
}
|
|
}
|
|
|
|
const stableSlot = `${date.getUTCFullYear()}.${date.getUTCMonth() + 1}${String(date.getUTCDate()).padStart(2, "0")}`;
|
|
const pattern = new RegExp(`^${stableSlot.replace(/\./g, '\\.')}\.(\\d+)$`);
|
|
let max = -1;
|
|
|
|
for (const packageName of packageNames) {
|
|
let versions = [];
|
|
|
|
if (versionsCache && Array.isArray(versionsCache[packageName])) {
|
|
versions = versionsCache[packageName];
|
|
} else {
|
|
try {
|
|
const raw = execSync(`npm view ${JSON.stringify(packageName)} versions --json`, {
|
|
encoding: "utf8",
|
|
stdio: ["ignore", "pipe", "ignore"],
|
|
}).trim();
|
|
|
|
if (raw) {
|
|
const parsed = JSON.parse(raw);
|
|
versions = Array.isArray(parsed) ? parsed : [parsed];
|
|
}
|
|
} catch {
|
|
versions = [];
|
|
}
|
|
}
|
|
|
|
for (const version of versions) {
|
|
const match = version.match(pattern);
|
|
if (!match) continue;
|
|
max = Math.max(max, Number(match[1]));
|
|
}
|
|
}
|
|
|
|
process.stdout.write(`${stableSlot}.${max + 1}`);
|
|
NODE
|
|
}
|
|
|
|
next_canary_version() {
|
|
local stable_version="$1"
|
|
shift
|
|
|
|
node - "$stable_version" "$@" <<'NODE'
|
|
const stable = process.argv[2];
|
|
const packageNames = process.argv.slice(3);
|
|
const { execSync } = require("node:child_process");
|
|
const { readFileSync } = require("node:fs");
|
|
|
|
// Optional pre-fetched version data (see release-registry-versions.mjs).
|
|
// Avoids one serial `npm view` round-trip per package.
|
|
let versionsCache = null;
|
|
if (process.env.RELEASE_PACKAGE_VERSIONS_FILE) {
|
|
try {
|
|
versionsCache = JSON.parse(readFileSync(process.env.RELEASE_PACKAGE_VERSIONS_FILE, "utf8"));
|
|
} catch {
|
|
versionsCache = null;
|
|
}
|
|
}
|
|
|
|
const pattern = new RegExp(`^${stable.replace(/\./g, '\\.')}-canary\\.(\\d+)$`);
|
|
let max = -1;
|
|
|
|
for (const packageName of packageNames) {
|
|
let versions = [];
|
|
|
|
if (versionsCache && Array.isArray(versionsCache[packageName])) {
|
|
versions = versionsCache[packageName];
|
|
} else {
|
|
try {
|
|
const raw = execSync(`npm view ${JSON.stringify(packageName)} versions --json`, {
|
|
encoding: "utf8",
|
|
stdio: ["ignore", "pipe", "ignore"],
|
|
}).trim();
|
|
|
|
if (raw) {
|
|
const parsed = JSON.parse(raw);
|
|
versions = Array.isArray(parsed) ? parsed : [parsed];
|
|
}
|
|
} catch {
|
|
versions = [];
|
|
}
|
|
}
|
|
|
|
for (const version of versions) {
|
|
const match = version.match(pattern);
|
|
if (!match) continue;
|
|
max = Math.max(max, Number(match[1]));
|
|
}
|
|
}
|
|
|
|
process.stdout.write(`${stable}-canary.${max + 1}`);
|
|
NODE
|
|
}
|
|
|
|
release_notes_file() {
|
|
printf '%s/releases/v%s.md\n' "$REPO_ROOT" "$1"
|
|
}
|
|
|
|
stable_tag_name() {
|
|
printf 'v%s\n' "$1"
|
|
}
|
|
|
|
canary_tag_name() {
|
|
printf 'canary/v%s\n' "$1"
|
|
}
|
|
|
|
npm_package_version_exists() {
|
|
local package_name="$1"
|
|
local version="$2"
|
|
local resolved
|
|
|
|
resolved="$(npm view "${package_name}@${version}" version 2>/dev/null || true)"
|
|
[ "$resolved" = "$version" ]
|
|
}
|
|
|
|
wait_for_npm_package_version() {
|
|
local package_name="$1"
|
|
local version="$2"
|
|
local attempts="${3:-12}"
|
|
local delay_seconds="${4:-5}"
|
|
local attempt=1
|
|
|
|
while [ "$attempt" -le "$attempts" ]; do
|
|
if npm_package_version_exists "$package_name" "$version"; then
|
|
return 0
|
|
fi
|
|
|
|
if [ "$attempt" -lt "$attempts" ]; then
|
|
sleep "$delay_seconds"
|
|
fi
|
|
attempt=$((attempt + 1))
|
|
done
|
|
|
|
return 1
|
|
}
|
|
|
|
is_npm_tlog_duplicate_error() {
|
|
local output="$1"
|
|
|
|
grep -q "TLOG_CREATE_ENTRY_ERROR" <<< "$output" &&
|
|
grep -q "equivalent entry already exists in the transparency log" <<< "$output"
|
|
}
|
|
|
|
package_publish_tool() {
|
|
node -e '
|
|
const pkg = require(process.cwd() + "/package.json");
|
|
const bundled = pkg.bundleDependencies ?? pkg.bundledDependencies ?? [];
|
|
process.stdout.write(bundled.length > 0 ? "npm" : "pnpm");
|
|
'
|
|
}
|
|
|
|
BUNDLED_NPM_PACK_VERSION="10.9.7"
|
|
BUNDLED_NPM_PUBLISH_VERSION="11.16.0"
|
|
|
|
run_bundled_npm_pack() {
|
|
npx --yes "npm@$BUNDLED_NPM_PACK_VERSION" "$@"
|
|
}
|
|
|
|
run_bundled_npm_publish() {
|
|
npx --yes "npm@$BUNDLED_NPM_PUBLISH_VERSION" "$@"
|
|
}
|
|
|
|
run_package_publish() {
|
|
local publish_tool="$1"
|
|
local dist_tag="$2"
|
|
local disable_provenance="${3:-false}"
|
|
|
|
if [ "$publish_tool" = "npm" ]; then
|
|
if [ "$disable_provenance" = "true" ]; then
|
|
run_bundled_npm_publish publish --tag "$dist_tag" --access public --provenance=false
|
|
else
|
|
run_bundled_npm_publish publish --tag "$dist_tag" --access public
|
|
fi
|
|
return
|
|
fi
|
|
|
|
if [ "$disable_provenance" = "true" ]; then
|
|
pnpm publish --no-git-checks --tag "$dist_tag" --access public --provenance=false
|
|
else
|
|
pnpm publish --no-git-checks --tag "$dist_tag" --access public
|
|
fi
|
|
}
|
|
|
|
publish_package_to_npm() {
|
|
local dist_tag="$1"
|
|
local package_name="$2"
|
|
local package_version="$3"
|
|
local publish_tool="${4:-pnpm}"
|
|
local publish_log
|
|
|
|
publish_log="$(mktemp "${TMPDIR:-/tmp}/paperclip-npm-publish.XXXXXX")"
|
|
|
|
if (set -o pipefail; run_package_publish "$publish_tool" "$dist_tag" false 2>&1 | tee "$publish_log"); then
|
|
rm -f "$publish_log"
|
|
return 0
|
|
fi
|
|
|
|
if ! is_npm_tlog_duplicate_error "$(cat "$publish_log")"; then
|
|
rm -f "$publish_log"
|
|
return 1
|
|
fi
|
|
|
|
release_warn "npm publish hit a duplicate Sigstore transparency-log entry for ${package_name}@${package_version}."
|
|
|
|
if npm_package_version_exists "$package_name" "$package_version"; then
|
|
release_warn "npm already exposes ${package_name}@${package_version}; continuing to registry verification."
|
|
rm -f "$publish_log"
|
|
return 0
|
|
fi
|
|
|
|
if [ "$dist_tag" != "canary" ]; then
|
|
release_warn "Not retrying ${package_name}@${package_version} without provenance for dist-tag ${dist_tag}."
|
|
rm -f "$publish_log"
|
|
return 1
|
|
fi
|
|
|
|
release_warn "Retrying ${package_name}@${package_version} once with npm provenance disabled."
|
|
if run_package_publish "$publish_tool" "$dist_tag" true; then
|
|
rm -f "$publish_log"
|
|
return 0
|
|
fi
|
|
|
|
rm -f "$publish_log"
|
|
return 1
|
|
}
|
|
|
|
wait_for_release_registry_state() {
|
|
local attempts="${1:-12}"
|
|
local delay_seconds="${2:-5}"
|
|
shift 2
|
|
local attempt=1
|
|
local output
|
|
local status
|
|
|
|
while [ "$attempt" -le "$attempts" ]; do
|
|
if output="$(node "$REPO_ROOT/scripts/verify-release-registry-state.mjs" "$@" 2>&1)"; then
|
|
[ -n "$output" ] && printf '%s\n' "$output"
|
|
return 0
|
|
fi
|
|
status=$?
|
|
|
|
printf '%s\n' "$output" >&2
|
|
|
|
if [ "$status" -eq 2 ]; then
|
|
return "$status"
|
|
fi
|
|
|
|
if [ "$attempt" -lt "$attempts" ]; then
|
|
release_warn "npm registry metadata has not converged yet (attempt ${attempt}/${attempts}); retrying in ${delay_seconds}s."
|
|
sleep "$delay_seconds"
|
|
fi
|
|
|
|
attempt=$((attempt + 1))
|
|
done
|
|
|
|
return "${status:-1}"
|
|
}
|
|
|
|
require_clean_worktree() {
|
|
if [ -n "$(git -C "$REPO_ROOT" status --porcelain)" ]; then
|
|
release_fail "working tree is not clean. Commit, stash, or remove changes before releasing."
|
|
fi
|
|
}
|
|
|
|
require_on_master_branch() {
|
|
local current_branch
|
|
current_branch="$(git_current_branch)"
|
|
if [ "$current_branch" != "master" ]; then
|
|
release_fail "this release step must run from branch master, but current branch is ${current_branch:-<detached>}."
|
|
fi
|
|
}
|
|
|
|
require_npm_publish_auth() {
|
|
local dry_run="$1"
|
|
|
|
if [ "$dry_run" = true ]; then
|
|
return
|
|
fi
|
|
|
|
if npm whoami >/dev/null 2>&1; then
|
|
release_info " ✓ Logged in to npm as $(npm whoami)"
|
|
return
|
|
fi
|
|
|
|
if [ "${GITHUB_ACTIONS:-}" = "true" ]; then
|
|
release_info " ✓ npm publish auth will be provided by GitHub Actions trusted publishing"
|
|
return
|
|
fi
|
|
|
|
release_fail "npm publish auth is not available. Use 'npm login' locally or run from GitHub Actions with trusted publishing."
|
|
}
|
|
|
|
list_public_package_info() {
|
|
node "$REPO_ROOT/scripts/release-package-map.mjs" list
|
|
}
|
|
|
|
set_public_package_version() {
|
|
node "$REPO_ROOT/scripts/release-package-map.mjs" set-version "$1"
|
|
}
|