mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 20:05:57 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The release subsystem publishes the public workspace packages and also powers release-related CI validation. > - The release flow currently asks npm for package versions one package at a time in multiple places. > - That serial registry latency slows the PR Canary Dry Run path and real release invocations even though the checks are independent. > - This pull request batches npm registry version lookups with bounded concurrency and reuses the result for version calculation. > - The benefit is shorter non-build release-script time while preserving the fresh target-version existence check before publishing. ## Linked Issues or Issue Description - No public GitHub issue exists for this release-script performance cleanup. ### Problem or motivation Release validation spends avoidable time on repeated serial `npm view` calls across the public package set. The slow path affects PR release validation and real release invocations because version discovery waits on independent registry reads one at a time. ### Proposed solution Fetch package version maps concurrently with bounded parallelism, reuse that map for stable/canary version calculation, and keep a fresh parallel absence check for the target publish version. ### Alternatives considered Keeping the existing serial shell loop is simpler, but it preserves the CI latency cost. Caching the final target-version existence check was rejected because release publish safety should still query npm freshly before publishing. ### Roadmap alignment This is a small release-tooling performance improvement. It does not duplicate any planned core product work found in `ROADMAP.md`. ## What Changed - Added `scripts/release-registry-versions.mjs` to fetch npm package version maps and assert target-version absence with bounded parallelism. - Updated `scripts/release.sh` to prefetch package versions once and to batch the final target-version absence check. - Updated `next_stable_version` and `next_canary_version` to use the prefetched version map when present, with the existing per-package npm fallback preserved. - Added release-registry helper coverage and included it in `pnpm run test:release-registry`. - Hardened the release publish helper tests so their fake `pnpm`/`npm` fixture PATH is preserved under non-login shell execution. ## Verification - `node --test scripts/release-registry-versions.test.mjs` - `pnpm run test:release-registry` - `bash -n scripts/release.sh scripts/release-lib.sh` - `git diff --check` - Safety scan before push: searched changed files for common key/token/password patterns and PII markers; only benign script-name text matched (`secrets:migrate-inline-env`). - Remote PR checks on the latest head passed, including `Typecheck + Release Registry`, `Canary Dry Run`, build, tests, e2e, policy, security scans, and commitperclip review. - Greptile reviewed the latest head with Confidence Score 5/5 and no blocking issues. ## Risks - Low risk. The release version helpers keep their original npm fallback when no prefetched version map is supplied. - The existence check remains fresh and uncached before publish, but now reports all matching package/version pairs from a parallel check. - If npm has transient failures during the prefetch step, missing or failed packages still map to an empty version list, matching the old helper behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent using GPT-5, with shell/tool execution in the local repository. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude <noreply@paperclip.ing>
198 lines
6.0 KiB
JavaScript
198 lines
6.0 KiB
JavaScript
import assert from "node:assert/strict";
|
|
import { execFileSync } from "node:child_process";
|
|
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import test from "node:test";
|
|
|
|
const repoRoot = new URL("..", import.meta.url).pathname.replace(/\/$/, "");
|
|
const scriptPath = join(repoRoot, "scripts", "release-registry-versions.mjs");
|
|
|
|
function writeExecutable(path, body) {
|
|
writeFileSync(path, body, { mode: 0o755 });
|
|
}
|
|
|
|
function makeFixture() {
|
|
const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-release-registry-"));
|
|
const binDir = join(fixtureDir, "bin");
|
|
const callLog = join(fixtureDir, "calls.log");
|
|
mkdirSync(binDir);
|
|
writeFileSync(callLog, "");
|
|
|
|
writeExecutable(
|
|
join(binDir, "npm"),
|
|
`#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
printf 'npm %s\\n' "$*" >> "$FAKE_CALL_LOG"
|
|
target="$2"
|
|
case "$target" in
|
|
"@paperclipai/present@"*)
|
|
printf '%s\\n' "\${target##*@}"
|
|
;;
|
|
"@paperclipai/absent@"*)
|
|
exit 1
|
|
;;
|
|
"@paperclipai/present")
|
|
echo '["1.0.0","2026.707.0","2026.707.1","2026.707.1-canary.4"]'
|
|
;;
|
|
*)
|
|
exit 1
|
|
;;
|
|
esac
|
|
`,
|
|
);
|
|
|
|
return { fixtureDir, binDir, callLog };
|
|
}
|
|
|
|
function runScript(args, { binDir, callLog }, extraEnv = {}) {
|
|
let status = 0;
|
|
let stdout = "";
|
|
let stderr = "";
|
|
try {
|
|
stdout = execFileSync("node", [scriptPath, ...args], {
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
PATH: `${binDir}:${process.env.PATH}`,
|
|
FAKE_CALL_LOG: callLog,
|
|
...extraEnv,
|
|
},
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
});
|
|
} catch (error) {
|
|
status = error.status ?? 1;
|
|
stdout = error.stdout ?? "";
|
|
stderr = error.stderr ?? "";
|
|
}
|
|
return { status, stdout, stderr, calls: readFileSync(callLog, "utf8") };
|
|
}
|
|
|
|
function runReleaseLibHelper(fnCall, { binDir, callLog }, extraEnv = {}) {
|
|
const script = `
|
|
set -euo pipefail
|
|
source "${repoRoot}/scripts/release-lib.sh"
|
|
${fnCall}
|
|
`;
|
|
let status = 0;
|
|
let output = "";
|
|
try {
|
|
output = execFileSync("bash", ["-c", script], {
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
PATH: `${binDir}:${process.env.PATH}`,
|
|
FAKE_CALL_LOG: callLog,
|
|
REPO_ROOT: repoRoot,
|
|
...extraEnv,
|
|
},
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
});
|
|
} catch (error) {
|
|
status = error.status ?? 1;
|
|
output = `${error.stdout ?? ""}${error.stderr ?? ""}`;
|
|
}
|
|
return { status, output, calls: readFileSync(callLog, "utf8") };
|
|
}
|
|
|
|
test("fetch prints a JSON version map and treats missing packages as empty", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(["fetch", "@paperclipai/present", "@paperclipai/missing"], fixture);
|
|
|
|
assert.equal(result.status, 0);
|
|
const map = JSON.parse(result.stdout);
|
|
assert.deepEqual(map["@paperclipai/present"], [
|
|
"1.0.0",
|
|
"2026.707.0",
|
|
"2026.707.1",
|
|
"2026.707.1-canary.4",
|
|
]);
|
|
assert.deepEqual(map["@paperclipai/missing"], []);
|
|
assert.match(result.calls, /^npm view @paperclipai\/present versions --json$/m);
|
|
assert.match(result.calls, /^npm view @paperclipai\/missing versions --json$/m);
|
|
});
|
|
|
|
test("assert-absent succeeds when no package has the version", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(
|
|
["assert-absent", "2026.707.2", "@paperclipai/absent", "@paperclipai/absent"],
|
|
fixture,
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.match(result.calls, /^npm view @paperclipai\/absent@2026\.707\.2 version$/m);
|
|
});
|
|
|
|
test("assert-absent fails and names packages that already have the version", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(
|
|
["assert-absent", "2026.707.2", "@paperclipai/present", "@paperclipai/absent"],
|
|
fixture,
|
|
);
|
|
|
|
assert.equal(result.status, 1);
|
|
assert.match(result.stderr, /npm version @paperclipai\/present@2026\.707\.2 already exists\./);
|
|
assert.doesNotMatch(result.stderr, /@paperclipai\/absent@/);
|
|
});
|
|
|
|
test("invalid concurrency fails instead of skipping registry checks", () => {
|
|
const fixture = makeFixture();
|
|
const result = runScript(["assert-absent", "2026.707.2", "@paperclipai/present"], fixture, {
|
|
RELEASE_REGISTRY_CONCURRENCY: "0",
|
|
});
|
|
|
|
assert.equal(result.status, 2);
|
|
assert.match(result.stderr, /RELEASE_REGISTRY_CONCURRENCY must be a positive integer\./);
|
|
assert.equal(result.calls, "");
|
|
});
|
|
|
|
test("next_stable_version reads RELEASE_PACKAGE_VERSIONS_FILE without calling npm", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.0", "2026.707.1", "2026.707.1-canary.4"],
|
|
"@paperclipai/b": [],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper(
|
|
'next_stable_version 2026-07-07 "@paperclipai/a" "@paperclipai/b"',
|
|
fixture,
|
|
{ RELEASE_PACKAGE_VERSIONS_FILE: versionsFile },
|
|
);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.2");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_canary_version reads RELEASE_PACKAGE_VERSIONS_FILE without calling npm", () => {
|
|
const fixture = makeFixture();
|
|
const versionsFile = join(fixture.fixtureDir, "versions.json");
|
|
writeFileSync(
|
|
versionsFile,
|
|
JSON.stringify({
|
|
"@paperclipai/a": ["2026.707.0", "2026.707.1", "2026.707.1-canary.4"],
|
|
}),
|
|
);
|
|
|
|
const result = runReleaseLibHelper('next_canary_version 2026.707.1 "@paperclipai/a"', fixture, {
|
|
RELEASE_PACKAGE_VERSIONS_FILE: versionsFile,
|
|
});
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.1-canary.5");
|
|
assert.doesNotMatch(result.calls, /npm view/);
|
|
});
|
|
|
|
test("next_stable_version falls back to npm view without a versions file", () => {
|
|
const fixture = makeFixture();
|
|
const result = runReleaseLibHelper('next_stable_version 2026-07-07 "@paperclipai/present"', fixture);
|
|
|
|
assert.equal(result.status, 0);
|
|
assert.equal(result.output, "2026.707.2");
|
|
assert.match(result.calls, /^npm view @paperclipai\/present versions --json$/m);
|
|
});
|