mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
feat(release): bootstrap new npm packages with a placeholder publish (#11757)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its release pipeline publishes a set of npm packages from CI with npm trusted publishing (GitHub OIDC), gated by `scripts/release-package-manifest.json` > - A brand-new package name cannot be published by CI directly: the PR bootstrap gate requires the name to resolve on npm, and a trusted-publisher rule can only be configured after the package page exists > - The current bootstrap helper closes that gap by building the package locally and publishing its real output from a maintainer machine — before the PR that adds the package has passed CI or review > - This pull request replaces that flow: the helper now publishes a minimal deprecated placeholder at version `0.0.0` that only reserves the name, so every real version ships from CI > - The benefit is that unreviewed build output never reaches npm, and the bootstrap runs from any checkout (including `master`, before the new package's PR merges) with no local build ## Linked Issues or Issue Description **What existing behavior does this improve?** The one-time npm bootstrap for a brand-new release package (`pnpm run release:bootstrap-package`). **Current behavior** The helper builds the target package locally and publishes the real build output from a maintainer machine. That content has not passed repository CI or review at publish time. The helper also requires the new package to exist in the local workspace, so it must run from the (unmerged) PR branch that adds the package. **Proposed behavior** The helper publishes a three-file placeholder at version `0.0.0` (manifest, README, and an `index.js` that throws a descriptive error), waits for the registry to show the package, then deprecates it. The PR bootstrap gate (`scripts/check-release-package-bootstrap.mjs`) only requires the name to resolve on the registry, so the placeholder satisfies it. The first real calver release from CI supersedes the placeholder, and a stable release moves `latest` off it — the same `latest` window that existed under the old flow, but containing an explicit inert stub instead of unreviewed code. **Reason and benefit** Real package content only ever reaches npm from CI, after review and merge. The bootstrap becomes safer (scope guard refuses names outside `@paperclipai/`, already-published names are rejected) and simpler (no local build, no workspace state, runs from any checkout). **Breaking changes** None at runtime. The helper's CLI surface changes: it now takes a package name only (no directory selector) and drops `--skip-build`. `doc/PUBLISHING.md` is updated to match. ## What Changed - `scripts/bootstrap-npm-package.mjs`: replaced the build-and-publish flow with a placeholder publish — stages `package.json` + `README.md` + throwing `index.js` at version `0.0.0` in a temp directory, previews with `npm publish --dry-run`, and publishes only with `--publish`. One-time passwords are prompted interactively (never passed as arguments, since they are single-use and would land in shell history), with re-prompt on a rejected or expired code. After publishing, the helper polls the registry until the package is visible (a first publish can lag by minutes; verified live at ~5 minutes), requiring two consecutive sightings before prompting for a second code and deprecating the placeholder so accidental installs warn loudly; on timeout or failure it prints the exact manual `npm deprecate` command. Added an `@paperclipai/`-scope guard and a fail-fast error when `--publish` runs without an interactive terminal. Removed the workspace-plan dependency so it runs from any checkout. - `scripts/bootstrap-npm-package.test.mjs`: rewrote for the new interface — argument parsing, scope validation, the generated placeholder files (manifest shape, throwing entry point, README), the OTP re-prompt loop, and the registry poll (consecutive-sighting requirement, timeout, transient-error tolerance) via injected fakes. - `doc/PUBLISHING.md`: rewrote the "One-time bootstrap sequence for a new package" section for the placeholder flow, including the `latest` dist-tag window and the trusted-publishing setup ordering (placeholder publish → trusted publisher rule → `"publishFromCi": true`). - `.github/scripts/check-pr-release-bootstrap.mjs` (+ test, + wiring in `run-quality-gates.mjs`): new informational commitperclip notice on PRs that need this bootstrap. It fires when the PR newly release-enables a package that is missing from npm, or adds an unpublished `publishFromCi: false` package that published packages declare a `workspace:*` dependency on, and names the exact maintainer command — so contributors know the red `policy` check is not theirs to fix. It never fails the gate (the `policy` job remains the enforcer), only looks up scope-validated names on the registry, and stays quiet on registry errors. ## Verification - `node --test scripts/bootstrap-npm-package.test.mjs`: 13/13 pass - `node --test .github/scripts/tests/*.test.mjs`: 147/147 pass (10 new for the PR notice) - `pnpm run test:release-registry`: 82/82 pass - Replayed the new PR notice against a real historical PR's live API data (files, manifest at base and head refs): with the registry in its pre-bootstrap state it produces the exact maintainer instruction; with the package bootstrapped it stays silent - Full live end-to-end run: the flow bootstrapped `@paperclipai/adapter-kimi-local` for real — dry-run preview (634-byte, 3-file tarball), publish, registry visibility after ~5 minutes of propagation lag, deprecation confirmed via `npm view ... deprecated` - Guards verified live: an already-published name is rejected, an out-of-scope name (`left-pad`) is rejected, unknown options (including the removed `--otp`) are rejected, and `--publish` in a non-interactive shell fails fast before any network call ## Risks - The `latest` dist-tag points at the deprecated `0.0.0` placeholder until the first stable release supersedes it. This window also existed under the old flow (which parked `latest` at a locally built version); internal consumers are unaffected because release version rewrites pin exact calver versions. - The registry poll caps at ~10 minutes. If propagation is slower than that, the helper prints the exact `npm deprecate ... --otp <code>` command to run manually once `npm view` resolves. - The helper no longer validates the name against the workspace release plan, so a typo within the `@paperclipai/` scope would reserve a wrong name. The dry-run preview shows the exact name before any publish. ## Model Used - Anthropic, **Claude Fable 5** (`claude-fable-5`) via Claude Code, with repository, shell, and Git tooling. It analyzed the existing bootstrap flow and the release scripts (`release-package-map.mjs`, `check-release-package-bootstrap.mjs`, `release.sh` dist-tag handling), wrote the replacement script and tests, updated the documentation, and ran the verification above. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
This commit is contained in:
1 parent
faab2620ad
commit
b5a3a863c3
6 files changed
+867
-250
No files matched your search
@@ -0,0 +1,265 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
addedWorkspaceDependencyNames,
|
||||
checkReleaseBootstrap,
|
||||
} from '../check-pr-release-bootstrap.mjs';
|
||||
|
||||
const MANIFEST_PATH = 'scripts/release-package-manifest.json';
|
||||
|
||||
function encodeManifest(entries) {
|
||||
return { content: Buffer.from(JSON.stringify(entries)).toString('base64') };
|
||||
}
|
||||
|
||||
function stubGitHub({ base = [], head = [] }) {
|
||||
return async (path) => {
|
||||
if (path.includes('ref=refs%2Fpull%2F')) return encodeManifest(head);
|
||||
if (path.includes(`/contents/`)) return encodeManifest(base);
|
||||
throw new Error(`unexpected fetch: ${path}`);
|
||||
};
|
||||
}
|
||||
|
||||
const manifestChangedFile = { filename: MANIFEST_PATH, status: 'modified' };
|
||||
|
||||
test('does nothing (and fetches nothing) when the manifest is untouched', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[{ filename: 'server/src/index.ts', status: 'modified' }],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: async () => { throw new Error('should not fetch'); },
|
||||
registryPackageExists: async () => { throw new Error('should not look up'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result, { passed: true, informational: [] });
|
||||
});
|
||||
|
||||
test('notices a new publishFromCi:true package that is missing from npm', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [{ dir: 'a', name: '@paperclipai/existing', publishFromCi: true }],
|
||||
head: [
|
||||
{ dir: 'a', name: '@paperclipai/existing', publishFromCi: true },
|
||||
{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true },
|
||||
],
|
||||
}),
|
||||
registryPackageExists: async (name) => name !== '@paperclipai/brand-new',
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/brand-new/);
|
||||
assert.match(result.informational[0], /release:bootstrap-package -- @paperclipai\/brand-new --publish/);
|
||||
assert.match(result.informational[0], /No contributor action/);
|
||||
});
|
||||
|
||||
test('stays quiet when the new package already exists on npm', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/already-bootstrapped', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => true,
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('notices a publishFromCi flip from false to true on a missing package', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [{ dir: 'b', name: '@paperclipai/flipped', publishFromCi: false }],
|
||||
head: [{ dir: 'b', name: '@paperclipai/flipped', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/flipped/);
|
||||
});
|
||||
|
||||
test('notices a publishFromCi:false package that published packages newly depend on', async () => {
|
||||
const files = [
|
||||
manifestChangedFile,
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
status: 'modified',
|
||||
patch: '@@ -1 +1 @@\n+ "@paperclipai/adapter-kimi-local": "workspace:*",',
|
||||
},
|
||||
];
|
||||
|
||||
const result = await checkReleaseBootstrap(
|
||||
files,
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/adapter-kimi-local', publishFromCi: false }],
|
||||
}),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /depend on it/);
|
||||
assert.match(result.informational[0], /"publishFromCi": true/);
|
||||
assert.match(result.informational[0], /drop the workspace dependency/);
|
||||
});
|
||||
|
||||
test('notices a newly added dependency on an existing unpublished package even when the manifest is untouched', async () => {
|
||||
const files = [
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
status: 'modified',
|
||||
patch: '@@ -1 +1 @@\n+ "@paperclipai/adapter-hermes-gateway": "workspace:*",',
|
||||
},
|
||||
];
|
||||
|
||||
const manifest = [{ dir: 'g', name: '@paperclipai/adapter-hermes-gateway', publishFromCi: false }];
|
||||
const result = await checkReleaseBootstrap(
|
||||
files,
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({ base: manifest, head: manifest }),
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
assert.match(result.informational[0], /@paperclipai\/adapter-hermes-gateway/);
|
||||
assert.match(result.informational[0], /depend on it/);
|
||||
});
|
||||
|
||||
test('stays quiet for a publishFromCi:false package nothing depends on', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/deliberately-private', publishFromCi: false }],
|
||||
}),
|
||||
registryPackageExists: async () => { throw new Error('should not look up'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('never looks up names outside the @paperclipai scope', async () => {
|
||||
const lookedUp = [];
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [
|
||||
{ dir: 'x', name: '@evil/probe', publishFromCi: true },
|
||||
{ dir: 'y', name: 'unscoped-name', publishFromCi: true },
|
||||
{ dir: 'z', name: '@paperclipai/UPPER', publishFromCi: true },
|
||||
],
|
||||
}),
|
||||
registryPackageExists: async (name) => {
|
||||
lookedUp.push(name);
|
||||
return false;
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(lookedUp, []);
|
||||
assert.deepEqual(result.informational, []);
|
||||
});
|
||||
|
||||
test('stays quiet when the registry lookup fails', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: stubGitHub({
|
||||
base: [],
|
||||
head: [{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true }],
|
||||
}),
|
||||
registryPackageExists: async () => { throw new Error('registry down'); },
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(result, { passed: true, informational: [] });
|
||||
});
|
||||
|
||||
test('treats a missing base manifest as empty (every head entry is new)', async () => {
|
||||
const result = await checkReleaseBootstrap(
|
||||
[manifestChangedFile],
|
||||
'token',
|
||||
'paperclipai/paperclip',
|
||||
9967,
|
||||
'master',
|
||||
{
|
||||
fetchFromGitHub: async (path) => {
|
||||
if (path.includes('ref=refs%2Fpull%2F')) {
|
||||
return encodeManifest([{ dir: 'b', name: '@paperclipai/brand-new', publishFromCi: true }]);
|
||||
}
|
||||
throw new Error('404 base manifest');
|
||||
},
|
||||
registryPackageExists: async () => false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.informational.length, 1);
|
||||
});
|
||||
|
||||
test('addedWorkspaceDependencyNames reads only added lines of package.json patches', () => {
|
||||
const names = addedWorkspaceDependencyNames([
|
||||
{
|
||||
filename: 'server/package.json',
|
||||
patch: [
|
||||
'@@ -1,3 +1,4 @@',
|
||||
' "@paperclipai/context-line": "workspace:*",',
|
||||
'- "@paperclipai/removed-dep": "workspace:*",',
|
||||
'+ "@paperclipai/added-dep": "workspace:*",',
|
||||
].join('\n'),
|
||||
},
|
||||
{ filename: 'ui/src/index.ts', patch: '+ "@paperclipai/not-a-pkg-json": "workspace:*",' },
|
||||
{ filename: 'node_modules/x/package.json', patch: '+ "@paperclipai/vendored": "workspace:*",' },
|
||||
]);
|
||||
|
||||
assert.deepEqual([...names], ['@paperclipai/added-dep']);
|
||||
});
|
||||
Reference in new issue
Block a user