Files
PaperClipAI/.github/scripts/check-lockfile-release-age.mjs
T
Nicky LeachandClaude Opus 5.5 e3041f5481 fix(ci): let the lockfile bot own the sandbox provider lockfiles (#19522)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The repository has a supply-chain rule: contributors do not commit
lockfiles. A bot and verified Dependabot commits write them.
> - The rule checks only the root `pnpm-lock.yaml`.
> - The sandbox providers are outside the root workspace, so they have
two more pnpm lockfiles. Pull requests edited them by hand, for example
#12791, #18839 and #19489.
> - This pull request makes the same rule and the same bot cover those
two lockfiles.
> - The benefit is one lockfile rule for every managed pnpm lockfile,
and no lockfile edits in contributor pull requests.

## Linked Issues or Issue Description

Refs #19489

**What existing behavior does this improve?**

The lockfile policy and the lockfile refresh bot.

**Subsystem affected**

CI: `.github/scripts/lockfile-trust.mjs`,
`.github/workflows/refresh-lockfile.yml`, and the sandbox provider
installs.

**Current behavior**

`lockfile-trust.mjs` blocks contributor edits to `pnpm-lock.yaml` only.
A contributor can edit
`scripts/typecheck-sandbox-providers/pnpm-lock.yaml` or
`packages/plugins/sandbox-providers/daytona/pnpm-lock.yaml`, and CI
passes. The refresh bot does not update these files.
`typecheck:test-programs` fails on a stale provider lockfile and tells
the contributor to commit a new one.

**Proposed behavior**

The policy blocks contributor edits to all three lockfiles. The refresh
bot regenerates all three after merge. CI resolves a stale provider
lockfile in place for the run, as it does for the root lockfile.

**Reason and benefit**

Lockfile content comes only from the bot or from verified Dependabot
commits. Contributors do not need to run pnpm in special folders.

**Breaking changes**

None. A pull request that edits a provider lockfile now fails the policy
check. Remove the lockfile change from the pull request. The bot commits
it after merge.

## What Changed

- `lockfile-trust.mjs` now exports `LOCKFILES`: the root lockfile, the
shared sandbox provider lockfile, and the Daytona lockfile. The policy
blocks contributor edits to each of them. A Dependabot pull request must
match a verified Dependabot commit for each lockfile that it changes.
The `merged` command and the reporting-lock proof still use only the
root lockfile.
- `check-pr-lockfile.mjs` (advisory check) uses the same list.
- `refresh-lockfile.yml`:
- A job-level `LOCKFILES` env lists the same three files. A test checks
that it equals the policy list.
- A new step regenerates the two provider lockfiles with
`--resolution-only --ignore-scripts --no-frozen-lockfile`.
- The unexpected-changes check, the change check, and `git add` cover
all three lockfiles.
  - The release-age gate checks the added versions in every lockfile.
- A held refresh saves all three lockfiles at their repository paths,
with a `source-locks.sha256` file. The next run reuses them only if
every master lockfile is unchanged. An artifact in the old format is not
reused.
  - The PR title is now `chore(lockfile): refresh pnpm lockfiles`.
- `check-lockfile-release-age.mjs` accepts more than one
`--base`/`--head` pair.
- `scripts/typecheck-test-programs.mjs`: if the frozen install of the
provider workspace fails, the script resolves the lockfile in place and
installs frozen again.
- `scripts/build-standalone-public-packages.mjs`: the same fallback
applies to a package with a committed lockfile (Daytona). `release.sh`
already restores tracked files when it exits.
- `doc/DEVELOPING.md` and `doc/RELEASE-AUTOMATION-SETUP.md` describe the
managed lockfiles. They also say that the Pi and Kimi distribution
lockfiles stay hand-managed, because they hold attested provider pins.

## Verification

- `node --test .github/scripts/tests/*.test.mjs
scripts/__tests__/build-standalone-concurrency.test.mjs`: 731 tests
pass. New tests cover:
- the policy for provider lockfiles, including a real git history with a
Dependabot commit and a later hand edit
  - the workflow and policy lists match
  - the unexpected-changes check, run with bash against a fixture
- a held refresh round trip, and its invalidation when a provider
lockfile changes on master
  - the release-age gate across lockfile pairs
  - the inline fallback in both install scripts
- `node --test scripts/typecheck-test-programs.test.mjs`: the install
tests pass. Two tests fail locally only because this worktree has no
root `node_modules` (no `tsc`). They also fail without this change.
- I ran the new bot commands on master with pnpm 9.15.4. No package
version changed. pnpm 9.15.4 removes `libc:` fields and adds
`deprecated:` notes, so the first refresh is a metadata-only change. The
release-age gate passes it because it adds no versions.
- I ran the release-age and held-lock steps locally with bash against
this commit. The gate reported `eligible=true`, and `sha256sum --check`
accepted the saved hashes.
- The workflow YAML parses, and `bash -n` accepts every `run` block.
- Not testable before merge: the policy job loads `lockfile-trust.mjs`
from the base branch, so the new rule applies only to pull requests
opened after this merges. The refresh workflow runs only on master.

## Risks

- The release build now resolves a stale Daytona lockfile in place, from
the registry. The root release install already resolves without a frozen
lockfile, so this is the same level of trust.
- The paid runner e2e job still installs Daytona with
`--frozen-lockfile`. After a Daytona manifest change merges, its Daytona
cases fail until the bot's refresh merges. This matches the rule that
paid jobs refuse a new root lockfile until its refresh merges.
- Dependabot pull requests for the provider entries in #19489 change a
provider lockfile. They pass only if each lockfile matches a verified
Dependabot commit, the same as for the root lockfile.
- Low risk otherwise: the bot keeps its existing token, environment, and
merge rules.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, and git).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 10:07:57 -08:00

175 lines
7.1 KiB
JavaScript

#!/usr/bin/env node
/**
* check-lockfile-release-age.mjs
* Decides whether the lockfile bot may merge refreshed pnpm lockfiles without review.
* Every package version that the refresh adds must be published on npm for at least
* the minimum age. A version that is too new, or that cannot be verified, holds the
* PR for a human.
*
* Exports: parseLockfilePackages(text), addedPackageVersions(baseText, headText),
* addedPackageVersionsAcross(pairs),
* checkReleaseAge(entries, { fetchPublishTime, now, minAgeMs, concurrency })
*
* CLI: node check-lockfile-release-age.mjs --base <file> --head <file> [--base <file> --head <file> ...]
* --min-age-days <n>
* Pass one --base/--head pair per lockfile, in the same order. Writes `eligible=true|false`
* to GITHUB_OUTPUT and a Markdown report to --report.
*/
import { appendFileSync, readFileSync, writeFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { parseArgs } from 'node:util';
const SEMVER = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$/;
// Every 2-space-indented line in `packages:` is an entry key. Keys can contain
// colons (URL and git dependencies), so match up to the final colon.
const PACKAGE_KEY = /^ {2}(\S.*):\s*$/;
const RESOLUTION = /^ {4}resolution: (.+?)\s*$/;
// A registry tarball is locked by integrity alone. A tarball URL, git repo or
// directory resolution names its own source and needs a human.
const REGISTRY_RESOLUTION = /^\{integrity: [^,{}]+\}$/;
const DAY_MS = 24 * 60 * 60 * 1000;
/** Maps each key of the lockfile's top-level `packages:` section to its resolution. */
export function parseLockfilePackages(text) {
const packages = new Map();
let inPackages = false;
let key = null;
for (const line of text.split('\n')) {
if (/^\S/.test(line)) {
inPackages = line.trimEnd() === 'packages:';
key = null;
continue;
}
if (!inPackages) continue;
const keyMatch = PACKAGE_KEY.exec(line);
if (keyMatch) {
key = keyMatch[1].replace(/^'(.*)'$/, '$1').replace(/\(.*$/, '');
packages.set(key, '');
continue;
}
const resolutionMatch = key && RESOLUTION.exec(line);
if (resolutionMatch) packages.set(key, resolutionMatch[1]);
}
return packages;
}
export function splitPackageKey(key) {
const at = key.lastIndexOf('@');
if (at <= 0) return { name: key, version: '' };
return { name: key.slice(0, at), version: key.slice(at + 1) };
}
/** Package entries in headText that baseText lacks or resolves differently. */
export function addedPackageVersions(baseText, headText) {
const base = parseLockfilePackages(baseText);
return [...parseLockfilePackages(headText)]
.filter(([key, resolution]) => base.get(key) !== resolution)
.sort(([a], [b]) => a.localeCompare(b))
.map(([key, resolution]) => ({ key, ...splitPackageKey(key), resolution }));
}
/** Added versions across several base/head lockfile pairs, each listed once. */
export function addedPackageVersionsAcross(pairs) {
const entries = new Map();
for (const [baseText, headText] of pairs) {
for (const entry of addedPackageVersions(baseText, headText)) entries.set(`${entry.key}\0${entry.resolution}`, entry);
}
return [...entries.values()].sort((a, b) => a.key.localeCompare(b.key));
}
export async function fetchNpmPublishTime(name, version, fetchImpl = fetch) {
const res = await fetchImpl(`https://registry.npmjs.org/${name.replace('/', '%2f')}`, {
headers: { Accept: 'application/json' },
signal: AbortSignal.timeout(30_000),
});
if (!res.ok) throw new Error(`npm registry returned ${res.status}`);
const doc = await res.json();
const published = doc.time?.[version];
return published ? new Date(published) : null;
}
/**
* Checks each added version. Fails closed: a lookup error, a missing publish time or a
* non-registry version holds the merge.
*/
export async function checkReleaseAge(entries, {
fetchPublishTime = fetchNpmPublishTime,
now = new Date(),
minAgeMs,
concurrency = 8,
}) {
const held = [];
const queue = [...entries];
const worker = async () => {
for (let entry = queue.shift(); entry; entry = queue.shift()) {
const reason = await holdReason(entry, fetchPublishTime, now, minAgeMs);
if (reason) held.push({ ...entry, reason });
}
};
await Promise.all(Array.from({ length: concurrency }, worker));
held.sort((a, b) => a.key.localeCompare(b.key));
return { eligible: held.length === 0, checked: entries.length, held };
}
async function holdReason({ name, version, resolution }, fetchPublishTime, now, minAgeMs) {
if (!SEMVER.test(version)) return 'not a registry version';
if (!REGISTRY_RESOLUTION.test(resolution ?? '')) return `not a registry resolution: ${resolution || 'missing'}`;
let published;
try {
published = await fetchPublishTime(name, version);
} catch (error) {
return `registry lookup failed: ${error.message}`;
}
if (!published || Number.isNaN(published.getTime())) return 'no publish time on npm';
const ageMs = now.getTime() - published.getTime();
if (ageMs < minAgeMs) {
return `published ${published.toISOString()} (${(ageMs / DAY_MS).toFixed(1)} days ago)`;
}
return null;
}
export function formatReport(result, minAgeDays) {
const lines = ['<!-- lockfile-release-age -->'];
if (result.eligible) {
lines.push(`All ${result.checked} added package versions are at least ${minAgeDays} days old. The lockfile bot merges this PR.`);
} else {
lines.push(
`The lockfile bot did not merge this PR. These added package versions are newer than ${minAgeDays} days or could not be verified:`,
'',
...result.held.map(({ key, reason }) => `- \`${key}\`: ${reason}`),
'',
'A maintainer can review and merge this PR. If no one does, the daily refresh tries again and merges it when every version is old enough.',
);
}
return `${lines.join('\n')}\n`;
}
async function main() {
const { values } = parseArgs({
options: {
base: { type: 'string', multiple: true },
head: { type: 'string', multiple: true },
'min-age-days': { type: 'string' },
report: { type: 'string' },
},
});
const minAgeDays = Number(values['min-age-days']);
const bases = values.base ?? [];
const heads = values.head ?? [];
if (bases.length === 0 || bases.length !== heads.length || !Number.isFinite(minAgeDays) || minAgeDays < 0) {
console.error('Usage: check-lockfile-release-age.mjs --base <file> --head <file> [--base <file> --head <file> ...] --min-age-days <n> [--report <file>]');
process.exit(2);
}
const entries = addedPackageVersionsAcross(bases.map((base, index) => [readFileSync(base, 'utf8'), readFileSync(heads[index], 'utf8')]));
const result = await checkReleaseAge(entries, { minAgeMs: minAgeDays * DAY_MS });
const report = formatReport(result, minAgeDays);
console.log(report);
if (values.report) writeFileSync(values.report, report);
if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `eligible=${result.eligible}\n`);
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
main().catch(e => { console.error(e.message); process.exit(1); });
}