Files
PaperClipAI/.github/scripts/tests/check-lockfile-release-age.test.mjs
T
Nicky LeachandClaude Opus 5.5 e3041f5481 fix(ci): let the lockfile bot own the sandbox provider lockfiles (#19522)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The repository has a supply-chain rule: contributors do not commit
lockfiles. A bot and verified Dependabot commits write them.
> - The rule checks only the root `pnpm-lock.yaml`.
> - The sandbox providers are outside the root workspace, so they have
two more pnpm lockfiles. Pull requests edited them by hand, for example
#12791, #18839 and #19489.
> - This pull request makes the same rule and the same bot cover those
two lockfiles.
> - The benefit is one lockfile rule for every managed pnpm lockfile,
and no lockfile edits in contributor pull requests.

## Linked Issues or Issue Description

Refs #19489

**What existing behavior does this improve?**

The lockfile policy and the lockfile refresh bot.

**Subsystem affected**

CI: `.github/scripts/lockfile-trust.mjs`,
`.github/workflows/refresh-lockfile.yml`, and the sandbox provider
installs.

**Current behavior**

`lockfile-trust.mjs` blocks contributor edits to `pnpm-lock.yaml` only.
A contributor can edit
`scripts/typecheck-sandbox-providers/pnpm-lock.yaml` or
`packages/plugins/sandbox-providers/daytona/pnpm-lock.yaml`, and CI
passes. The refresh bot does not update these files.
`typecheck:test-programs` fails on a stale provider lockfile and tells
the contributor to commit a new one.

**Proposed behavior**

The policy blocks contributor edits to all three lockfiles. The refresh
bot regenerates all three after merge. CI resolves a stale provider
lockfile in place for the run, as it does for the root lockfile.

**Reason and benefit**

Lockfile content comes only from the bot or from verified Dependabot
commits. Contributors do not need to run pnpm in special folders.

**Breaking changes**

None. A pull request that edits a provider lockfile now fails the policy
check. Remove the lockfile change from the pull request. The bot commits
it after merge.

## What Changed

- `lockfile-trust.mjs` now exports `LOCKFILES`: the root lockfile, the
shared sandbox provider lockfile, and the Daytona lockfile. The policy
blocks contributor edits to each of them. A Dependabot pull request must
match a verified Dependabot commit for each lockfile that it changes.
The `merged` command and the reporting-lock proof still use only the
root lockfile.
- `check-pr-lockfile.mjs` (advisory check) uses the same list.
- `refresh-lockfile.yml`:
- A job-level `LOCKFILES` env lists the same three files. A test checks
that it equals the policy list.
- A new step regenerates the two provider lockfiles with
`--resolution-only --ignore-scripts --no-frozen-lockfile`.
- The unexpected-changes check, the change check, and `git add` cover
all three lockfiles.
  - The release-age gate checks the added versions in every lockfile.
- A held refresh saves all three lockfiles at their repository paths,
with a `source-locks.sha256` file. The next run reuses them only if
every master lockfile is unchanged. An artifact in the old format is not
reused.
  - The PR title is now `chore(lockfile): refresh pnpm lockfiles`.
- `check-lockfile-release-age.mjs` accepts more than one
`--base`/`--head` pair.
- `scripts/typecheck-test-programs.mjs`: if the frozen install of the
provider workspace fails, the script resolves the lockfile in place and
installs frozen again.
- `scripts/build-standalone-public-packages.mjs`: the same fallback
applies to a package with a committed lockfile (Daytona). `release.sh`
already restores tracked files when it exits.
- `doc/DEVELOPING.md` and `doc/RELEASE-AUTOMATION-SETUP.md` describe the
managed lockfiles. They also say that the Pi and Kimi distribution
lockfiles stay hand-managed, because they hold attested provider pins.

## Verification

- `node --test .github/scripts/tests/*.test.mjs
scripts/__tests__/build-standalone-concurrency.test.mjs`: 731 tests
pass. New tests cover:
- the policy for provider lockfiles, including a real git history with a
Dependabot commit and a later hand edit
  - the workflow and policy lists match
  - the unexpected-changes check, run with bash against a fixture
- a held refresh round trip, and its invalidation when a provider
lockfile changes on master
  - the release-age gate across lockfile pairs
  - the inline fallback in both install scripts
- `node --test scripts/typecheck-test-programs.test.mjs`: the install
tests pass. Two tests fail locally only because this worktree has no
root `node_modules` (no `tsc`). They also fail without this change.
- I ran the new bot commands on master with pnpm 9.15.4. No package
version changed. pnpm 9.15.4 removes `libc:` fields and adds
`deprecated:` notes, so the first refresh is a metadata-only change. The
release-age gate passes it because it adds no versions.
- I ran the release-age and held-lock steps locally with bash against
this commit. The gate reported `eligible=true`, and `sha256sum --check`
accepted the saved hashes.
- The workflow YAML parses, and `bash -n` accepts every `run` block.
- Not testable before merge: the policy job loads `lockfile-trust.mjs`
from the base branch, so the new rule applies only to pull requests
opened after this merges. The refresh workflow runs only on master.

## Risks

- The release build now resolves a stale Daytona lockfile in place, from
the registry. The root release install already resolves without a frozen
lockfile, so this is the same level of trust.
- The paid runner e2e job still installs Daytona with
`--frozen-lockfile`. After a Daytona manifest change merges, its Daytona
cases fail until the bot's refresh merges. This matches the rule that
paid jobs refuse a new root lockfile until its refresh merges.
- Dependabot pull requests for the provider entries in #19489 change a
provider lockfile. They pass only if each lockfile matches a verified
Dependabot commit, the same as for the root lockfile.
- Low risk otherwise: the bot keeps its existing token, environment, and
merge rules.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, and git).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 10:07:57 -08:00

182 lines
6.7 KiB
JavaScript

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import path from 'node:path';
import {
addedPackageVersions,
addedPackageVersionsAcross,
checkReleaseAge,
formatReport,
parseLockfilePackages,
} from '../check-lockfile-release-age.mjs';
const DAY_MS = 24 * 60 * 60 * 1000;
const now = new Date('2026-10-30T00:00:00Z');
const daysAgo = days => new Date(now.getTime() - days * DAY_MS);
const REGISTRY = '{integrity: sha512-x}';
// Each entry is a key, or [key, resolution] for a non-registry resolution.
const lockfile = entries => [
"lockfileVersion: '9.0'",
'',
'importers:',
'',
' .:',
' dependencies:',
' left-pad:',
' specifier: ^1.3.0',
' version: 1.3.0',
'',
'packages:',
'',
...entries.flatMap(entry => {
const [key, resolution] = Array.isArray(entry) ? entry : [entry, REGISTRY];
return [` ${key}:`, ` resolution: ${resolution}`, ' engines: {node: \'>=18\'}', ''];
}),
'snapshots:',
'',
' snapshot-only@9.9.9:',
' dependencies: {}',
'',
].join('\n');
const timesFrom = table => async (name, version) => {
const value = table[`${name}@${version}`];
if (value instanceof Error) throw value;
return value ?? null;
};
test('parses plain, quoted and scoped package keys from the packages section only', () => {
const packages = parseLockfilePackages(lockfile([
"'@scope/pkg@2.0.0'",
'left-pad@1.3.0',
['gitdep@https://codeload.github.com/o/r/tar.gz/abc', '{tarball: https://codeload.github.com/o/r/tar.gz/abc}'],
]));
assert.deepEqual([...packages].sort(), [
['@scope/pkg@2.0.0', REGISTRY],
['gitdep@https://codeload.github.com/o/r/tar.gz/abc', '{tarball: https://codeload.github.com/o/r/tar.gz/abc}'],
['left-pad@1.3.0', REGISTRY],
]);
});
test('lists only package versions the refresh adds', () => {
const base = lockfile(['left-pad@1.3.0', 'old@1.0.0']);
const head = lockfile(['left-pad@1.3.0', 'old@1.1.0', "'@scope/new@0.1.0'"]);
assert.deepEqual(addedPackageVersions(base, head), [
{ key: '@scope/new@0.1.0', name: '@scope/new', version: '0.1.0', resolution: REGISTRY },
{ key: 'old@1.1.0', name: 'old', version: '1.1.0', resolution: REGISTRY },
]);
});
test('lists the versions each lockfile adds, once', () => {
const pairs = [
[lockfile(['left-pad@1.3.0']), lockfile(['left-pad@1.3.0', 'shared@2.0.0', 'root-only@1.0.0'])],
[lockfile([]), lockfile(['shared@2.0.0', 'provider-only@3.0.0'])],
];
assert.deepEqual(addedPackageVersionsAcross(pairs).map(entry => entry.key), ['provider-only@3.0.0', 'root-only@1.0.0', 'shared@2.0.0']);
});
test('the CLI rejects unpaired lockfiles before any registry lookup', () => {
const root = mkdtempSync(path.join(tmpdir(), 'release-age-cli-'));
try {
const file = path.join(root, 'pnpm-lock.yaml');
writeFileSync(file, lockfile([]));
const result = spawnSync(process.execPath, [
path.resolve('.github/scripts/check-lockfile-release-age.mjs'),
'--base', file, '--head', file, '--base', file, '--min-age-days', '3',
], { encoding: 'utf8' });
assert.equal(result.status, 2, result.stderr);
assert.match(result.stderr, /Usage/);
} finally {
rmSync(root, { recursive: true, force: true });
}
});
test('lists an existing version whose resolution changed', () => {
const swapped = ['left-pad@1.3.0', '{tarball: https://example.com/left-pad-1.3.0.tgz}'];
assert.deepEqual(addedPackageVersions(lockfile(['left-pad@1.3.0']), lockfile([swapped])), [
{ key: 'left-pad@1.3.0', name: 'left-pad', version: '1.3.0', resolution: swapped[1] },
]);
});
test('holds unquoted URL keys and registry-looking keys with non-registry resolutions', async () => {
const entries = addedPackageVersions(lockfile([]), lockfile([
['gitdep@https://codeload.github.com/o/r/tar.gz/abc', '{tarball: https://codeload.github.com/o/r/tar.gz/abc}'],
['sneaky@1.0.0', '{integrity: sha512-x, tarball: https://example.com/sneaky.tgz}'],
]));
const result = await checkReleaseAge(entries, {
fetchPublishTime: timesFrom({ 'sneaky@1.0.0': daysAgo(400) }),
now,
minAgeMs: DAY_MS,
});
assert.equal(result.eligible, false);
assert.equal(result.checked, 2);
assert.deepEqual(
Object.fromEntries(result.held.map(h => [h.name, h.reason])),
{
gitdep: 'not a registry version',
sneaky: 'not a registry resolution: {integrity: sha512-x, tarball: https://example.com/sneaky.tgz}',
},
);
});
test('is eligible when every added version is old enough', async () => {
const entries = addedPackageVersions(lockfile([]), lockfile(['a@1.0.0', "'@s/b@2.0.0'"]));
const result = await checkReleaseAge(entries, {
fetchPublishTime: timesFrom({ 'a@1.0.0': daysAgo(30), '@s/b@2.0.0': daysAgo(3.5) }),
now,
minAgeMs: 3 * DAY_MS,
});
assert.deepEqual(result, { eligible: true, checked: 2, held: [] });
});
test('is eligible when the refresh adds no package versions', async () => {
const result = await checkReleaseAge([], { fetchPublishTime: timesFrom({}), now, minAgeMs: DAY_MS });
assert.equal(result.eligible, true);
});
test('holds a version newer than the minimum age', async () => {
const entries = addedPackageVersions(lockfile([]), lockfile(['old@1.0.0', 'fresh@1.0.1']));
const result = await checkReleaseAge(entries, {
fetchPublishTime: timesFrom({ 'old@1.0.0': daysAgo(30), 'fresh@1.0.1': daysAgo(0.5) }),
now,
minAgeMs: 3 * DAY_MS,
});
assert.equal(result.eligible, false);
assert.deepEqual(result.held.map(h => h.key), ['fresh@1.0.1']);
assert.match(result.held[0].reason, /0\.5 days ago/);
});
test('fails closed on lookup errors, missing publish times and non-registry versions', async () => {
const entries = addedPackageVersions(
lockfile([]),
lockfile(['broken@1.0.0', 'unpublished@1.0.0', "'gitdep@https://codeload.github.com/o/r/tar.gz/abc'"]),
);
const result = await checkReleaseAge(entries, {
fetchPublishTime: timesFrom({ 'broken@1.0.0': new Error('npm registry returned 503') }),
now,
minAgeMs: DAY_MS,
});
assert.equal(result.eligible, false);
assert.deepEqual(
Object.fromEntries(result.held.map(h => [h.name, h.reason])),
{
broken: 'registry lookup failed: npm registry returned 503',
gitdep: 'not a registry version',
unpublished: 'no publish time on npm',
},
);
});
test('report lists held versions behind a stable marker', () => {
const report = formatReport(
{ eligible: false, checked: 1, held: [{ key: 'fresh@1.0.1', reason: 'too new' }] },
3,
);
assert.match(report, /^<!-- lockfile-release-age -->/);
assert.match(report, /- `fresh@1\.0\.1`: too new/);
});