mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 01:50:13 +01:00
fix(ci): let the lockfile bot own the sandbox provider lockfiles (#19522)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The repository has a supply-chain rule: contributors do not commit lockfiles. A bot and verified Dependabot commits write them. > - The rule checks only the root `pnpm-lock.yaml`. > - The sandbox providers are outside the root workspace, so they have two more pnpm lockfiles. Pull requests edited them by hand, for example #12791, #18839 and #19489. > - This pull request makes the same rule and the same bot cover those two lockfiles. > - The benefit is one lockfile rule for every managed pnpm lockfile, and no lockfile edits in contributor pull requests. ## Linked Issues or Issue Description Refs #19489 **What existing behavior does this improve?** The lockfile policy and the lockfile refresh bot. **Subsystem affected** CI: `.github/scripts/lockfile-trust.mjs`, `.github/workflows/refresh-lockfile.yml`, and the sandbox provider installs. **Current behavior** `lockfile-trust.mjs` blocks contributor edits to `pnpm-lock.yaml` only. A contributor can edit `scripts/typecheck-sandbox-providers/pnpm-lock.yaml` or `packages/plugins/sandbox-providers/daytona/pnpm-lock.yaml`, and CI passes. The refresh bot does not update these files. `typecheck:test-programs` fails on a stale provider lockfile and tells the contributor to commit a new one. **Proposed behavior** The policy blocks contributor edits to all three lockfiles. The refresh bot regenerates all three after merge. CI resolves a stale provider lockfile in place for the run, as it does for the root lockfile. **Reason and benefit** Lockfile content comes only from the bot or from verified Dependabot commits. Contributors do not need to run pnpm in special folders. **Breaking changes** None. A pull request that edits a provider lockfile now fails the policy check. Remove the lockfile change from the pull request. The bot commits it after merge. ## What Changed - `lockfile-trust.mjs` now exports `LOCKFILES`: the root lockfile, the shared sandbox provider lockfile, and the Daytona lockfile. The policy blocks contributor edits to each of them. A Dependabot pull request must match a verified Dependabot commit for each lockfile that it changes. The `merged` command and the reporting-lock proof still use only the root lockfile. - `check-pr-lockfile.mjs` (advisory check) uses the same list. - `refresh-lockfile.yml`: - A job-level `LOCKFILES` env lists the same three files. A test checks that it equals the policy list. - A new step regenerates the two provider lockfiles with `--resolution-only --ignore-scripts --no-frozen-lockfile`. - The unexpected-changes check, the change check, and `git add` cover all three lockfiles. - The release-age gate checks the added versions in every lockfile. - A held refresh saves all three lockfiles at their repository paths, with a `source-locks.sha256` file. The next run reuses them only if every master lockfile is unchanged. An artifact in the old format is not reused. - The PR title is now `chore(lockfile): refresh pnpm lockfiles`. - `check-lockfile-release-age.mjs` accepts more than one `--base`/`--head` pair. - `scripts/typecheck-test-programs.mjs`: if the frozen install of the provider workspace fails, the script resolves the lockfile in place and installs frozen again. - `scripts/build-standalone-public-packages.mjs`: the same fallback applies to a package with a committed lockfile (Daytona). `release.sh` already restores tracked files when it exits. - `doc/DEVELOPING.md` and `doc/RELEASE-AUTOMATION-SETUP.md` describe the managed lockfiles. They also say that the Pi and Kimi distribution lockfiles stay hand-managed, because they hold attested provider pins. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/__tests__/build-standalone-concurrency.test.mjs`: 731 tests pass. New tests cover: - the policy for provider lockfiles, including a real git history with a Dependabot commit and a later hand edit - the workflow and policy lists match - the unexpected-changes check, run with bash against a fixture - a held refresh round trip, and its invalidation when a provider lockfile changes on master - the release-age gate across lockfile pairs - the inline fallback in both install scripts - `node --test scripts/typecheck-test-programs.test.mjs`: the install tests pass. Two tests fail locally only because this worktree has no root `node_modules` (no `tsc`). They also fail without this change. - I ran the new bot commands on master with pnpm 9.15.4. No package version changed. pnpm 9.15.4 removes `libc:` fields and adds `deprecated:` notes, so the first refresh is a metadata-only change. The release-age gate passes it because it adds no versions. - I ran the release-age and held-lock steps locally with bash against this commit. The gate reported `eligible=true`, and `sha256sum --check` accepted the saved hashes. - The workflow YAML parses, and `bash -n` accepts every `run` block. - Not testable before merge: the policy job loads `lockfile-trust.mjs` from the base branch, so the new rule applies only to pull requests opened after this merges. The refresh workflow runs only on master. ## Risks - The release build now resolves a stale Daytona lockfile in place, from the registry. The root release install already resolves without a frozen lockfile, so this is the same level of trust. - The paid runner e2e job still installs Daytona with `--frozen-lockfile`. After a Daytona manifest change merges, its Daytona cases fail until the bot's refresh merges. This matches the rule that paid jobs refuse a new root lockfile until its refresh merges. - Dependabot pull requests for the provider entries in #19489 change a provider lockfile. They pass only if each lockfile matches a verified Dependabot commit, the same as for the root lockfile. - Low risk otherwise: the bot keeps its existing token, environment, and merge rules. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, and git). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1485d3de1b
commit
e3041f5481
14 files changed
+443
-91
No files matched your search
@@ -24,6 +24,12 @@ jobs:
|
||||
actions: read
|
||||
contents: write
|
||||
pull-requests: write
|
||||
env:
|
||||
# Every lockfile this workflow owns: the root one, then the sandbox
|
||||
# providers' own lockfiles. Keep it equal to LOCKFILES in
|
||||
# .github/scripts/lockfile-trust.mjs, which blocks contributor edits to
|
||||
# the same files. A test checks that the two lists match.
|
||||
LOCKFILES: pnpm-lock.yaml scripts/typecheck-sandbox-providers/pnpm-lock.yaml packages/plugins/sandbox-providers/daytona/pnpm-lock.yaml
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
@@ -43,9 +49,9 @@ jobs:
|
||||
# claim the shared cache key with an empty archive before full installs.
|
||||
package-manager-cache: false
|
||||
|
||||
# Start from the lockfile the previous run held, so resolution keeps those
|
||||
# Start from the lockfiles the previous run held, so resolution keeps those
|
||||
# versions while their manifests are unchanged and they can age past the
|
||||
# gate. Read it from that run's artifact, not the PR branch: anyone with
|
||||
# gate. Read them from that run's artifact, not the PR branch: anyone with
|
||||
# write access can push to chore/refresh-lockfile.
|
||||
- name: Reuse versions from a held refresh
|
||||
env:
|
||||
@@ -61,19 +67,34 @@ jobs:
|
||||
echo "The previous refresh held nothing. Resolving from master."
|
||||
exit 0
|
||||
fi
|
||||
# Reuse the held lock only while master still has the lock it was
|
||||
# resolved from. A lock change on master, such as a maintainer's
|
||||
# correction, supersedes it.
|
||||
if [ "$(sha256sum < pnpm-lock.yaml)" != "$(cat "$held/source-lock.sha256")" ]; then
|
||||
echo "The lockfile on master changed after run $run_id held its refresh. Resolving from master."
|
||||
# An artifact saved before the provider lockfiles has no
|
||||
# source-locks.sha256. Resolve from master instead.
|
||||
if [ ! -f "$held/source-locks.sha256" ]; then
|
||||
echo "Run $run_id held its refresh in an older format. Resolving from master."
|
||||
exit 0
|
||||
fi
|
||||
cp "$held/pnpm-lock.yaml" pnpm-lock.yaml
|
||||
echo "Resolving from the lockfile held by run $run_id."
|
||||
# Reuse the held locks only while master still has every lock they
|
||||
# were resolved from. A lock change on master, such as a maintainer's
|
||||
# correction, supersedes them.
|
||||
if ! sha256sum --check --status "$held/source-locks.sha256"; then
|
||||
echo "A lockfile on master changed after run $run_id held its refresh. Resolving from master."
|
||||
exit 0
|
||||
fi
|
||||
for lock in $LOCKFILES; do
|
||||
cp "$held/$lock" "$lock"
|
||||
done
|
||||
echo "Resolving from the lockfiles held by run $run_id."
|
||||
|
||||
- name: Refresh pnpm lockfile
|
||||
run: pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
|
||||
|
||||
# Sandbox providers stay out of the root workspace. The shared typecheck
|
||||
# workspace has its own pnpm-workspace.yaml, and Daytona installs alone.
|
||||
- name: Refresh sandbox provider lockfiles
|
||||
run: |
|
||||
pnpm --dir scripts/typecheck-sandbox-providers install --resolution-only --ignore-scripts --no-frozen-lockfile
|
||||
pnpm --dir packages/plugins/sandbox-providers/daytona install --ignore-workspace --resolution-only --ignore-scripts --no-frozen-lockfile
|
||||
|
||||
# A verified Dependabot lockfile merged to master is trusted lock content.
|
||||
# Report its hash so the proof refresh can admit it.
|
||||
- name: Identify a verified Dependabot lockfile on master
|
||||
@@ -99,7 +120,9 @@ jobs:
|
||||
echo "Lockfile is already up to date."
|
||||
exit 0
|
||||
fi
|
||||
unexpected="$(printf '%s\n' "$changed" | awk '$2 != "pnpm-lock.yaml" && $2 != "tests/runner-e2e/report-lock.json" && $2 != "tests/runner-e2e/report-lock.patch" { print }')"
|
||||
unexpected="$(printf '%s\n' "$changed" | awk -v allowed="$LOCKFILES tests/runner-e2e/report-lock.json tests/runner-e2e/report-lock.patch" '
|
||||
BEGIN { n = split(allowed, files, " "); for (i = 1; i <= n; i++) ok[files[i]] = 1 }
|
||||
!($2 in ok) { print }')"
|
||||
if [ -n "$unexpected" ]; then
|
||||
echo "Unexpected files changed during lockfile refresh:"
|
||||
echo "$changed"
|
||||
@@ -114,8 +137,8 @@ jobs:
|
||||
run: |
|
||||
# A verified Dependabot lockfile that already resolves changes only
|
||||
# the reporting-lock proof. Commit that proof too.
|
||||
if git diff --quiet -- pnpm-lock.yaml tests/runner-e2e/report-lock.json tests/runner-e2e/report-lock.patch; then
|
||||
echo "Lockfile and reporting-lock proof unchanged, nothing to do."
|
||||
if git diff --quiet -- $LOCKFILES tests/runner-e2e/report-lock.json tests/runner-e2e/report-lock.patch; then
|
||||
echo "Lockfiles and reporting-lock proof unchanged, nothing to do."
|
||||
echo "pr_url=" >> "$GITHUB_OUTPUT"
|
||||
echo "head_sha=" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
@@ -126,8 +149,8 @@ jobs:
|
||||
git config user.email "lockfile-bot@users.noreply.github.com"
|
||||
|
||||
git checkout -B "$BRANCH"
|
||||
git add pnpm-lock.yaml tests/runner-e2e/report-lock.json tests/runner-e2e/report-lock.patch
|
||||
git commit -m "chore(lockfile): refresh pnpm-lock.yaml"
|
||||
git add $LOCKFILES tests/runner-e2e/report-lock.json tests/runner-e2e/report-lock.patch
|
||||
git commit -m "chore(lockfile): refresh pnpm lockfiles"
|
||||
git push --force origin "$BRANCH"
|
||||
|
||||
# Only reuse an open PR from this repository owner, not a fork with the same branch name.
|
||||
@@ -164,6 +187,7 @@ jobs:
|
||||
## What Changed
|
||||
|
||||
- Refresh the dependency lock from the manifests on master.
|
||||
- Refresh the sandbox provider lockfiles from their manifests on master.
|
||||
- Include the matching offline report correction and hashes.
|
||||
|
||||
## Verification
|
||||
@@ -188,7 +212,7 @@ jobs:
|
||||
PR_BODY
|
||||
pr_url="$(gh pr create \
|
||||
--head "$BRANCH" \
|
||||
--title "chore(lockfile): refresh pnpm-lock.yaml" \
|
||||
--title "chore(lockfile): refresh pnpm lockfiles" \
|
||||
--body-file "$pr_body_file")"
|
||||
echo "Created new PR: $pr_url"
|
||||
else
|
||||
@@ -204,10 +228,18 @@ jobs:
|
||||
id: release-age
|
||||
if: steps.upsert-pr.outputs.pr_url != ''
|
||||
run: |
|
||||
git show "$GITHUB_SHA:pnpm-lock.yaml" > "$RUNNER_TEMP/base-pnpm-lock.yaml"
|
||||
# Keep each base lockfile at its repository path. "Stage held
|
||||
# lockfile" hashes them from there.
|
||||
base="$RUNNER_TEMP/base-locks"
|
||||
pairs=()
|
||||
for lock in $LOCKFILES; do
|
||||
mkdir -p "$base/$(dirname "$lock")"
|
||||
# A lockfile that master does not have yet has no base, so all of its entries count as added.
|
||||
git show "$GITHUB_SHA:$lock" > "$base/$lock" 2>/dev/null || : > "$base/$lock"
|
||||
pairs+=(--base "$base/$lock" --head "$lock")
|
||||
done
|
||||
node .github/scripts/check-lockfile-release-age.mjs \
|
||||
--base "$RUNNER_TEMP/base-pnpm-lock.yaml" \
|
||||
--head pnpm-lock.yaml \
|
||||
"${pairs[@]}" \
|
||||
--min-age-days 3 \
|
||||
--report "$RUNNER_TEMP/release-age-report.md"
|
||||
cat "$RUNNER_TEMP/release-age-report.md" >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -236,9 +268,14 @@ jobs:
|
||||
- name: Stage held lockfile
|
||||
if: steps.release-age.outputs.eligible == 'false'
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/held-upload"
|
||||
cp pnpm-lock.yaml "$RUNNER_TEMP/held-upload/pnpm-lock.yaml"
|
||||
sha256sum < "$RUNNER_TEMP/base-pnpm-lock.yaml" > "$RUNNER_TEMP/held-upload/source-lock.sha256"
|
||||
held="$RUNNER_TEMP/held-upload"
|
||||
for lock in $LOCKFILES; do
|
||||
mkdir -p "$held/$(dirname "$lock")"
|
||||
cp "$lock" "$held/$lock"
|
||||
done
|
||||
# Hash the master lockfiles this refresh started from, by repository
|
||||
# path, so the next run can check them with sha256sum --check.
|
||||
(cd "$RUNNER_TEMP/base-locks" && sha256sum $LOCKFILES) > "$held/source-locks.sha256"
|
||||
|
||||
- name: Save held lockfile for the next refresh
|
||||
if: steps.release-age.outputs.eligible == 'false'
|
||||
|
||||
Reference in new issue
Block a user