Commit Graph
2 Commits
Author SHA1 Message Date
Nicky LeachandClaude Opus 5.5 18ff71bca3 fix(ci): load the lockfile policy from the current base-branch tip (#19390)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Only trusted environments may change `pnpm-lock.yaml`. #19356 moved
that rule into `.github/scripts/lockfile-trust.mjs`.
> - The `pr-trusted.yml` policy job loads that script from trusted code,
so a pull request cannot change the check that judges it.
> - The job reads the trusted commit from `pull_request.base.sha`.
GitHub does not move that value forward on every push.
> - #17364 showed the result. Its recorded base was still the `master`
commit from before #19356, so the job used the earlier rule.
> - The earlier rule exempts any branch named `chore/refresh-lockfile`,
including one from a fork.
> - This pull request loads the policy from the first parent of GitHub's
merge commit, which is the current base-branch tip.
> - The benefit is that every pull request gets the current lockfile
policy, including pull requests opened before #19356.

## Linked Issues or Issue Description

Refs #19356, #17364.

**What happened?**
On #17364 at `482d76686e`, the "Block untrusted lockfile edits" step
printed nothing and passed. `pull_request.base.sha` was `da3084a05d`,
the `master` commit from before #19356. That commit has no
`lockfile-trust.mjs`, so the step used the earlier rule. The merge
commit that CI tested already contained #19356.

**Expected behavior**
The policy job runs the current `lockfile-trust.mjs` from `master` for
every pull request.

**Steps to reproduce**
1. Open a pull request before #19356 merged, or keep a pull request
whose recorded base predates it.
2. Push a commit. GitHub keeps the old `pull_request.base.sha`.
3. The policy job uses the earlier rule. A fork branch named
`chore/refresh-lockfile` can still change `pnpm-lock.yaml`.

**Also found**
In a local test, the script exited 0 without any check when it ran from
a symlinked temporary directory. Its entry-point check compared a
symlinked path with a real path. GitHub-hosted Linux runners do not use
a symlinked `RUNNER_TEMP`, but the check must fail closed.

## What Changed

- `pr-trusted.yml` "Block untrusted lockfile edits":
- Load `lockfile-trust.mjs` from `$GITHUB_SHA^1`, the first parent of
GitHub's merge commit. Do this only when `$GITHUB_SHA^2` is the PR head.
Otherwise, keep `pull_request.base.sha`.
  - Give the script the same trusted base for its diff.
  - Fail the step if the script writes no `writer=` output.
- `lockfile-trust.mjs`: compare real paths in the entry-point check.
- `lockfile-trust.test.mjs`: run the real step script against a git
history with a stale recorded base and a merge commit. Check that it
blocks a contributor lockfile change on a `chore/refresh-lockfile`
branch, allows a docs-only change, and falls back to the recorded base
when no merge commit is present.
- `doc/RELEASE-AUTOMATION-SETUP.md`: state where the job loads the
policy from.

## Verification

- `node --test '.github/scripts/tests/*.test.mjs'`: 709 passed.
- The new stale-base test fails when the workflow change is removed.
- The policy-step tests run with `RUNNER_TEMP` behind a symlink. They
fail when the realpath fix is removed.
- A test loads a policy that exits 0 without writing `writer=`. It fails
when the fail-closed check is removed.
- This pull request's own CI uses `pr-trusted.yml` from `master`. The
new step takes effect after merge.

## Risks

- If the checkout is not GitHub's merge commit, the step falls back to
`pull_request.base.sha`, which is the current behavior. The
`pull_request` event always checks out the merge commit by default.
- The fail-closed check can fail a pull request if the script stops
writing `writer=`. The script writes it on every path that returns.
- Low risk otherwise. The change touches one workflow step, one script
line, tests, and docs.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution, in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 07:25:41 -08:00
Nicky LeachandClaude Opus 5.5 3a8f95d928 fix(ci): let verified Dependabot lockfiles merge through the reporting-lock gate (#19356)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Only trusted environments may change `pnpm-lock.yaml`: the lockfile
refresh workflow and Dependabot. Contributors never commit it.
> - Paid evaluation jobs also install a lock only when the
reporting-lock proof (`tests/runner-e2e/report-lock.json` and `.patch`)
admits it.
> - #19283 added a CI test that checks the committed lock against that
proof. Only the refresh workflow writes the proof, so every Dependabot
lockfile pull request now fails CI. Examples are #17364, #17172, and
#17911.
> - A merged Dependabot lockfile also breaks the refresh workflow.
`prepare-report-lock.mjs --refresh` refuses to renew the proof for a
lock that the proof does not already admit.
> - This pull request lets a verified Dependabot lockfile pass, and it
keeps both lockfile rules strict. It also closes a gap in the
contributor rule.
> - The benefit is that Dependabot security updates can merge again, and
only trusted environments write lock content.

## Linked Issues or Issue Description

Refs #19283, #17364, #17172, #17911, #19303.

**What happened?**
`server/scripts/reporting-lock-policy.test.mjs` fails on each Dependabot
pull request that changes `pnpm-lock.yaml`. The error is "Reporting lock
changed; review the dependency correction before install". A Dependabot
pull request cannot change the proof files, so it can never pass. After
a merge, the lockfile refresh also stops at "Committed report lock is
not reviewed; refusing to refresh its proof".

**Expected behavior**
- Contributors cannot change `pnpm-lock.yaml` or the reporting-lock
proof.
- Dependabot can change `pnpm-lock.yaml`.
- The lockfile refresh workflow can change `pnpm-lock.yaml` and the
proof.
- After a Dependabot lockfile merges, the refresh workflow admits it and
renews the proof.

**Steps to reproduce**
1. Open a Dependabot pull request that changes `pnpm-lock.yaml`, such as
#17364.
2. Let CI run. The `server (9/12)` shard fails on the reporting-lock
policy test.

**Security gap found during this work**
The old rule skipped the lockfile check when `github.head_ref` was
`chore/refresh-lockfile`. A contributor could open a pull request from a
fork branch with that name and commit a lockfile change. The old rule
also exempted any pull request that Dependabot opened, including commits
that other people pushed to its branch.

## What Changed

- Add `.github/scripts/lockfile-trust.mjs`. It decides which writers may
change lock content:
- The refresh workflow: the head is `chore/refresh-lockfile` in this
repository, and `github-actions[bot]` opened the pull request.
- Dependabot: Dependabot opened the pull request from this repository,
and the head `pnpm-lock.yaml` matches a Dependabot commit. That commit
must have `dependabot[bot]` as its author, GitHub (`web-flow`) as its
committer, and a verified signature. The script reads these fields from
the GitHub API.
- The reporting-lock proof files count as lock content. Only the refresh
workflow may change them.
- `pr-trusted.yml` policy job: replace the inline `head_ref` and author
exemption with `lockfile-trust.mjs pr`. The job runs the script copy
from the base commit, so a pull request cannot change the check that
judges it. If the base predates the script, the job uses the earlier
rule.
- `refresh-lockfile.yml`: add `lockfile-trust.mjs merged`. It checks
that the last lockfile change on `master` is a GitHub-signed merge of a
Dependabot pull request, and that the lockfile at the pull request head
matches a verified Dependabot commit. If both checks pass, it gives the
lockfile SHA-256 to `prepare-report-lock.mjs --refresh
--trusted-lock-sha256`.
- `refresh-lockfile.yml` PR step: check `pnpm-lock.yaml` and both proof
files before it exits as unchanged. A refresh that renews only the proof
now opens a pull request.
- `prepare-report-lock.mjs` and its `.d.mts` declaration:
`refreshReportLock` accepts a trusted lock hash. If the trusted lock
already resolves, the proof admits it exactly with an empty patch. If
not, the proof records the resolution as the reviewed correction, as
before.
- `reporting-lock-policy.test.mjs`: skip on Dependabot pull requests.
The policy job verifies those lockfiles. The general test job gets
`PAPERCLIP_CI_PR_AUTHOR` for this purpose. This keeps the test, so it
still merges cleanly with #19303.
- `check-pr-lockfile.mjs` (advisory gate): allow Dependabot lockfile
changes, and flag changes to the proof files.
- `CODEOWNERS`: add the two proof files.
- `doc/RELEASE-AUTOMATION-SETUP.md`: describe the writers, the
`@dependabot recreate` path, and the proof renewal after merge.

## Verification

- `node --test '.github/scripts/tests/*.test.mjs'`: 707 passed. This
includes 17 new tests in `lockfile-trust.test.mjs` and 3 new tests in
`check-pr-lockfile.test.mjs`. The new tests run the `pr` command against
real git histories with a stub `gh`.
- `tests/runner-e2e/prepare-report-lock.test.ts`: 10 passed, including 3
new trusted-lock refresh tests. I ran it with a minimal Vitest config,
because the runner-e2e config builds a provider bundle first.
- `server/scripts/reporting-lock-policy.test.mjs`: passes normally, and
skips with `PAPERCLIP_CI_PR_AUTHOR=dependabot[bot]`.
- Live check of the `pr` mode against #17364 with the real GitHub API:
- With the real author, it reports "pnpm-lock.yaml matches verified
Dependabot commit 83c328c15c…" and `writer=dependabot`.
- With a contributor as the author and `chore/refresh-lockfile` as the
head ref, it fails.
- Live check of the `merged` mode at the real merge commits of #16018,
#16020, and #15361. All three report a trusted lock hash. For #16018 and
#15361, `master` changed after Dependabot's commit, so GitHub's merge
produced different lockfile bytes. An exact-bytes rule would refuse
them. That is why the rule trusts GitHub-signed merges of a verified
Dependabot head.
- Greptile reviewed `513ca2def8` at 4/5 with one P1: a proof-only
refresh was never committed, because the PR step checked only
`pnpm-lock.yaml`. `edbeb1ea0d` fixes it. A new test in
`lockfile-refresh-merge.test.mjs` runs the step's change check on a
proof-only change. The test fails without the fix.
- CI `Typecheck + Release Registry` failed at `edbeb1ea0d`:
`prepare-report-lock.d.mts` did not declare the new options argument
(TS2554 twice in `prepare-report-lock.test.ts`). `9139aaad85` updates
the declaration. A local `tsc -p tests/runner-e2e/tsconfig.json` shows
the 2 errors with the old declaration and none with the new one.
- CI at `9139aaad85` failed in six general test shards. None of them run
code that this pull request changes, and PR CI loads `pr-trusted.yml`
from `master`. The failures:
- `heartbeat-plugin-environment.test.ts`: "expected 'running' to be
'succeeded'"
  - `status-card-evaluation-work.test.ts`: a deadline comparison
  - `isolated-repository-rehome.test.ts`: timed out after 15 seconds
- `chat-channels.integration.test.ts`: GitHub webhook recovery
scheduling
- `company-import-export-e2e.test.ts`: child server start timeout, which
also fails on unrelated branches
- `workspaces-b`: every test passed, but an `EPIPE` error came after
`acpx-engine/execute.test.ts`
  I started a rerun of the failed jobs.
- Superagent found a P1 at `9139aaad85`: a verified signature proves who
committed, not who authored. A collaborator could sign a commit with
their own key and set Dependabot as the author. `4fe180d6c1` also
requires GitHub (`web-flow`) as the committer. New tests cover the
spoofed-author case. #17364, #16018, #16020 and #15361 still verify
against the GitHub API.
- `git merge-file` merges this test change with #19303 without
conflicts.
- Not run: the full repository test suite and typecheck. These changes
touch only CI scripts, workflows, one helper, and tests. GitHub CI
covers full regression.

## Risks

- `pr-trusted.yml` loads from `master`, so the new policy takes effect
only after merge. This pull request's own CI uses the earlier rule.
- A Dependabot pull request that is behind `master` now fails if someone
merges `master` into it and the lockfile changes. Use `@dependabot
recreate`. The policy job error says this.
- Paid evaluation jobs refuse a merged Dependabot lockfile until the
refresh pull request merges. The refresh merges without review when it
adds no new package versions. If the release-age gate holds it, the wait
lasts until a maintainer merges it or the daily refresh runs. During
that time, the reporting-lock policy test also fails on other pull
requests, which is the current behavior for any unrenewed lock.
- The rule trusts GitHub as the signer of Dependabot commits and of
merge commits (`web-flow`). Superagent uses the same trust root.
- People with write access can still push to `chore/refresh-lockfile`.
This was already true. The bot merges only its own head commit
(`--match-head-commit`), and code-owner review covers anything else.
- A future manual proof renewal, such as #19283, now needs the refresh
workflow or an admin merge. A contributor pull request that changes the
proof files fails.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code
execution, in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused tests; see
Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:42:26 -08:00