feat(docker): build the image in the merge queue and promote it on master (#21096)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - `docker.yml` builds, probes, gates and signs the standard Core image
for every master push. Paperclip Cloud deploys a Core commit only after
that image is published.
> - Each master push rebuilds the image from scratch: about 10.6 min for
amd64 and 7.8 min for arm64.
> - The owner approved a merge queue for Core. A merge queue tests the
exact commit that master will become, before the merge.
> - This pull request builds and probes the image for that exact commit
in the queue, with no registry credentials. The later master push pushes
those exact bytes instead of rebuilding. When no matching verified build
exists, master builds from source exactly as today.
> - The benefit is that the image legs on master drop from about 8–11
min to about 3–4 min, and nothing changes until the owner turns the
queue on.

## Linked Issues or Issue Description

Refs #21013 (it measured the image path and proposed this plan).

**What existing behavior does this improve?**
Every master push rebuilds the standard image, even when the same commit
could already have been built while it waited to merge.

**Subsystem affected**
`.github/workflows/docker.yml` (`build-and-push` only), new
`.github/workflows/docker-merge-group.yml`, new
`scripts/merge-group-image.mjs`.

**Current behavior**
`build-and-push` always runs the full BuildKit build on master: 639 s
(amd64) and 468 s (arm64) in run 43133035209.

**Proposed behavior**
On a `merge_group` event, build both architectures for the merge-group
commit, probe PID 1, and upload an OCI archive plus a record of commit,
tree, architecture and digest. On the master push for the same commit,
each leg pushes that archive by digest instead of building. Everything
after the leg (probe, exact-source gate, signed-index check,
attestation) is unchanged.

**Reason and benefit**
The build moves before the merge, where it runs in parallel with the
queue's other checks. The master leg only downloads and pushes.

**Breaking changes**
None. Without `merge_group` events, `find` returns no run and the leg
builds as before.

**Additional context**
This PR does not touch the exact-source gate jobs
(`source_verification`, `publication_verification`, `merge-and-push`),
`scripts/watch-release-source.mjs` or `cloud-readiness.yml`. Another
change owns them.

## What Changed

- **`docker-merge-group.yml` (new, `on: merge_group` only).** It runs
only for this repository's master queue. Workflow permissions are `{}`
and the job has `contents: read` only. There are no secrets, no registry
login, no `cache-to`, no `push: true`, no environments, and checkout
uses `persist-credentials: false`. It runs on GitHub-hosted runners, not
the paid AWS fleets. Steps: build `target: production` with the same
build args as master (`PAPERCLIP_BUILD_COMMIT` is the merge-group SHA,
which becomes the master SHA). Read the public registry cache. Write
`type=oci` to a tar. `docker load` it and run
`scripts/assert-orphan-reaping.sh`. Only then write `record.json` and
upload `merge-group-image-<arch>` (3-day retention).
- **`scripts/merge-group-image.mjs` (new).**
- `find` (master push only): selects a completed, successful run of this
workflow, with event `merge_group`, the same `head_sha`, repository and
head repository both `1170821064`, and a `gh-readonly-queue/master/`
head branch. It needs exactly one unexpired artifact for the
architecture.
- `record` (merge group only): reads the archive's `index.json`, checks
each blob's sha256 and that the image is `linux/<arch>`, and writes
`{sha, tree, arch, digest}`.
- `verify` (master push only): re-reads the archive the same way and
requires record SHA = `GITHUB_SHA`, record tree = `git rev-parse
HEAD^{tree}` of the master checkout, same architecture, and archive
digest = recorded digest.
- **`docker.yml` `build-and-push`.**
  - The leg gets `actions: read` so it can download the artifact.
- New steps run after the registry login: find, download, then promote
(`verify`, then `skopeo copy --all --preserve-digests oci-archive:…
docker://ghcr.io/…@<digest>`). The step sets its `digest` output only
after the push succeeds.
- All three new steps are `continue-on-error`. Buildx, labels, metadata
and "Build and push by digest" run when `steps.promote.outputs.digest ==
''`, so any failure builds from source.
- "Select image digest" feeds the existing PID 1 probe and digest
export.
- **Tests (`scripts/merge-group-image.test.mjs`, 8 tests):**
- Run selection rejects another commit, another event, another workflow,
a fork head, another repository, another queue base, and failed,
cancelled or running builds.
  - Artifact selection.
- Archive digest with and without provenance, a tampered blob, the wrong
architecture, and a real tar read through the system `tar`.
  - Record checks for commit, tree, architecture and digest mismatch.
- The CLI refuses `record` outside `merge_group` and `find`/`verify`
outside a canonical master push.
- Static checks of the merge-group workflow: credentials, triggers,
permissions and runners.
- Static checks of the `docker.yml` wiring: promotion is gated and falls
back, the output is set only after the push, and every build step has
the fallback condition.

### How promotion proves it publishes the verified tree

1. `find` accepts only a run whose `head_sha` equals `github.sha` of the
push, meaning the queue landed the commit it tested (unverified: see
"Merge method" in Risks).
2. `verify` compares the record's tree with the tree of the master
checkout, and its commit with `GITHUB_SHA`.
3. The archive is content-addressed. The digest is recomputed from the
archive blobs and must equal the recorded digest. `skopeo
--preserve-digests` to that digest reference fails rather than produce
other bytes.
4. The leg then runs the PID 1 probe on that registry digest.
`merge-and-push` still signs only an index whose Linux images are the
probed digests (#21013), after the unchanged exact-source gate.
5. The pipeline itself is trusted only as master already ran it. `find`
and `verify` run the copy of `scripts/merge-group-image.mjs` from the
push's `before` commit (`git show "$BEFORE:…"`), not the copy the push
brings. `verify` refuses promotion if anything in `before..HEAD` (every
commit of a multi-PR group, not only the tip) changes
`docker-merge-group.yml`, `docker.yml`, the promotion script or the PID
1 probe. Such a push builds from source. So does the first push after
this PR merges, because its `before` has no verifier.

## Verification

- `node --test scripts/merge-group-image.test.mjs`: 10 pass. One builds
a real git history in which a two-PR push changes the pipeline in its
first commit only, and checks that the range check catches it.
- `node --test` on the workflow and contract suites (cloud-readiness,
docker-canary-promotion, post-merge-runner-routing,
lockfile-refresh-workflows, release-verify-workflow, reuse-canary-image,
standard-image-contract, cloud-source-verification, merge-group-image):
492 pass.
- `vitest run server/src/__tests__/docker-build-stamp.test.ts
server/src/__tests__/container-init-reaping.test.ts`: 22 pass.
- `actionlint` on both workflows: no new findings (the 6 remaining are
pre-existing `ls` and word-splitting notes in `docker.yml`).
- Local: `skopeo 1.24.1 copy --all --preserve-digests` between OCI
layouts kept the index digest. It accepted a `docker://…@sha256:`
destination and failed only at the network step.

Cannot be exercised before the queue exists, because no `merge_group`
event fires:
- the merge-group run itself: the hosted-runner build time, disk use
with tar plus `docker load`, `docker load` of a BuildKit OCI tar, and a
~5 GB artifact upload;
- the cross-run artifact download on master;
- whether `skopeo` exists on the runs-on AMI and accepts digest
destinations in its installed version.

Each master-side failure falls back to the full build. A merge-group
failure only blocks the queue if its check is required, which is why
that check is required last in the steps below.

## Risks

- **Inert until enabled.** No `merge_group` events fire before the owner
adds a merge queue. On master, `find` lists runs (one API call per leg,
about 1 s), finds none, and the leg builds as today. The only always-on
change is the leg's `actions: read` permission.
- **Build cache freshness.** A promoted master push writes no
`buildcache-<arch>`, and merge-group builds read but never write it.
With every push promoted, the cache ages, so dependency or weekly
tool-layer changes rebuild in each merge-group run until a fallback
build refreshes the cache. A follow-up can refresh the cache from master
without blocking publication.
- **Publication time is still gate-bound.** The exact-source gate (about
13–14 min after push) still decides when the image publishes. See
"Expected timing".
- **Merge method (unverified).** Promotion needs the master commit to be
exactly the commit the queue tested, because the image stamps the commit
SHA (`PAPERCLIP_BUILD_COMMIT`, `build-info.json`, the provider pack
revision). Matching only the tree would publish an image stamped with
another SHA, so this PR does not relax the check. I expect the queue to
fast-forward master to its tested commit. GitHub's merge-queue page does
not state this, and it cannot be tested before the queue exists. If the
queue lands a different SHA, promotion never matches and every push
builds from source. That is safe but inert. Step 4 below checks this.
- **skopeo on the AWS fleet.** The promote step checks for `skopeo`
first and logs "Merge-queue promotion unavailable" if it is missing,
then builds from source. If that notice appears, add skopeo to the fleet
image.
- **Hosted runner capacity.** amd64 merge-group builds run on 4-core
hosted runners, colder and slower than the AWS fleet. That adds queue
latency, not master latency.

### Settings the owner must enable, in order

1. **Prerequisite, a code change, not a setting.** The `master` ruleset
(id 13619726) requires `ci / verify` and `ci / e2e`. Those come from
`pr.yml`, which triggers only on `pull_request`. A merge queue waits for
required checks on the `merge_group` event too, so every entry would
stall until its timeout. First decide, and merge in a separate PR, how
`ci / verify` and `ci / e2e` report on `merge_group`: a full CI run on
the merge-group commit, or a fast pass that relies on the PR-head
result. Today the ruleset does not require up-to-date branches either.
2. Merge this PR. Nothing changes yet.
3. In Settings → Rules → Rulesets → `master` (13619726), add **Require
merge queue**:
   - merge method Squash;
- build concurrency 5, minimum group size 1, maximum group size 5, wait
0 min;
   - "Require all queue entries to pass required checks";
   - status check timeout 60 min.
Do not add the image build to the required checks yet. Promotion is then
opportunistic: if the merge-group build has not finished when master is
pushed, master builds from source.
4. After the first queued merge, confirm promotion can match. The master
commit SHA must equal the `head_sha` of its `Docker merge-queue build`
run, and the next master `Docker` run's "Promote the merge-queue image"
step must succeed. If the SHAs differ, promotion stays inert under this
merge method. Stop here and report it.
5. After about a week of green `Docker merge-queue build` runs, add
`build (linux/amd64, ubuntu-latest, amd64)` and `build (linux/arm64,
ubuntu-24.04-arm, arm64)` (app: GitHub Actions) to the ruleset's
required status checks. Then every queued commit has a verified image
ready to promote.
6. The existing bypass actors (org admins, repository admins,
integration 5534676) can still merge outside the queue. Those pushes
have no merge-group run and build from source.

### Rollback

- Remove the image checks from the required status checks (step 5), or
remove **Require merge queue** (step 3). Master pushes then find no
merge-group run and build from source. Nothing to migrate.
- Code: revert this PR. Old merge-group artifacts expire after 3 days.
Promoted images are ordinary digests under
`ghcr.io/paperclipai/paperclip` and stay valid.

### Expected timing

- Image leg on master: about 3–4 min (download about 5 GB, push the new
layers by digest, PID 1 probe) instead of 10.6 min (amd64) and 7.8 min
(arm64).
- Push to published: still bounded by the exact-source gate on master,
about 14–15 min (gate plus about 45 s for `merge-and-push`). Today it is
the same bound, but with the legs as a second 11-min path and the
serialized-run queue in front.
- If the gate later accepts proof produced in the merge queue (owned by
the gate change), push to published would be about 4–5 min (leg
promotion plus `merge-and-push`).

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, read-only GitHub API and registry reads).
Default reasoning mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-12 07:10:11 -06:00
1 parent 849f109f62
commit 1d153f62b6
5 files changed
+666 -2

No files matched your search

+169
View File
@@ -0,0 +1,169 @@
name: Docker merge-queue build
# Build-before-merge for the standard image. While a change waits in the merge
# queue, build the production image for the exact merge-group commit, probe it,
# and keep it as a run artifact. When the queue fast-forwards master to that
# commit, docker.yml pushes these exact bytes (scripts/merge-group-image.mjs)
# instead of rebuilding them. Without a matching successful run, docker.yml
# builds from source as before. Nothing here can publish: this workflow has no
# registry credentials, no secrets, and read-only contents access.
#
# Inert until the merge queue is enabled for master: no merge_group event fires
# before then.
on:
merge_group:
permissions: {}
# Each merge group has its own ref; a group that the queue rebuilt replaces the
# older run for that ref.
concurrency:
group: docker-merge-group-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
if: github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.event.merge_group.base_ref == 'refs/heads/master'
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
# GitHub-hosted runners: the protected release fleets admit only
# docker.yml on master.
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v7
with:
persist-credentials: false
# Same history as docker.yml, for `git describe` below.
fetch-depth: 0
filter: blob:none
# The merge queue fast-forwards master to this exact commit, so these
# stamps match what a master build of the same commit would carry.
- name: Compute build version
id: build-version
run: |
set -euo pipefail
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
echo "version=${version}" >> "$GITHUB_OUTPUT"
echo "Stamping build version: ${version:-<none>}"
- name: Compute tool cache epoch
id: tools-epoch
run: echo "epoch=$(date -u +%G-W%V)" >> "$GITHUB_OUTPUT"
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
- name: Refresh lockfile for Docker build context
run: |
set -euo pipefail
pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
changed="$(git status --porcelain)"
if [ -n "$changed" ] && printf '%s\n' "$changed" | grep -Fvq ' pnpm-lock.yaml'; then
echo "Unexpected files changed during lockfile refresh:"
echo "$changed"
exit 1
fi
- name: Free runner disk
run: |
set -euo pipefail
pnpm store prune || true
sudo apt-get clean || true
sudo rm -rf /usr/share/dotnet /usr/share/swift /usr/local/lib/android /usr/local/share/boost \
/usr/local/share/powershell /opt/ghc /opt/hostedtoolcache/CodeQL /opt/hostedtoolcache/PyPy /opt/hostedtoolcache/Ruby || true
docker system prune -af || true
df -h
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["mirror.gcr.io"]
- name: Compute schema migration labels
id: schema
run: |
set -euo pipefail
last=$(find packages/db/src/migrations -maxdepth 1 -name '*.sql' -printf '%f\n' | LC_ALL=C sort | tail -1)
count=$(find packages/db/src/migrations -maxdepth 1 -name '*.sql' | wc -l | tr -d ' ')
echo "last=${last}" >> "$GITHUB_OUTPUT"
echo "count=${count}" >> "$GITHUB_OUTPUT"
# Same labels as a master build. The version label names master, not the
# temporary merge-queue branch this commit is tested on.
- name: Docker meta
id: meta
uses: docker/metadata-action@v6
with:
images: ghcr.io/${{ github.repository }}
labels: |
org.opencontainers.image.version=master
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
# Reads the public registry cache anonymously and writes none of it.
- name: Build image archive
id: build
uses: docker/build-push-action@v7
with:
context: .
target: production
build-args: |
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
PAPERCLIP_BUILD_COMMIT=${{ github.sha }}
CLI_TOOLS_CACHE_EPOCH=${{ steps.tools-epoch.outputs.epoch }}
platforms: ${{ matrix.platform }}
push: false
outputs: type=oci,dest=${{ runner.temp }}/merge-group-image/image.tar,name=paperclip-merge-group:${{ matrix.arch }}
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-${{ matrix.arch }}
labels: ${{ steps.meta.outputs.labels }}
# PID 1 must be an init that reaps adopted orphans; see docker.yml.
- name: Verify PID 1 reaps orphaned processes
env:
ARCHIVE: ${{ runner.temp }}/merge-group-image/image.tar
PLATFORM: ${{ matrix.platform }}
IMAGE: paperclip-merge-group:${{ matrix.arch }}
run: |
set -euo pipefail
docker load --input "$ARCHIVE"
docker run --rm -i --pull never --platform "$PLATFORM" "$IMAGE" sh -s < scripts/assert-orphan-reaping.sh
docker image rm "$IMAGE"
# Written only after the probe passes. Promotion trusts nothing without it.
- name: Record the verified image
run: node scripts/merge-group-image.mjs record --arch "${{ matrix.arch }}" --archive "${{ runner.temp }}/merge-group-image/image.tar" --out "${{ runner.temp }}/merge-group-image/record.json"
- name: Upload the verified image
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: merge-group-image-${{ matrix.arch }}
path: |
${{ runner.temp }}/merge-group-image/image.tar
${{ runner.temp }}/merge-group-image/record.json
if-no-files-found: error
# Layers are already compressed.
compression-level: 0
retention-days: 3
+88 -2
View File
@@ -130,6 +130,7 @@ jobs:
permissions:
contents: read
packages: write
actions: read # download a verified merge-queue image artifact of this commit
steps:
- name: Checkout
uses: actions/checkout@v7
@@ -238,7 +239,77 @@ jobs:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
# Build-before-merge. When the merge queue already built and probed this
# exact commit (docker-merge-group.yml, with no registry credentials),
# push those bytes instead of rebuilding. scripts/merge-group-image.mjs
# requires a successful merge-group run of this repository for this SHA
# and checks the record's commit, tree, architecture and archive digest.
# Any missing or mismatched proof, or any failure here, leaves
# `steps.promote.outputs.digest` empty and the build below runs as
# before. The probe below and the exact-source gate apply either way.
- name: Find the merge-queue build of this commit
id: find
if: github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.event_name == 'push' && github.ref == 'refs/heads/master'
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
BEFORE: ${{ github.event.before }}
ARCH: ${{ matrix.arch }}
# Master's copy from before this push selects the run; see the promote
# step below.
run: |
set -euo pipefail
trusted="$RUNNER_TEMP/merge-group-image-trusted.mjs"
git show "$BEFORE:scripts/merge-group-image.mjs" > "$trusted"
node "$trusted" find --arch "$ARCH"
- name: Download the merge-queue image
if: steps.find.outputs.run_id != ''
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: merge-group-image-${{ matrix.arch }}
run-id: ${{ steps.find.outputs.run_id }}
github-token: ${{ github.token }}
path: ${{ runner.temp }}/merge-group-image
- name: Promote the merge-queue image
id: promote
if: steps.find.outputs.run_id != ''
continue-on-error: true
env:
ARCH: ${{ matrix.arch }}
DIRECTORY: ${{ runner.temp }}/merge-group-image
IMAGE_NAME: ghcr.io/${{ github.repository }}
BEFORE: ${{ github.event.before }}
run: |
set -euo pipefail
# Say plainly when this runner image cannot promote, instead of
# failing later inside the copy.
if ! command -v skopeo >/dev/null; then
echo "::notice title=Merge-queue promotion unavailable::skopeo is not installed on this runner; building from source."
exit 1
fi
echo "Promoting with $(skopeo --version)"
# Run the verifier as master had it before this push, not the copy
# this push brings, so a push cannot weaken the check it must pass.
# The verifier refuses any push that changes the pipeline since
# BEFORE. A push whose BEFORE has no verifier builds from source.
trusted="$RUNNER_TEMP/merge-group-image-trusted.mjs"
git show "$BEFORE:scripts/merge-group-image.mjs" > "$trusted"
# Verify without writing a step output: the digest is set only after
# the exact bytes are in the registry.
verified="$(GITHUB_OUTPUT='' node "$trusted" verify --arch "$ARCH" --dir "$DIRECTORY" --since "$BEFORE")"
digest="$(printf '%s\n' "$verified" | sed -n 's/^digest=//p')"
printf '%s' "$digest" | grep -Eq '^sha256:[a-f0-9]{64}$'
# --preserve-digests fails rather than re-encode, so the registry
# holds exactly the archived image under the recorded digest.
skopeo copy --all --preserve-digests "oci-archive:$DIRECTORY/image.tar" "docker://$IMAGE_NAME@$digest"
echo "Promoted the merge-queue image $IMAGE_NAME@$digest"
echo "digest=$digest" >> "$GITHUB_OUTPUT"
- name: Set up Docker Buildx
if: steps.promote.outputs.digest == ''
uses: docker/setup-buildx-action@v4
with:
buildkitd-config-inline: |
@@ -252,6 +323,7 @@ jobs:
# way to check image/migrator compatibility up front.
- name: Compute schema migration labels
id: schema
if: steps.promote.outputs.digest == ''
run: |
set -euo pipefail
last=$(ls packages/db/src/migrations/*.sql | sed 's|.*/||' | LC_ALL=C sort | tail -1)
@@ -267,6 +339,7 @@ jobs:
# so parallel canary builds cannot race it backwards.
- name: Docker meta
id: meta
if: steps.promote.outputs.digest == ''
uses: docker/metadata-action@v6
with:
images: ghcr.io/${{ github.repository }}
@@ -283,6 +356,7 @@ jobs:
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
- name: Build and push by digest
id: build
if: steps.promote.outputs.digest == ''
uses: docker/build-push-action@v7
with:
context: .
@@ -311,6 +385,18 @@ jobs:
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-${{ matrix.arch }},mode=max
labels: ${{ steps.meta.outputs.labels }}
# Exactly one of the two paths produced this leg's digest.
- name: Select image digest
id: image
env:
PROMOTED: ${{ steps.promote.outputs.digest }}
BUILT: ${{ steps.build.outputs.digest }}
run: |
set -euo pipefail
digest="${PROMOTED:-$BUILT}"
test -n "$digest"
echo "digest=$digest" >> "$GITHUB_OUTPUT"
# PID 1 must be an init that reaps adopted orphans. With node there, the
# orphans agent runs leave behind are never wait()ed and pin as zombies
# until the cgroup pid limit is exhausted and every fork() in the
@@ -325,7 +411,7 @@ jobs:
- name: Verify PID 1 reaps orphaned processes
env:
# Through the environment, not interpolated into the script body.
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
IMAGE_DIGEST: ${{ steps.image.outputs.digest }}
IMAGE_NAME: ghcr.io/${{ github.repository }}
PLATFORM: ${{ matrix.platform }}
run: |
@@ -341,7 +427,7 @@ jobs:
run: |
set -euo pipefail
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
digest="${{ steps.image.outputs.digest }}"
test -n "$digest"
touch "/tmp/digests/${digest#sha256:}"
+15
View File
@@ -333,6 +333,21 @@ suppresses push-triggered workflow runs for those pushes. The release jobs
therefore dispatch `docker.yml` explicitly at the new tag ref; the tag
mapping keys off `github.ref` either way.
When the merge queue is enabled for `master`,
[`docker-merge-group.yml`](../.github/workflows/docker-merge-group.yml) builds
and probes the production image for each merge-group commit. It has no registry
credentials and uploads an OCI archive with a record of the commit, tree,
architecture and digest. The `master` push for that same commit then pushes
those exact bytes by digest instead of rebuilding
(`scripts/merge-group-image.mjs`). A commit that changes that pipeline (the
merge-group workflow, `docker.yml`, the promotion script or the PID 1 probe)
anywhere in the pushed range is always built from source. `docker.yml` runs the
promotion script as master had it before the push, so promotion trusts only a
pipeline that master already ran.
Without a matching successful merge-group
run, or on any mismatch or error, the push builds from source. The PID 1 probe,
exact-source gate and signing apply on both paths.
Canary dispatches first look up the full-SHA master image. Reuse requires a
valid multi-platform index with both `linux/amd64` and `linux/arm64`, a matching
registry digest, and signed provenance from this repository's `docker.yml`
+184
View File
@@ -0,0 +1,184 @@
// Build-before-merge image handoff between the merge queue and master.
//
// A `merge_group` run of docker-merge-group.yml builds the production image for
// the exact merge-group commit with no registry credentials, probes it, and
// uploads an OCI archive plus a record of what it built. When the merge queue
// then fast-forwards master to that commit, docker.yml may push those exact
// bytes instead of rebuilding them. Anything that does not prove the match
// returns no promotion, and docker.yml falls back to its normal build.
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { createHash } from "node:crypto";
import { appendFile, readFile, writeFile } from "node:fs/promises";
import { pathToFileURL } from "node:url";
export const repository = "paperclipai/paperclip";
export const repositoryId = 1170821064;
export const workflowPath = ".github/workflows/docker-merge-group.yml";
export const recordSchema = "paperclip/merge-group-image/v1";
export const artifactName = arch => `merge-group-image-${arch}`;
const architectures = ["amd64", "arm64"];
const indexTypes = ["application/vnd.oci.image.index.v1+json", "application/vnd.docker.distribution.manifest.list.v2+json"];
const manifestTypes = ["application/vnd.oci.image.manifest.v1+json", "application/vnd.docker.distribution.manifest.v2+json"];
const fullSha = /^[a-f0-9]{40}$/;
const digestPattern = /^sha256:[a-f0-9]{64}$/;
const sha256 = bytes => `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
/**
* Choose the merge-queue build a master push may promote. Only a completed,
* successful run of this repository's merge-group workflow, on a merge-queue
* branch for master, for exactly this commit, qualifies.
*
* The commit must match, not only the tree: the image stamps the commit SHA
* (PAPERCLIP_BUILD_COMMIT, build-info.json, the provider pack revision). The
* merge queue lands the commit it tested for every merge method, squash
* included, by fast-forwarding master to it; a push that did not come through
* the queue has no matching run and builds from source.
*/
export function selectVerifiedRun({ sha, runs }) {
assert.match(sha ?? "", fullSha);
const eligible = runs.filter(run => run.head_sha === sha && run.event === "merge_group" && run.path === workflowPath &&
run.repository?.id === repositoryId && run.head_repository?.id === repositoryId &&
typeof run.head_branch === "string" && run.head_branch.startsWith("gh-readonly-queue/master/"));
if (eligible.length === 0) return { reason: "no merge-queue image build exists for this commit" };
const passed = eligible.filter(run => run.status === "completed" && run.conclusion === "success").sort((a, b) => b.id - a.id);
if (passed.length === 0) return { reason: "the merge-queue image build for this commit did not succeed" };
return { runId: passed[0].id };
}
/** The run's unexpired image artifact for one architecture, if there is exactly one. */
export function selectArtifact({ artifacts, arch }) {
assert.ok(architectures.includes(arch));
const matches = artifacts.filter(item => item.name === artifactName(arch) && !item.expired);
return matches.length === 1 ? { artifactId: matches[0].id } : { reason: `expected one unexpired ${artifactName(arch)} artifact, found ${matches.length}` };
}
/**
* The digest of the image an OCI archive holds, checked against the archive's
* own content-addressed blobs. `readEntry` returns the bytes of a tar entry.
*/
export function archiveImageDigest({ readEntry, arch }) {
assert.ok(architectures.includes(arch));
const layout = JSON.parse(readEntry("index.json").toString("utf8"));
assert.equal(layout.manifests?.length, 1, "The OCI archive must hold exactly one image");
const top = layout.manifests[0];
assert.match(top.digest ?? "", digestPattern);
const blob = name => {
const bytes = readEntry(`blobs/sha256/${name.slice("sha256:".length)}`);
assert.equal(sha256(bytes), name, `Archive blob ${name} does not match its digest`);
return JSON.parse(bytes.toString("utf8"));
};
const image = blob(top.digest);
const platformImages = indexTypes.includes(image.mediaType)
? (image.manifests ?? []).filter(item => item.platform?.os === "linux")
: [{ digest: top.digest, platform: undefined }];
assert.equal(platformImages.length, 1, "The OCI archive must hold exactly one Linux image");
const manifest = platformImages[0].digest === top.digest ? image : blob(platformImages[0].digest);
assert.ok(manifestTypes.includes(manifest.mediaType), "The OCI archive does not hold an image manifest");
const config = blob(manifest.config.digest);
assert.equal(config.os, "linux");
assert.equal(config.architecture, arch, `The OCI archive holds a ${config.architecture} image, not ${arch}`);
return top.digest;
}
/** What the merge-group build attests about the archive it uploads. */
export function createRecord({ sha, tree, arch, digest }) {
assert.match(sha ?? "", fullSha);
assert.match(tree ?? "", fullSha);
assert.ok(architectures.includes(arch));
assert.match(digest ?? "", digestPattern);
return { schema: recordSchema, repository, sha, tree, arch, digest };
}
/**
* Prove the archive is the build of the tree this master push publishes. The
* commit SHA covers its tree; the tree is compared too, so a record can never
* vouch for different content under a reused name.
*/
export function verifyRecord({ record, sha, tree, arch, archiveDigest }) {
assert.equal(record?.schema, recordSchema, "Unknown merge-group image record");
assert.equal(record.repository, repository);
assert.equal(record.sha, sha, "The merge-group build is for a different commit");
assert.equal(record.tree, tree, "The merge-group build is for a different tree");
assert.equal(record.arch, arch, "The merge-group build is for a different architecture");
assert.match(record.digest ?? "", digestPattern);
assert.equal(archiveDigest, record.digest, "The archive is not the image the merge-group build recorded");
return record.digest;
}
/**
* Files that produce, promote or vouch for a merge-group image. The merge-group
* run uses the queued commits' own copies, so a push that changes any of them
* is built from source on master instead: promotion then trusts only a
* pipeline that master already ran before this push. docker.yml runs this
* check from the copy of this script in the push's `before` commit, so a push
* cannot remove the check it is subject to.
*/
export const pipelineFiles = [workflowPath, ".github/workflows/docker.yml", "scripts/merge-group-image.mjs", "scripts/assert-orphan-reaping.sh"];
export function assertPipelineUnchanged(changedFiles) {
const changed = changedFiles.filter(file => pipelineFiles.includes(file));
assert.equal(changed.length, 0, `This push changes the merge-group image pipeline (${changed.join(", ")})`);
}
/** Pipeline files changed by everything this push landed: `since` (the push's `before`) to HEAD. */
export function changedSince(since, cwd = process.cwd()) {
assert.match(since ?? "", fullSha, "--since must be the push's full `before` SHA");
// Throws unless `since` is an ancestor: a push is a fast-forward of master.
execFileSync("git", ["merge-base", "--is-ancestor", since, "HEAD"], { cwd, stdio: "ignore" });
return execFileSync("git", ["diff", "--name-only", since, "HEAD", "--", ...pipelineFiles], { cwd, encoding: "utf8" }).split("\n").filter(Boolean);
}
const tarReader = archive => entry => execFileSync("tar", ["-xOf", archive, entry], { maxBuffer: 16 * 1024 * 1024 });
const gitTree = () => execFileSync("git", ["rev-parse", "HEAD^{tree}"], { encoding: "utf8" }).trim();
const option = name => {
const index = process.argv.indexOf(`--${name}`);
const value = index > 0 ? process.argv[index + 1] : undefined;
assert.ok(value, `--${name} is required`);
return value;
};
const output = async (key, value) => {
if (process.env.GITHUB_OUTPUT) await appendFile(process.env.GITHUB_OUTPUT, `${key}=${value}\n`);
console.log(`${key}=${value}`);
};
async function main(command) {
const sha = process.env.GITHUB_SHA;
const arch = option("arch");
if (command === "record") {
// Runs in the credential-free merge-group build.
assert.equal(process.env.GITHUB_EVENT_NAME, "merge_group", "Records are written only by merge-group builds");
const digest = archiveImageDigest({ readEntry: tarReader(option("archive")), arch });
await writeFile(option("out"), `${JSON.stringify(createRecord({ sha, tree: gitTree(), arch, digest }), null, 2)}\n`);
return output("digest", digest);
}
assert.ok(process.env.GITHUB_REPOSITORY === repository && process.env.GITHUB_REF === "refs/heads/master" &&
process.env.GITHUB_EVENT_NAME === "push", "Promotion runs only for canonical master pushes");
if (command === "find") {
const token = process.env.GITHUB_TOKEN;
assert.ok(token, "GITHUB_TOKEN is required");
const api = async path => {
const response = await fetch(`https://api.github.com/repos/${repository}${path}`, { redirect: "error",
headers: { Authorization: `Bearer ${token}`, Accept: "application/vnd.github+json" }, signal: AbortSignal.timeout(30_000) });
assert.ok(response.ok, `GitHub API ${path}: HTTP ${response.status}`);
return response.json();
};
const { workflow_runs: runs } = await api(`/actions/workflows/docker-merge-group.yml/runs?event=merge_group&head_sha=${sha}&per_page=20`);
const run = selectVerifiedRun({ sha, runs });
if (!run.runId) { console.log(`No promotion: ${run.reason}. Building from source.`); return; }
const { artifacts } = await api(`/actions/runs/${run.runId}/artifacts?per_page=100`);
const artifact = selectArtifact({ artifacts, arch });
if (!artifact.artifactId) { console.log(`No promotion: ${artifact.reason}. Building from source.`); return; }
return output("run_id", run.runId);
}
if (command === "verify") {
const directory = option("dir");
assertPipelineUnchanged(changedSince(option("since")));
const record = JSON.parse(await readFile(`${directory}/record.json`, "utf8"));
const archiveDigest = archiveImageDigest({ readEntry: tarReader(`${directory}/image.tar`), arch });
return output("digest", verifyRecord({ record, sha, tree: gitTree(), arch, archiveDigest }));
}
throw new Error(`Unknown command: ${command}`);
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) await main(process.argv[2]);
+210
View File
@@ -0,0 +1,210 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { createHash } from "node:crypto";
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import test from "node:test";
import { fileURLToPath } from "node:url";
import {
archiveImageDigest, assertPipelineUnchanged, changedSince, createRecord, repositoryId, selectArtifact, selectVerifiedRun, verifyRecord, workflowPath,
} from "./merge-group-image.mjs";
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const sha = "a".repeat(40), tree = "b".repeat(40);
const digestOf = bytes => `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
const run = overrides => ({ id: 10, head_sha: sha, event: "merge_group", path: workflowPath, status: "completed", conclusion: "success",
head_branch: `gh-readonly-queue/master/pr-1-${"c".repeat(40)}`, repository: { id: repositoryId }, head_repository: { id: repositoryId }, ...overrides });
test("promotes only a successful merge-queue build of this repository for this exact commit", () => {
assert.deepEqual(selectVerifiedRun({ sha, runs: [run()] }), { runId: 10 });
assert.deepEqual(selectVerifiedRun({ sha, runs: [run({ id: 10 }), run({ id: 12 }), run({ id: 14, conclusion: "failure" })] }), { runId: 12 });
for (const [label, candidate] of [
["another commit", run({ head_sha: "d".repeat(40) })],
["a pull request run", run({ event: "pull_request" })],
["a push run", run({ event: "push" })],
["another workflow", run({ path: ".github/workflows/docker.yml" })],
["a fork head repository", run({ head_repository: { id: 1 } })],
["another repository", run({ repository: { id: 1 } })],
["a queue for another branch", run({ head_branch: "gh-readonly-queue/release/pr-1-x" })],
["an ordinary branch", run({ head_branch: "master" })],
]) assert.match(selectVerifiedRun({ sha, runs: [candidate] }).reason, /no merge-queue image build/, label);
for (const candidate of [run({ conclusion: "failure" }), run({ conclusion: "cancelled" }), run({ status: "in_progress", conclusion: null })]) {
assert.match(selectVerifiedRun({ sha, runs: [candidate] }).reason, /did not succeed/);
}
assert.throws(() => selectVerifiedRun({ sha: "master", runs: [] }));
});
test("requires exactly one unexpired image artifact for the architecture", () => {
const item = (id, name, expired = false) => ({ id, name, expired });
assert.deepEqual(selectArtifact({ arch: "amd64", artifacts: [item(1, "merge-group-image-amd64"), item(2, "merge-group-image-arm64")] }), { artifactId: 1 });
assert.match(selectArtifact({ arch: "amd64", artifacts: [item(1, "merge-group-image-amd64", true)] }).reason, /found 0/);
assert.match(selectArtifact({ arch: "amd64", artifacts: [item(1, "merge-group-image-amd64"), item(2, "merge-group-image-amd64")] }).reason, /found 2/);
assert.throws(() => selectArtifact({ arch: "riscv64", artifacts: [] }));
});
// An OCI layout as BuildKit writes it for one platform with provenance: the
// layout index names a per-platform index of the image and its attestation.
function layout({ arch = "amd64", provenance = true, tamper } = {}) {
const blobs = new Map();
const put = value => { const bytes = Buffer.from(JSON.stringify(value)); const digest = digestOf(bytes); blobs.set(digest, bytes); return { digest, size: bytes.length }; };
const config = put({ architecture: arch, os: "linux", rootfs: { type: "layers", diff_ids: [] } });
const manifest = put({ schemaVersion: 2, mediaType: "application/vnd.oci.image.manifest.v1+json",
config: { mediaType: "application/vnd.oci.image.config.v1+json", ...config }, layers: [] });
const attestation = put({ schemaVersion: 2, mediaType: "application/vnd.oci.image.manifest.v1+json", config: { ...config }, layers: [] });
const top = provenance ? { mediaType: "application/vnd.oci.image.index.v1+json", ...put({ schemaVersion: 2, mediaType: "application/vnd.oci.image.index.v1+json", manifests: [
{ mediaType: "application/vnd.oci.image.manifest.v1+json", ...manifest, platform: { os: "linux", architecture: arch } },
{ mediaType: "application/vnd.oci.image.manifest.v1+json", ...attestation, platform: { os: "unknown", architecture: "unknown" } },
] }) } : { mediaType: "application/vnd.oci.image.manifest.v1+json", ...manifest };
if (tamper) blobs.set(manifest.digest, Buffer.from("{}"));
const entries = new Map([["index.json", Buffer.from(JSON.stringify({ schemaVersion: 2, manifests: [top] }))]]);
for (const [digest, bytes] of blobs) entries.set(`blobs/sha256/${digest.slice(7)}`, bytes);
return { entries, digest: top.digest };
}
const reader = entries => name => { assert.ok(entries.has(name), `missing ${name}`); return entries.get(name); };
test("reads the image digest from an OCI archive and checks its content-addressed blobs", () => {
for (const provenance of [true, false]) {
const { entries, digest } = layout({ provenance });
assert.equal(archiveImageDigest({ readEntry: reader(entries), arch: "amd64" }), digest);
}
assert.throws(() => archiveImageDigest({ readEntry: reader(layout().entries), arch: "arm64" }), /amd64 image, not arm64/);
assert.throws(() => archiveImageDigest({ readEntry: reader(layout({ tamper: true }).entries), arch: "amd64" }), /does not match its digest/);
const two = layout().entries;
two.set("index.json", Buffer.from(JSON.stringify({ manifests: [{ digest: `sha256:${"0".repeat(64)}` }, { digest: `sha256:${"1".repeat(64)}` }] })));
assert.throws(() => archiveImageDigest({ readEntry: reader(two), arch: "amd64" }), /exactly one image/);
});
test("reads a real tar archive through the system tar", () => {
const directory = mkdtempSync(path.join(tmpdir(), "merge-group-image-"));
try {
const { entries, digest } = layout({ arch: "arm64" });
for (const [name, bytes] of entries) { mkdirSync(path.dirname(path.join(directory, "layout", name)), { recursive: true }); writeFileSync(path.join(directory, "layout", name), bytes); }
execFileSync("tar", ["-cf", path.join(directory, "image.tar"), "-C", path.join(directory, "layout"), "index.json", "blobs"]);
const readEntry = name => execFileSync("tar", ["-xOf", path.join(directory, "image.tar"), name]);
assert.equal(archiveImageDigest({ readEntry, arch: "arm64" }), digest);
} finally { rmSync(directory, { recursive: true, force: true }); }
});
test("promotion proves commit, tree, architecture and archive digest", () => {
const digest = `sha256:${"e".repeat(64)}`;
const record = createRecord({ sha, tree, arch: "amd64", digest });
assert.equal(verifyRecord({ record, sha, tree, arch: "amd64", archiveDigest: digest }), digest);
for (const [change, pattern] of [
[{ sha: "f".repeat(40) }, /different commit/],
[{ tree: "f".repeat(40) }, /different tree/],
[{ arch: "arm64" }, /different architecture/],
[{ archiveDigest: `sha256:${"f".repeat(64)}` }, /not the image the merge-group build recorded/],
]) assert.throws(() => verifyRecord({ record, sha, tree, arch: "amd64", archiveDigest: digest, ...change }), pattern);
assert.throws(() => verifyRecord({ record: { ...record, schema: "other" }, sha, tree, arch: "amd64", archiveDigest: digest }), /Unknown/);
assert.throws(() => verifyRecord({ record: { ...record, repository: "fork/paperclip" }, sha, tree, arch: "amd64", archiveDigest: digest }));
assert.throws(() => createRecord({ sha, tree: "HEAD", arch: "amd64", digest }));
});
test("a push that changes the merge-group pipeline is built from source, not promoted", () => {
assert.doesNotThrow(() => assertPipelineUnchanged([]));
assert.doesNotThrow(() => assertPipelineUnchanged(["Dockerfile", "server/src/index.ts", "scripts/standard-image-contract.mjs"]));
for (const file of [".github/workflows/docker-merge-group.yml", ".github/workflows/docker.yml", "scripts/merge-group-image.mjs", "scripts/assert-orphan-reaping.sh"]) {
assert.throws(() => assertPipelineUnchanged(["server/src/index.ts", file]), /changes the merge-group image pipeline/, file);
}
});
test("the pipeline check covers every commit the push landed, not only the last", () => {
const directory = mkdtempSync(path.join(tmpdir(), "merge-group-push-"));
const git = (...args) => execFileSync("git", ["-c", "user.name=t", "-c", "user.email=t@example.invalid", "-c", "commit.gpgsign=false", ...args],
{ cwd: directory, encoding: "utf8" }).trim();
const commit = (file, text) => {
mkdirSync(path.dirname(path.join(directory, file)), { recursive: true });
writeFileSync(path.join(directory, file), text);
git("add", "-A"); git("commit", "-q", "-m", file);
return git("rev-parse", "HEAD");
};
try {
git("init", "-q", "-b", "main");
const before = commit("scripts/merge-group-image.mjs", "v1");
// A merge group of two PRs lands in one push: the first edits the
// pipeline, the second (the push's HEAD) only edits app code.
commit("scripts/merge-group-image.mjs", "v2");
commit("server/src/index.ts", "app");
assert.deepEqual(changedSince(git("rev-parse", "HEAD~1"), directory), []);
assert.deepEqual(changedSince(before, directory), ["scripts/merge-group-image.mjs"]);
assert.throws(() => assertPipelineUnchanged(changedSince(before, directory)), /changes the merge-group image pipeline/);
assert.throws(() => changedSince("HEAD~2", directory), /full `before` SHA/);
git("checkout", "-q", "--orphan", "other"); const unrelated = commit("README.md", "x");
git("checkout", "-q", "main");
assert.throws(() => changedSince(unrelated, directory), "a non-ancestor `before` is refused");
} finally { rmSync(directory, { recursive: true, force: true }); }
});
test("the CLI refuses to record outside a merge group or promote outside a canonical master push", () => {
const script = path.join(root, "scripts", "merge-group-image.mjs");
const run = (command, env) => {
try { execFileSync(process.execPath, [script, command, "--arch", "amd64", "--archive", "x", "--out", "y", "--dir", "z"],
{ env: { PATH: process.env.PATH, GITHUB_SHA: sha, ...env }, stdio: "pipe" }); return ""; }
catch (error) { return String(error.stderr); }
};
assert.match(run("record", { GITHUB_EVENT_NAME: "push" }), /only by merge-group builds/);
for (const env of [
{ GITHUB_EVENT_NAME: "merge_group", GITHUB_REPOSITORY: "paperclipai/paperclip", GITHUB_REF: "refs/heads/master" },
{ GITHUB_EVENT_NAME: "push", GITHUB_REPOSITORY: "fork/paperclip", GITHUB_REF: "refs/heads/master" },
{ GITHUB_EVENT_NAME: "push", GITHUB_REPOSITORY: "paperclipai/paperclip", GITHUB_REF: "refs/tags/v1" },
{ GITHUB_EVENT_NAME: "pull_request", GITHUB_REPOSITORY: "paperclipai/paperclip", GITHUB_REF: "refs/heads/master" },
]) for (const command of ["find", "verify"]) assert.match(run(command, env), /only for canonical master pushes/);
});
test("the merge-queue build has no publishing credentials or paid infrastructure", () => {
const workflow = readFileSync(path.join(root, ".github", "workflows", "docker-merge-group.yml"), "utf8");
const triggers = workflow.split("\non:\n")[1].split("\n\n")[0];
assert.equal(triggers.trim(), "merge_group:", "only the merge queue triggers this build");
assert.match(workflow, /^permissions: \{\}$/m);
const permissions = [...workflow.matchAll(/^ permissions:\n((?: {6}.+\n)+)/gm)].map(match => match[1].trim());
assert.ok(permissions.length > 0);
for (const block of permissions) assert.equal(block, "contents: read");
assert.doesNotMatch(workflow, /secrets\.|packages: write|id-token|attestations|docker\/login-action|cache-to:|push: true|environment:|runs-on\//);
assert.doesNotMatch(workflow, /pull_request_target|workflow_run|workflow_dispatch/);
assert.match(workflow, /persist-credentials: false/);
assert.match(workflow, /github\.event\.merge_group\.base_ref == 'refs\/heads\/master'/);
assert.match(workflow, /github\.repository_id == '1170821064'/);
assert.match(workflow, /push: false/);
assert.match(workflow, /scripts\/assert-orphan-reaping\.sh/);
assert.ok(workflow.indexOf("assert-orphan-reaping.sh") < workflow.indexOf("merge-group-image.mjs record"),
"the record is written only after the probe passes");
});
test("master promotes only after proof and otherwise builds exactly as before", () => {
const workflow = readFileSync(path.join(root, ".github", "workflows", "docker.yml"), "utf8");
const leg = workflow.split("\n build-and-push:\n")[1].split("\n publication_verification:\n")[0];
const steps = leg.split(/\n(?= - name: )/);
const step = name => { const found = steps.find(item => item.startsWith(` - name: ${name}\n`)); assert.ok(found, name); return found; };
const order = name => steps.indexOf(step(name));
assert.match(step("Find the merge-queue build of this commit"),
/if: github.repository == 'paperclipai\/paperclip' && github.repository_id == '1170821064' && github.event_name == 'push' && github.ref == 'refs\/heads\/master'/);
for (const name of ["Find the merge-queue build of this commit", "Download the merge-queue image", "Promote the merge-queue image"]) {
assert.match(step(name), /continue-on-error: true/, `${name} must fall back to the build`);
}
const promote = step("Promote the merge-queue image");
// Both helpers run master's copy from before the push, so a push cannot
// weaken the checks it must pass.
for (const name of ["Find the merge-queue build of this commit", "Promote the merge-queue image"]) {
assert.match(step(name), /BEFORE: \$\{\{ github.event.before \}\}/);
assert.match(step(name), /git show "\$BEFORE:scripts\/merge-group-image.mjs" > "\$trusted"/);
assert.doesNotMatch(step(name), /node scripts\/merge-group-image\.mjs/);
}
assert.match(promote, /node "\$trusted" verify --arch "\$ARCH" --dir "\$DIRECTORY" --since "\$BEFORE"/);
assert.ok(promote.indexOf('"$trusted" verify') < promote.indexOf("skopeo copy"));
assert.ok(promote.indexOf("skopeo copy --all --preserve-digests") < promote.indexOf('echo "digest=$digest" >> "$GITHUB_OUTPUT"'),
"the promoted digest becomes an output only after the exact bytes are pushed");
assert.match(promote, /GITHUB_OUTPUT='' node "\$trusted" verify/);
assert.ok(promote.indexOf("command -v skopeo") < promote.indexOf('"$trusted" verify'),
"a runner without skopeo reports that promotion is unavailable before it tries");
for (const name of ["Set up Docker Buildx", "Compute schema migration labels", "Docker meta", "Build and push by digest"]) {
assert.match(step(name), /if: steps.promote.outputs.digest == ''/, `${name} runs whenever nothing was promoted`);
assert.ok(order(name) > order("Promote the merge-queue image"));
}
assert.match(step("Select image digest"), /digest="\$\{PROMOTED:-\$BUILT\}"/);
assert.ok(order("Select image digest") < order("Verify PID 1 reaps orphaned processes"));
assert.match(step("Verify PID 1 reaps orphaned processes"), /IMAGE_DIGEST: \$\{\{ steps.image.outputs.digest \}\}/);
assert.match(step("Export digest"), /digest="\$\{\{ steps.image.outputs.digest \}\}"/);
assert.match(leg, /permissions:\n\s+contents: read\n\s+packages: write\n\s+actions: read/);
});