chore(ci): remove the Docker dependency cache check of the Runner stages (#21331)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner is the part that executes agent work. The application
image contains it
> - Today the application image compiles the Runner in its own stages,
with a cargo-chef dependency layer
> - A pull-request job proves that a fresh builder reuses that layer.
The job runs a script that appends to the `Dockerfile` and names those
stages
> - The next pull request removes those stages. The application image
then takes its Runner from the `runner-artifacts` stages. The job cannot
run against that `Dockerfile`
> - This pull request removes the job and its script
> - The benefit is that the next pull request changes only the image
build, and its checks stay green

## Linked Issues or Issue Description

Refs #21204, #21229, #21230, #21306 and #21307 (the Runner image and the
preparation for this change). No issue exists for this change.

**Problem or motivation**

`.github/workflows/docker-runner-check.yml` has a pull-request job,
`Compile isolated native Runner`. It runs
`scripts/check-docker-runner-cache.sh`. The script adds stages to a copy
of the `Dockerfile` that start from `runner-build` and copy from
`runner-plan`. It reads the build log for the `runner-deps` step. The
job runs on every pull request that changes the `Dockerfile`. The pull
request that removes those stages would fail this job.

**Proposed solution**

Remove the job, its trigger and its script first, in a pull request of
its own.

**Alternatives considered**

- Remove the job in the same pull request as the stages. That pull
request is already large.
- Rewrite the script for the `runner-artifacts` stages. Those stages
have no dependency layer, so the property does not exist.
`runner-image.yml` already builds them on a pull request.

This is the first of three pull requests that move the application image
onto the `runner-artifacts` stages. It has to merge first: the removed
job runs on every `Dockerfile` change and builds stages that the second
pull request deletes. The other two follow on top of it.

## What Changed

- Delete `scripts/check-docker-runner-cache.sh`.
- Remove the `runner` job and the `pull_request` trigger from
`docker-runner-check.yml`. Only the manual jobs remain. The concurrency
group no longer reads a pull request number.
- Remove `Docker Runner check` from the workflow list of
`pr-merge-review.yml`. That notifier acts only on pull-request runs, and
the workflow has none now.
- Remove the paragraph of `doc/DOCKER.md` that describes the check.

## Verification

- `node --test .github/scripts/tests/notify-pr-merge-review.test.mjs`:
22 pass.
- `vitest run server/src/__tests__/runner-image-workflow.test.ts`: pass
(1 test is skipped on macOS, where bash is 3.2).
- `vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/workflow-security.test.ts
tests/runner-e2e/workflow-target.test.ts`: 28 pass on macOS. These files
read `docker-runner-check.yml`.
- The same commands in a `node:24.22.3-bookworm` container: all pass,
except one test of `workflow-target.test.ts`. It fails with `jq: command
not found`. It reads another workflow, and the container has no `jq`.
- Not run: the full test suite, and the workflows themselves. Only CI
can run the workflows.

## Risks

- Until the next pull request merges, a pull request that changes only
the native inputs of the Runner (`packages/paperclip-runner/runner/**`,
`protocol/**`, `rust-toolchain.toml`) does not build the `runner-build`
stage of the application image. `runner-image.yml` builds the daemon
from the same files in the `runner-artifacts` stages on such a pull
request. `docker-app-check.yml` builds `runner-build` when the
`Dockerfile` or `.dockerignore` changes.
- No check proves the cargo-chef layer reuse after this change. The next
pull request removes that layer.
- The job was not a required check.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), from Anthropic, through Claude
Code. The model used tools: it edited files, ran the tests and ran the
Docker builds that this description reports.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Fable 5.2 authored and GitHub committed 2026-11-13 08:20:29 -08:00
1 parent b0e99dfcfa
commit bef2aac905
4 files changed
+1 -149

No files matched your search

+1 -34
View File
@@ -35,48 +35,15 @@ on:
description: "Build the canonical eval report viewer on hosted Linux"
type: boolean
default: false
pull_request:
paths:
- .github/workflows/docker-runner-check.yml
- Dockerfile
- .dockerignore
- scripts/check-docker-runner-cache.sh
- packages/paperclip-runner/rust-toolchain.toml
- packages/paperclip-runner/runner/**
- packages/paperclip-runner/protocol/**
permissions: {}
concurrency:
# Publishing a newer image must not discard an earlier verification's evidence.
group: docker-runner-check-${{ github.event.pull_request.number || github.ref }}-${{ inputs.verify_source && 'verification' || inputs.verify_public_install && 'public-install' || inputs.build_eval_viewer && 'viewer' || 'image' }}
group: docker-runner-check-${{ github.ref }}-${{ inputs.verify_source && 'verification' || inputs.verify_public_install && 'public-install' || inputs.build_eval_viewer && 'viewer' || 'image' }}
cancel-in-progress: true
jobs:
runner:
if: github.event_name == 'pull_request'
name: Compile isolated native Runner
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Compile the real target, then change source in a disposable context.
# A fresh builder must import dependencies and produce changed binary metadata.
# The baseline build anonymously seeds from the public BuildKit cache at
# ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64}; the verification
# build imports only this run's locally exported cache on a fresh builder.
# No registry credentials or image publication.
- name: Verify native build and dependency cache reuse
run: bash scripts/check-docker-runner-cache.sh
authorize_manual:
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
-1
View File
@@ -6,7 +6,6 @@ on:
- PR
- Runner Windows
- Sentry SDK contract
- Docker Runner check
- Runner image
- commitperclip PR Review
- Storybook Visual
-15
View File
@@ -424,18 +424,3 @@ input file times are normalized in both stages so fresh checkouts do not force
Cargo to rebuild unchanged source. Changes made by build scripts still reach
Cargo's normal validation. The final application copy excludes Cargo's target
directory as before. Cache misses only cost compilation time.
Pull requests that change the Dockerfile, Docker ignore rules, or Runner native
inputs also build the isolated `runner-build` target in `Docker Runner check`.
The check runs `bash scripts/check-docker-runner-cache.sh` against a disposable
copy of tracked source and the actual Docker ignore rules. It compiles a baseline
and exports a local cache, removes that builder, changes a Rust metadata constant,
and rebuilds on a fresh builder using only the exported cache. It requires a
cached dependency build, an unchanged dependency recipe, and changed metadata
from the real binary. It also verifies that a dependency declaration change
alters the recipe. The probe exports small metadata results instead of importing
a large test image into the Docker daemon. Temporary builders and cache files
are removed afterward. It catches missing embedded inputs before the post-merge
build. It uses a GitHub-hosted runner with read-only repository access and never
publishes images or registry caches. Allow up to 20 minutes for its cold build and
source rebuild.
-99
View File
@@ -1,99 +0,0 @@
#!/usr/bin/env bash
# Build the real Docker target on two fresh builders using an exported cache.
# Export only metadata, avoiding a multi-gigabyte test image in the daemon.
# External access: the baseline build anonymously reads the public BuildKit
# cache at ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64} (see
# RUNNER_CHECK_SEED_CACHE below). No credentials are used or required, and
# nothing is pushed.
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
probe_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-runner-cache.XXXXXX")"
baseline_builder="${probe_dir##*/}-baseline"
rebuild_builder="${probe_dir##*/}-rebuild"
cleanup() {
for builder in "$baseline_builder" "$rebuild_builder"; do
docker buildx rm "$builder" >/dev/null 2>&1 || true
done
rm -rf "$probe_dir"
}
trap cleanup EXIT
mkdir "$probe_dir/context"
cd "$repo_root"
git ls-files -z | tar -cf - --null -T - | tar -xf - -C "$probe_dir/context"
cd "$probe_dir/context"
export PROBE_DIR="$probe_dir"
cp Dockerfile "$probe_dir/cache-probe.Dockerfile"
cat >> "$probe_dir/cache-probe.Dockerfile" <<'DOCKER'
FROM runner-build AS cache-proof
RUN ./runner/target/release/paperclip-runnerd --build-metadata > /metadata.json
FROM scratch AS cache-proof-export
COPY --from=cache-proof /metadata.json /metadata.json
COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json
FROM scratch AS recipe-proof-export
COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json
DOCKER
build_proof() {
local result="$1" builder="$2"
shift 2
docker buildx build --builder "$builder" --file "$probe_dir/cache-probe.Dockerfile" --target cache-proof-export --output "type=local,dest=$probe_dir/$result" --progress plain "$@" . 2>&1 | tee "$probe_dir/$result.log"
}
docker buildx create --name "$baseline_builder" --driver docker-container
# Seed only the baseline with the public BuildKit cache that docker.yml
# refreshes on every master push. The rust stages consume none of that
# build's args, so their layer keys match, and mode=max re-exports the
# imported layers into $probe_dir/cache — the verification build below
# still proves what it always proved from this run's exported cache
# alone, on a fresh builder. Anonymous pull only, nothing is pushed; a
# missing or unreachable ref is a BuildKit warning and the baseline
# degrades to the previous cold compile. Set RUNNER_CHECK_SEED_CACHE to
# another ref, or to the empty string to force the cold path.
if [[ -z "${RUNNER_CHECK_SEED_CACHE+x}" ]]; then
case "$(uname -m)" in
x86_64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-amd64" ;;
aarch64 | arm64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-arm64" ;;
*) RUNNER_CHECK_SEED_CACHE="" ;;
esac
fi
seed_args=()
if [[ -n "$RUNNER_CHECK_SEED_CACHE" ]]; then
seed_args=(--cache-from "type=registry,ref=${RUNNER_CHECK_SEED_CACHE}")
fi
build_proof baseline "$baseline_builder" ${seed_args[@]+"${seed_args[@]}"} --cache-to "type=local,dest=$probe_dir/cache,mode=max"
# Removing the first builder proves the second build cannot use daemon-local
# state, and releases its disk space before importing the exported cache.
docker buildx rm "$baseline_builder"
docker buildx create --name "$rebuild_builder" --driver docker-container
python3 - <<'CHECK'
from pathlib import Path
p=Path('packages/paperclip-runner/runner/crates/runner-core/src/bin/paperclip-runnerd.rs')
s=p.read_text(); needle='paperclip-runner/runnerd-build-metadata/v1'
assert s.count(needle)==1
p.write_text(s.replace(needle,needle+'-cache-probe'))
CHECK
build_proof source-change "$rebuild_builder" --cache-from "type=local,src=$probe_dir/cache"
python3 - <<'CHECK'
import os,json,re
from pathlib import Path
root=Path(os.environ['PROBE_DIR'])
before=json.loads((root/'baseline/metadata.json').read_text())
after=json.loads((root/'source-change/metadata.json').read_text())
assert before['schema']=='paperclip-runner/runnerd-build-metadata/v1'
assert after['schema']==before['schema']+'-cache-probe'
assert (root/'baseline/recipe.json').read_bytes()==(root/'source-change/recipe.json').read_bytes()
log=(root/'source-change.log').read_text()
step=re.search(r'#(\d+) \[runner-deps[^\n]+ RUN cargo chef cook',log)[1]
assert f'#{step} CACHED' in log
assert 'Compiling paperclip-runner-core' in log
print('PASS: fresh builder imported compiled dependencies; real binary changed.')
p=Path('packages/paperclip-runner/runner/Cargo.toml')
s=p.read_text(); assert 'serde_json = "1.0"' in s
p.write_text(s.replace('serde_json = "1.0"','serde_json = ">=1.0.0, <2.0.0"'))
CHECK
docker buildx build --builder "$rebuild_builder" --file "$probe_dir/cache-probe.Dockerfile" --target recipe-proof-export --output "type=local,dest=$probe_dir/manifest-change" --progress plain .
python3 - <<'CHECK'
from pathlib import Path
import os
root=Path(os.environ['PROBE_DIR'])
assert (root/'source-change/recipe.json').read_bytes()!=(root/'manifest-change/recipe.json').read_bytes()
print('PASS: dependency declaration change invalidates the recipe.')
CHECK