From bef2aac9051ed99c23f62fcd47387591a3889d88 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Fri, 13 Nov 2026 08:20:29 -0800 Subject: [PATCH] chore(ci): remove the Docker dependency cache check of the Runner stages (#21331) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner is the part that executes agent work. The application image contains it > - Today the application image compiles the Runner in its own stages, with a cargo-chef dependency layer > - A pull-request job proves that a fresh builder reuses that layer. The job runs a script that appends to the `Dockerfile` and names those stages > - The next pull request removes those stages. The application image then takes its Runner from the `runner-artifacts` stages. The job cannot run against that `Dockerfile` > - This pull request removes the job and its script > - The benefit is that the next pull request changes only the image build, and its checks stay green ## Linked Issues or Issue Description Refs #21204, #21229, #21230, #21306 and #21307 (the Runner image and the preparation for this change). No issue exists for this change. **Problem or motivation** `.github/workflows/docker-runner-check.yml` has a pull-request job, `Compile isolated native Runner`. It runs `scripts/check-docker-runner-cache.sh`. The script adds stages to a copy of the `Dockerfile` that start from `runner-build` and copy from `runner-plan`. It reads the build log for the `runner-deps` step. The job runs on every pull request that changes the `Dockerfile`. The pull request that removes those stages would fail this job. **Proposed solution** Remove the job, its trigger and its script first, in a pull request of its own. **Alternatives considered** - Remove the job in the same pull request as the stages. That pull request is already large. - Rewrite the script for the `runner-artifacts` stages. Those stages have no dependency layer, so the property does not exist. `runner-image.yml` already builds them on a pull request. This is the first of three pull requests that move the application image onto the `runner-artifacts` stages. It has to merge first: the removed job runs on every `Dockerfile` change and builds stages that the second pull request deletes. The other two follow on top of it. ## What Changed - Delete `scripts/check-docker-runner-cache.sh`. - Remove the `runner` job and the `pull_request` trigger from `docker-runner-check.yml`. Only the manual jobs remain. The concurrency group no longer reads a pull request number. - Remove `Docker Runner check` from the workflow list of `pr-merge-review.yml`. That notifier acts only on pull-request runs, and the workflow has none now. - Remove the paragraph of `doc/DOCKER.md` that describes the check. ## Verification - `node --test .github/scripts/tests/notify-pr-merge-review.test.mjs`: 22 pass. - `vitest run server/src/__tests__/runner-image-workflow.test.ts`: pass (1 test is skipped on macOS, where bash is 3.2). - `vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/workflow-security.test.ts tests/runner-e2e/workflow-target.test.ts`: 28 pass on macOS. These files read `docker-runner-check.yml`. - The same commands in a `node:24.22.3-bookworm` container: all pass, except one test of `workflow-target.test.ts`. It fails with `jq: command not found`. It reads another workflow, and the container has no `jq`. - Not run: the full test suite, and the workflows themselves. Only CI can run the workflows. ## Risks - Until the next pull request merges, a pull request that changes only the native inputs of the Runner (`packages/paperclip-runner/runner/**`, `protocol/**`, `rust-toolchain.toml`) does not build the `runner-build` stage of the application image. `runner-image.yml` builds the daemon from the same files in the `runner-artifacts` stages on such a pull request. `docker-app-check.yml` builds `runner-build` when the `Dockerfile` or `.dockerignore` changes. - No check proves the cargo-chef layer reuse after this change. The next pull request removes that layer. - The job was not a required check. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), from Anthropic, through Claude Code. The model used tools: it edited files, ran the tests and ran the Docker builds that this description reports. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 --- .github/workflows/docker-runner-check.yml | 35 +------- .github/workflows/pr-merge-review.yml | 1 - doc/DOCKER.md | 15 ---- scripts/check-docker-runner-cache.sh | 99 ----------------------- 4 files changed, 1 insertion(+), 149 deletions(-) delete mode 100644 scripts/check-docker-runner-cache.sh diff --git a/.github/workflows/docker-runner-check.yml b/.github/workflows/docker-runner-check.yml index 6e72256daf3..eb5135eebea 100644 --- a/.github/workflows/docker-runner-check.yml +++ b/.github/workflows/docker-runner-check.yml @@ -35,48 +35,15 @@ on: description: "Build the canonical eval report viewer on hosted Linux" type: boolean default: false - pull_request: - paths: - - .github/workflows/docker-runner-check.yml - - Dockerfile - - .dockerignore - - scripts/check-docker-runner-cache.sh - - packages/paperclip-runner/rust-toolchain.toml - - packages/paperclip-runner/runner/** - - packages/paperclip-runner/protocol/** permissions: {} concurrency: # Publishing a newer image must not discard an earlier verification's evidence. - group: docker-runner-check-${{ github.event.pull_request.number || github.ref }}-${{ inputs.verify_source && 'verification' || inputs.verify_public_install && 'public-install' || inputs.build_eval_viewer && 'viewer' || 'image' }} + group: docker-runner-check-${{ github.ref }}-${{ inputs.verify_source && 'verification' || inputs.verify_public_install && 'public-install' || inputs.build_eval_viewer && 'viewer' || 'image' }} cancel-in-progress: true jobs: - runner: - if: github.event_name == 'pull_request' - name: Compile isolated native Runner - runs-on: ubuntu-latest - timeout-minutes: 20 - permissions: - contents: read - steps: - - name: Checkout repository - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - with: - persist-credentials: false - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4 - # Compile the real target, then change source in a disposable context. - # A fresh builder must import dependencies and produce changed binary metadata. - # The baseline build anonymously seeds from the public BuildKit cache at - # ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64}; the verification - # build imports only this run's locally exported cache on a fresh builder. - # No registry credentials or image publication. - - name: Verify native build and dependency cache reuse - run: bash scripts/check-docker-runner-cache.sh - - authorize_manual: if: github.event_name == 'workflow_dispatch' runs-on: ubuntu-latest diff --git a/.github/workflows/pr-merge-review.yml b/.github/workflows/pr-merge-review.yml index 193119c4581..888060e889e 100644 --- a/.github/workflows/pr-merge-review.yml +++ b/.github/workflows/pr-merge-review.yml @@ -6,7 +6,6 @@ on: - PR - Runner Windows - Sentry SDK contract - - Docker Runner check - Runner image - commitperclip PR Review - Storybook Visual diff --git a/doc/DOCKER.md b/doc/DOCKER.md index 86b7d32d261..7585abe97e2 100644 --- a/doc/DOCKER.md +++ b/doc/DOCKER.md @@ -424,18 +424,3 @@ input file times are normalized in both stages so fresh checkouts do not force Cargo to rebuild unchanged source. Changes made by build scripts still reach Cargo's normal validation. The final application copy excludes Cargo's target directory as before. Cache misses only cost compilation time. - -Pull requests that change the Dockerfile, Docker ignore rules, or Runner native -inputs also build the isolated `runner-build` target in `Docker Runner check`. -The check runs `bash scripts/check-docker-runner-cache.sh` against a disposable -copy of tracked source and the actual Docker ignore rules. It compiles a baseline -and exports a local cache, removes that builder, changes a Rust metadata constant, -and rebuilds on a fresh builder using only the exported cache. It requires a -cached dependency build, an unchanged dependency recipe, and changed metadata -from the real binary. It also verifies that a dependency declaration change -alters the recipe. The probe exports small metadata results instead of importing -a large test image into the Docker daemon. Temporary builders and cache files -are removed afterward. It catches missing embedded inputs before the post-merge -build. It uses a GitHub-hosted runner with read-only repository access and never -publishes images or registry caches. Allow up to 20 minutes for its cold build and -source rebuild. diff --git a/scripts/check-docker-runner-cache.sh b/scripts/check-docker-runner-cache.sh deleted file mode 100644 index a4b2e0c17a0..00000000000 --- a/scripts/check-docker-runner-cache.sh +++ /dev/null @@ -1,99 +0,0 @@ -#!/usr/bin/env bash -# Build the real Docker target on two fresh builders using an exported cache. -# Export only metadata, avoiding a multi-gigabyte test image in the daemon. -# External access: the baseline build anonymously reads the public BuildKit -# cache at ghcr.io/paperclipai/paperclip:buildcache-{amd64,arm64} (see -# RUNNER_CHECK_SEED_CACHE below). No credentials are used or required, and -# nothing is pushed. -set -euo pipefail -repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -probe_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-runner-cache.XXXXXX")" -baseline_builder="${probe_dir##*/}-baseline" -rebuild_builder="${probe_dir##*/}-rebuild" -cleanup() { - for builder in "$baseline_builder" "$rebuild_builder"; do - docker buildx rm "$builder" >/dev/null 2>&1 || true - done - rm -rf "$probe_dir" -} -trap cleanup EXIT -mkdir "$probe_dir/context" -cd "$repo_root" -git ls-files -z | tar -cf - --null -T - | tar -xf - -C "$probe_dir/context" -cd "$probe_dir/context" -export PROBE_DIR="$probe_dir" -cp Dockerfile "$probe_dir/cache-probe.Dockerfile" -cat >> "$probe_dir/cache-probe.Dockerfile" <<'DOCKER' -FROM runner-build AS cache-proof -RUN ./runner/target/release/paperclip-runnerd --build-metadata > /metadata.json -FROM scratch AS cache-proof-export -COPY --from=cache-proof /metadata.json /metadata.json -COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json -FROM scratch AS recipe-proof-export -COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json -DOCKER -build_proof() { - local result="$1" builder="$2" - shift 2 - docker buildx build --builder "$builder" --file "$probe_dir/cache-probe.Dockerfile" --target cache-proof-export --output "type=local,dest=$probe_dir/$result" --progress plain "$@" . 2>&1 | tee "$probe_dir/$result.log" -} -docker buildx create --name "$baseline_builder" --driver docker-container -# Seed only the baseline with the public BuildKit cache that docker.yml -# refreshes on every master push. The rust stages consume none of that -# build's args, so their layer keys match, and mode=max re-exports the -# imported layers into $probe_dir/cache — the verification build below -# still proves what it always proved from this run's exported cache -# alone, on a fresh builder. Anonymous pull only, nothing is pushed; a -# missing or unreachable ref is a BuildKit warning and the baseline -# degrades to the previous cold compile. Set RUNNER_CHECK_SEED_CACHE to -# another ref, or to the empty string to force the cold path. -if [[ -z "${RUNNER_CHECK_SEED_CACHE+x}" ]]; then - case "$(uname -m)" in - x86_64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-amd64" ;; - aarch64 | arm64) RUNNER_CHECK_SEED_CACHE="ghcr.io/paperclipai/paperclip:buildcache-arm64" ;; - *) RUNNER_CHECK_SEED_CACHE="" ;; - esac -fi -seed_args=() -if [[ -n "$RUNNER_CHECK_SEED_CACHE" ]]; then - seed_args=(--cache-from "type=registry,ref=${RUNNER_CHECK_SEED_CACHE}") -fi -build_proof baseline "$baseline_builder" ${seed_args[@]+"${seed_args[@]}"} --cache-to "type=local,dest=$probe_dir/cache,mode=max" -# Removing the first builder proves the second build cannot use daemon-local -# state, and releases its disk space before importing the exported cache. -docker buildx rm "$baseline_builder" -docker buildx create --name "$rebuild_builder" --driver docker-container -python3 - <<'CHECK' -from pathlib import Path -p=Path('packages/paperclip-runner/runner/crates/runner-core/src/bin/paperclip-runnerd.rs') -s=p.read_text(); needle='paperclip-runner/runnerd-build-metadata/v1' -assert s.count(needle)==1 -p.write_text(s.replace(needle,needle+'-cache-probe')) -CHECK -build_proof source-change "$rebuild_builder" --cache-from "type=local,src=$probe_dir/cache" -python3 - <<'CHECK' -import os,json,re -from pathlib import Path -root=Path(os.environ['PROBE_DIR']) -before=json.loads((root/'baseline/metadata.json').read_text()) -after=json.loads((root/'source-change/metadata.json').read_text()) -assert before['schema']=='paperclip-runner/runnerd-build-metadata/v1' -assert after['schema']==before['schema']+'-cache-probe' -assert (root/'baseline/recipe.json').read_bytes()==(root/'source-change/recipe.json').read_bytes() -log=(root/'source-change.log').read_text() -step=re.search(r'#(\d+) \[runner-deps[^\n]+ RUN cargo chef cook',log)[1] -assert f'#{step} CACHED' in log -assert 'Compiling paperclip-runner-core' in log -print('PASS: fresh builder imported compiled dependencies; real binary changed.') -p=Path('packages/paperclip-runner/runner/Cargo.toml') -s=p.read_text(); assert 'serde_json = "1.0"' in s -p.write_text(s.replace('serde_json = "1.0"','serde_json = ">=1.0.0, <2.0.0"')) -CHECK -docker buildx build --builder "$rebuild_builder" --file "$probe_dir/cache-probe.Dockerfile" --target recipe-proof-export --output "type=local,dest=$probe_dir/manifest-change" --progress plain . -python3 - <<'CHECK' -from pathlib import Path -import os -root=Path(os.environ['PROBE_DIR']) -assert (root/'source-change/recipe.json').read_bytes()!=(root/'manifest-change/recipe.json').read_bytes() -print('PASS: dependency declaration change invalidates the recipe.') -CHECK