Files
PaperClipAI/server
Nicky LeachandClaude Fable 5.2 84a4d345b3 refactor(docker): take the application image's Runner from the runner-artifacts stage (#21332)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner is the part that executes agent work. `master` publishes
it alone as the image `ghcr.io/paperclipai/paperclip-runner`, built by
the `runner-artifacts` stages of the `Dockerfile`
> - The application image does not use those stages. It has five stages
of its own that compile the daemon, it builds the Runner again inside
the server build, and it builds a second provider pack
> - So one `Dockerfile` holds two recipes for the Runner, and the two
results differ: another base image and another daemon binary
> - This pull request removes the second recipe. The application image
copies the Runner out of the `runner-artifacts` stage, which the same
build runs once
> - The benefit is one Runner recipe, and an application image whose
Runner is the Runner the contract describes. A later pull request can
replace the stage with the published image

## Linked Issues or Issue Description

This is the second part of a chain. The first part is #21331, which
removes the cache check that cannot run on this `Dockerfile`. Refs
#21204, #21229, #21230 (the Runner image), #21306 (the no-rebuild mode
of the vendor step), #21307 (the pull-request check of the application
image) and #21312. No issue exists for this change.

**Problem or motivation**

The stages `rust-toolchain`, `rust-chef`, `runner-plan`, `runner-deps`
and `runner-build` compile the daemon on Debian trixie. The server build
then runs the full build of the Runner package. The
`runner-provider-pack` stage builds the provider pack. The
`runner-artifacts` stages do all of that again for the Runner image, on
Debian bookworm, with a stripped daemon.

**Proposed solution**

Read only the `runner-artifacts` stage from the application stages.
Build it inline in every build. Do not pass a published image yet.

**Alternatives considered**

- Pass the published Runner image to the build in this pull request.
That needs the resolver, labels and a rule for commits without an image.
It is the next step, and this one must be safe alone.
- Keep the old stages as a fallback. Two recipes is the problem.

## What Changed

- `Dockerfile`: remove the five stages. `build` starts from `base`. It
copies `/runner-package/dist` and the build stamp from
`runner-artifacts`, sets `PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1` and
runs the vendor check before the first application build.
- `Dockerfile`: `runner-provider-pack` starts from `base`. One `RUN`
with a bind mount of `runner-artifacts` copies the pack, the receipt and
the archives, and moves the large files to their own tree. `build` and
`runner-provider-pack` now come after the stages they read.
- `Dockerfile`: the three `COPY` instructions of `production` that take
the pack are byte-identical. A test pins all three.
- `Dockerfile`: `build` declares the build argument
`PAPERCLIP_RUNNER_EXPECT_INPUTS_DIGEST`, empty by default, and one step
runs the new `scripts/check-runner-image-expectation.mjs` on
`/runner-image.json` of `runner-artifacts`. **This pull request declares
the argument. No build sets it. The next pull request, which gives a
published Runner image to the build, is its first user.** With the
argument set, the build fails unless `runner-artifacts` is the published
image of exactly those inputs. `docker-app-check.yml` also runs when the
script changes.
- Comments: the `Dockerfile` has 220 comment lines, down from 247. The
comments of the two moved stages moved with them, unchanged, so the diff
shows them as added lines. Eight comments that named a removed stage or
a step that moved get the smallest correction. Two comments in
`prepare-runner-vendor.mjs` and `build-inputs.mjs` said that the
`Dockerfile` relies on the standalone rebuild; that clause is removed.
- Tests: replace the two cargo-chef tests of
`docker-build-stamp.test.ts` with three tests of the new shape. Update
the pack test, `docker-image-layer-reuse.test.ts`,
`vite-build-commit.test.ts`, the stage rules of
`runner-image-identity.test.ts`, and the include-layout test of
`runner-release-crate-cache.test.mjs`, which now reads
`runner-artifacts-runnerd`.
- Docs: `doc/DOCKER.md`, `doc/DEVELOPING.md`, and a new section "The
application image" in `doc/runner-image-contract.md`.

## Verification

All builds: `linux/arm64`, local, `--build-arg
PAPERCLIP_BUILD_COMMIT=<sha>` with one commit for every build. The
builds ran before the rebase onto `master`, on the head of #21310 as it
was before it merged. `master` changed neither the `Dockerfile` nor an
input of the Runner in between.

- `docker buildx build --target production`, cold, before and after: 506
s and 410 s. Compressed size: 2,971 MB and 2,900 MB.
- The image config and the text of all 33 history rows are identical.
The row `COPY /provider-pack-transport/ /opt/paperclip-runner/ #
buildkit` matches the pattern of `retained-provider-pack-oci.ts` exactly
once.
- A comparison of every file of the two images (path, type, size, mode,
link target, and SHA-256 under `/app` and `/opt`). The differences are
in "Risks". All other files are byte-identical, the root `node_modules`
included.
- From the image after: the four `/app/packages/paperclip-runner/...`
paths and the contents of `/opt/paperclip-runner/` exist.
`paperclip-runnerd --build-metadata` runs. `provider-pack.json` has
`runnerSourceRevision` equal to the build commit and no
`runnerInputsDigest`. The entry points `index.js`, `live/index.js` and
`testing.js` of the Runner import.
- The vendor step logged `reusing the current Runner build` in both of
its runs. Every `Compiling` line of the log is in
`runner-artifacts-runnerd`.
- `--target build`, `--target runner-provider-pack` and `--target cloud`
built. The `build` stage has no `gcc`, `cc`, `cargo`, `rustc` or
`pkg-config`.
- `--target runner-artifacts` and `runner-artifacts-slim`, before and
after: `runner-image-layout.mjs check` and `runner-image-oci.mjs
check-layers` pass on both, in `node:24.22.3-bookworm`. The daemon,
`provider-pack.json` and the two stable layers have the same digests
before and after.
- Without a build commit: `build` and `runner-provider-pack` built, and
the pack was skipped with the same message. This run was on the next
branch of this chain, after the rebase. The last steps of `production`
were not run for this case.
- The expectation argument, on this `Dockerfile`, `--target build
--output type=cacheonly`, after the rebase:
- No argument, inline Runner: the build passes (233 s). The step passes.
- Argument, inline Runner: the build fails at the step. The message says
that the `runner-artifacts` of the build are not a published Runner
image.
- Argument `d4df566f…` and the published image of those inputs as
`--build-context
runner-artifacts=docker-image://ghcr.io/paperclipai/paperclip-runner@sha256:0cd8250b…`:
the step passes, and the build runs no Runner stage. The vendor step
then fails with `inputs changed since the recorded build`, because this
tree differs from the tree of that image by one comment line in
`build-inputs.mjs`. With that line put back, the same build completes,
and the vendor step reuses the published build.
- Another digest with the same context: the build fails at the step. The
message names the inputs the image has.
- A tree whose lockfile is behind the manifests of the Runner fails at
the frozen install of the Runner stages, after 169 s, with the message
that stage already had.
- `docker buildx build --check`: no warning, before and after.
`scripts/check-docker-deps-stage.mjs` and
`scripts/check-node-version-policy.mjs` pass.
- Tests, on macOS and in `node:24.22.3-bookworm`, after the rebase: the
changed server test files with the other tests that read the
`Dockerfile` or the image workflows (231 pass in 10 files on Linux),
`vite-build-commit.test.ts` (10 pass),
`runner-release-crate-cache.test.mjs` (11 pass),
`docker-deps-postinstall.test.mjs`. The changed server test files
typecheck.
- Not run: the full test suite, any `linux/amd64` build, `podman`, the
workflows and the release pipeline. Only CI can prove amd64,
`docker.yml`, `docker-merge-group.yml`, `release.yml`,
`docker-app-check.yml` and `runner-image.yml`.

## Risks

- **The digest of the Runner image changes.** Its identity covers the
whole `Dockerfile`. The next `master` push publishes a new
`inputs-<digest>`. The contents do not change: see the comparison above.
- **A build that is given a Runner of other inputs fails; it does not
ship that Runner.** With the argument, it fails at the expectation step.
Without the argument, it fails at the vendor step when the build inputs
of the Runner differ. One comment line in `build-inputs.mjs` changes
those build inputs in this pull request, so the image published before
this pull request cannot be vendored into a tree after it.
- **What a consumer of the application image sees** (measured on
`linux/arm64`):
- The daemon is the stripped binary compiled on Debian bookworm: 23.2
MB, was 36.4 MB, at
`/app/packages/paperclip-runner/dist/bin/paperclip-runnerd` and at the
vendored copy in `server/dist`. Its hash changes. It has no symbols for
a crash backtrace.
- `/app/packages/paperclip-eval-kernel/dist` is absent. Only
`dist/eval/workflow-harness.js` of the Runner imports that package. Only
`eval/index.js` exports the harness again, and no module imports that
file. The server runtime bundle does not name the package.
- The provider pack: `provider-pack.json` is byte-identical. Two
provider executables and one licence file are hard links to their other
copy in the pack, where they were separate files. One empty directory is
absent. The pack is 195 MB smaller, and the layer of its large files
gets a new digest once.
- The transport archives have other bytes, because the Runner build
gives large files a fixed timestamp. The receipt names the same files
with the same digests.
- Paths, modes, owners, the image config and the layer order do not
change.
- **What a contributor and a maintainer of the image build see:**
- **The image build does not run the drift checks of the Runner.**
Before, the server build ran the full `build` script of the Runner, so a
failed drift check failed the image build on `master`. Now the Runner
stages run only the output steps of that script. Repository CI runs the
checks on every pull request. `.github/docker-context-checks.Dockerfile`
and three exceptions in `.dockerignore` exist for those checks in the
image build; a separate follow-up removes them, and this chain does not
edit them.
- **The dependency-caching layer of the daemon is gone.** Before,
cargo-chef compiled the Rust dependencies in a layer of their own, and a
change to the Rust source compiled only the Runner's own crates. Now
such a change compiles the dependencies again (146 s here for the whole
daemon). A change to the server, the UI or the Runner's TypeScript still
reuses the compiled daemon.
- **A lockfile behind the manifests of the Runner fails the build.**
Before, `docker build` resolved it inside the image. The image workflows
refresh the lockfile first. A local build prints the command to run. The
next pull request makes this failure early and clear.
- **The build needs more disk**: about 25 GB of builder storage here,
was about 20 GB, with the image export.
- Rollback: revert this pull request. Images that exist do not change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), from Anthropic, through Claude
Code. The model used tools: it edited files, ran the tests and ran the
Docker builds that this description reports.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 09:48:49 -08:00
..