From e363b0e52d0c736b35d48262ba37352110bd1bba Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Thu, 12 Nov 2026 22:55:23 -0800 Subject: [PATCH] fix(docker): keep local build output out of the Runner image's build context (#21312) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner is the part that executes agent work. `master` publishes it alone as the image `ghcr.io/paperclipai/paperclip-runner`, one time for each set of Runner inputs (#21204, #21229, #21230) > - The name of a published image is a digest of every file the Runner build receives from the build context > - `.dockerignore` keeps the Runner's own build output (`packages/paperclip-runner/dist`) out of the context, but not the eval kernel's (`packages/paperclip-eval-kernel/dist`), which the Runner's build also writes > - A tree where the Runner was built therefore has another inputs digest than a clean checkout of the same commit > - The Runner package's Vite builds write five more directories with the same problem > - This pull request adds the missing ignore rules, and a test that finds the next such directory > - The benefit is that a developer's tree and a clean checkout of the same commit name the same Runner image ## Linked Issues or Issue Description Refs #21204, which added the image identity. No issue exists for this change. **Problem or motivation** The inputs digest covers every file the build receives under `packages/paperclip-eval-kernel`. The Runner's build compiles the eval kernel and writes `packages/paperclip-eval-kernel/dist`. After a local Runner build, that directory is in the build context, so the digest of the tree differs from the digest of a clean checkout. A lookup of the published Runner image for such a tree finds nothing. CI checkouts are clean, so no published image is affected. **Proposed solution** Add `packages/paperclip-eval-kernel/dist` and `packages/paperclip-runner/dist-*` to `.dockerignore`, next to the rule for the Runner's own `dist`. Add a test that derives every build-output directory inside the image's inputs and requires a rule for each. **Alternatives considered** - Exclude the directory in the identity script only. Then the digest would not describe what the build receives, which is the property the identity is built on. **Roadmap alignment** This is build infrastructure. It adds no product feature and does not overlap an item in `ROADMAP.md`. This pull request stands alone. It is based on `master` and needs no other open pull request. ## What Changed - `.dockerignore`: two rules. - `packages/paperclip-eval-kernel/dist`: the Runner's own build writes it. - `packages/paperclip-runner/dist-*`: the output of the Runner package's Vite builds (`dist-browser`, `dist-issue-thread`, `dist-scenarios`, `dist-sdk`, `dist-standalone`). The Runner's `build` script does not write them, but a tree where one of those builds ran had the same digest drift. No tracked path matches the rule, and no later `!` rule re-includes those directories. - `server/src/__tests__/runner-image-identity.test.ts`: one new test. For each package among the identity's inputs it collects the `outDir` of every tsconfig a script compiles with `tsc -p`, the `target` directory beside every Cargo manifest a script names, and the `outDir` of every Vite configuration. It requires that the last `.dockerignore` rule matching each directory is an exclusion. Today that is eight directories. A new output directory inside the inputs fails the test until it has a rule. - No image build reads that directory from the host. The Runner stages, the `build` stage of the application image and `docker/daytona-runner` all compile the kernel themselves (`tsc -p`, not incremental), and the eval kernel is a development dependency of the Runner only. ## Verification The inputs digest, as the `runner-artifacts-identity` build target reports it, on a `linux/arm64` Docker host: | Tree | Without the rule | With the rule | | --- | --- | --- | | clean export at `62d673cc3f` | `0e6662ad…` | `0e6662ad…` | | the same, with an eval-kernel `dist` | `e4cf3095…` | `0e6662ad…` | | clean export at `bc71c23e0a` | `88fe2929…` | `88fe2929…` | | the same, with an eval-kernel `dist` | `ac331275…` | `88fe2929…` | | clean export at `731b52453c` | `88fe2929…` | `88fe2929…` | | the same, with two files in `dist-browser` | `44fe867d…` | `88fe2929…` | | the same, with one file in each of the five Vite directories | not measured | `88fe2929…` | - `0e6662ad…` is the digest of the image that `master` published at `25e9a9c2e9`, so the rule does not change the digest of a clean checkout. - A worktree after a real `pnpm --filter @paperclipai/paperclip-eval-kernel build` also gave the clean digest with the rule. - `runner-image-identity.test.ts` (42 tests) and `tests/runner-e2e/daytona-image.test.ts` (19 tests), which reads `.dockerignore`, pass on macOS and in `node:24.22.3-bookworm`. - The new test fails on the `.dockerignore` of `master` and names `packages/paperclip-eval-kernel/dist`. It fails without the `dist-*` rule and names `packages/paperclip-runner/dist-browser`. It fails without either of the two older Runner rules, and with a trailing `!` rule that re-includes one of the directories. Never run: - Any image build with the new rules beyond the identity target, including `.github/docker-context-checks.Dockerfile`. On a CI checkout the directory does not exist, so the rules remove nothing there. - `linux/amd64`. - The full test suite and the full typecheck. CI must supply them. ## Risks - **The sandbox image's content id changes one time.** `.dockerignore` is an input of that id (`tests/runner-e2e/daytona-image-content.ts`). At `62d673cc3f` the id goes from `be208823…` to `1e13a151…`. The next end-to-end run builds a new sandbox image, and the restart-chaos harness needs an image built from this tree. The image's contents do not change. - Low otherwise: the rules remove directories that no image build reads. The `Dockerfile`, `docker/daytona-runner/Dockerfile` and `.github/docker-context-checks.Dockerfile` name none of them and run no Vite build. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude Code with tool use (shell, file edits, local Docker builds). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 --- .dockerignore | 2 ++ .../__tests__/runner-image-identity.test.ts | 29 ++++++++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/.dockerignore b/.dockerignore index 12b8054137b..f8c9dc4a4b4 100644 --- a/.dockerignore +++ b/.dockerignore @@ -9,6 +9,8 @@ data tmp *.log packages/paperclip-runner/dist +packages/paperclip-runner/dist-* +packages/paperclip-eval-kernel/dist packages/paperclip-runner/runner/target packages/paperclip-runner/devtools packages/paperclip-runner/docs diff --git a/server/src/__tests__/runner-image-identity.test.ts b/server/src/__tests__/runner-image-identity.test.ts index af441f6b70a..2048d2d87f8 100644 --- a/server/src/__tests__/runner-image-identity.test.ts +++ b/server/src/__tests__/runner-image-identity.test.ts @@ -1,4 +1,4 @@ -import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import path from "node:path"; import { fileURLToPath } from "node:url"; @@ -181,6 +181,33 @@ describe("Runner image identity", () => { rmSync(path.join(root, "packages/paperclip-runner/styles.css")); expect(digestOf(root)).not.toBe(before); }); + + it("keeps out of the build context every directory a build script or a Vite configuration of a package among the identity's inputs writes its output into, so a tree with a local build and a clean checkout of one commit give the build the same files", () => { + const rules = readFileSync(path.join(repoRoot, ".dockerignore"), "utf8").split(/\r?\n/).filter((rule) => rule && !rule.startsWith("#")); + const lastRuleFor = (target: string) => rules.filter((rule) => target.split("/").some((_, index, segments) => + path.matchesGlob(segments.slice(0, index + 1).join("/"), rule.replace(/^!/, "").replace(/^\//, "")))).at(-1); + const outputs = RUNNER_IMAGE_CONTEXT_INPUTS.filter((input) => existsSync(path.join(repoRoot, input, "package.json"))).flatMap((input) => { + const manifest = JSON.parse(readFileSync(path.join(repoRoot, input, "package.json"), "utf8")); + const scripts = Object.values(manifest.scripts as Record).join("\n"); + const emitted = [...scripts.matchAll(/\btsc -p (\S+)(?!\S| --noEmit)/g)].map((match) => + JSON.parse(readFileSync(path.join(repoRoot, input, match[1]), "utf8")).compilerOptions.outDir as string); + const compiled = [...scripts.matchAll(/--manifest-path (\S+)\/Cargo\.toml/g)].map((match) => `${match[1]}/target`); + const bundled = readdirSync(path.join(repoRoot, input)).filter((name) => /^vite\..*config\.[cm]?[jt]s$/.test(name)).flatMap((name) => + [...readFileSync(path.join(repoRoot, input, name), "utf8").matchAll(/\boutDir:\s*(?:resolve\(packageRoot,\s*)?"([^"]+)"/g)].map((match) => match[1])); + return [...emitted, ...compiled, ...bundled].map((output) => path.posix.join(input, output)); + }); + expect(outputs).toEqual(expect.arrayContaining([ + "packages/paperclip-eval-kernel/dist", + "packages/paperclip-runner/dist", + "packages/paperclip-runner/dist-browser", + "packages/paperclip-runner/dist-issue-thread", + "packages/paperclip-runner/dist-scenarios", + "packages/paperclip-runner/dist-sdk", + "packages/paperclip-runner/dist-standalone", + "packages/paperclip-runner/runner/target", + ])); + for (const output of new Set(outputs)) expect(lastRuleFor(output)?.startsWith("!"), output).toBe(false); + }); }); });