Files
PaperClipAI/server/src/__tests__/docker-build-stamp.test.ts
T
DottaandPaperclip 6395cae072 fix(runner): ship provider pack in the standard Docker image (#14854)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote OpenCode and ACPX runs need a provider pack from the
application build.
> - Cloud now builds its application image from the standard production
image.
> - The provider pack was added only to the legacy cloud image target.
> - The standard image therefore cannot supply the pack to downstream
Cloud images.
> - This pull request adds the pack to the production image and lets the
cloud target inherit it.
> - Remote runs can then use the pack that matches the application
source commit.

## Linked Issues or Issue Description

Refs #13827. Refs #14024.

The standard production image does not include the remote provider pack.
Downstream Cloud images inherit that omission. Remote OpenCode and ACPX
runs fail with `runner_remote_provider_artifact_incompatible` and ask
for `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`.

## What Changed

- Build and copy the provider pack into the standard production image.
- Set the pack path and check that an unprivileged user can read its
artifacts and execute Node.
- Let the legacy cloud target inherit the pack from production.
- Add regression checks for production packaging and cloud inheritance.
- Document stamped image behavior and the default pack path.

## Verification

- The 12 focused Docker stamp and provider-pack reuse tests pass on
commit `4a11f8d52aead55f85527c8e82c6d7f2644ce0da`.
- The new packaging regression failed against the old Dockerfile and
passed with the fix.
- On the current commit, `pnpm build` and `pnpm -r typecheck` pass. All
seven standard-image contract tests also pass.
- The current-head CI build, typecheck, test, browser, and native Runner
checks passed. The local full suite hit one chat-channel assertion
failure; that exact test passed in isolation. The remaining local run
was stopped after CI completed to avoid duplicating its full suite. An
earlier run on the pre-rebase base had a heartbeat comment batching
timeout; the external chat wait integration suite passed all 142 tests
in isolation.
- [The stamped preview image build
passed](https://github.com/paperclipai/paperclip/actions/runs/36885002850/job/110446106393),
including the production-stage provider pack build, copy, and
unprivileged artifact readability/executable check. Publication,
compatibility validation, and deployment of this exact commit to a
staging QA instance passed.
- Reproduced the exact missing-pack error on an existing staging image
with Paperclip Runner, ACPX, and Claude in a remote Daytona computer.
The legacy Claude adapter succeeds with the same account and computer.
After deploying this commit, the same native task succeeded: it computed
`5050` with a real remote shell command, wrote a proof file, read it
back in a separate call, uploaded the file as a deliverable, and
completed the task. The uploaded file contents and Done state persisted
after a page reload. The run trace confirms Paperclip Runner, ACPX, and
Claude. The first run took 2m 59s, including approximately 97s of remote
artifact preparation. A second native run read the unchanged file from
the prior run and completed successfully. Its startup took about 120s;
this verifies repeated execution and file persistence, not fast
provider-pack reuse.

## Risks

- Stamped standard images now include the provider pack and its build
cost. A pack build failure now fails the production image build.
- Unstamped local builds still skip pack generation. Setting the path
alone does not create a pack.
- No database, provider authentication, or runner verification rules
change.

## Model Used

OpenAI Codex, GPT-6. The exact serving model identifier and context
window are not exposed in this session. Capabilities used: repository
inspection, code editing, shell verification, and browser testing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-01 11:07:50 -05:00

138 lines
7.5 KiB
TypeScript

import { readFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
/**
* Drift guard for the Docker build-stamp wiring.
*
* The server build runs scripts/write-build-stamp.mjs, which stamps the built
* commit into dist/build-info.json. The build context has no .git, so the
* script reads PAPERCLIP_BUILD_COMMIT instead. Docker exposes an ARG to the
* next RUN as an environment variable, but an ARG goes out of scope at the end
* of its stage. So the build stage must declare `ARG PAPERCLIP_BUILD_COMMIT`
* before the server build; the production ARG alone stamps nothing, because
* the server build already ran in the earlier stage.
*
* This guard fails if a refactor drops the build-stage ARG, moves it after the
* server build, or removes the build-arg the docker workflow passes.
*/
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
const dockerfile = readFileSync(path.join(repoRoot, "Dockerfile"), "utf8");
const workflow = readFileSync(path.join(repoRoot, ".github", "workflows", "docker.yml"), "utf8");
const previewWorkflow = readFileSync(path.join(repoRoot, ".github", "workflows", "release.yml"), "utf8");
/**
* Return the text of the Dockerfile stage that starts at the named target.
* A stage runs from its `FROM ... AS <name>` line to the next `FROM` line.
*/
function stageBody(source: string, stageName: string): string {
const froms = [...source.matchAll(/^FROM .*$/gm)];
const startIdx = froms.findIndex((m) => new RegExp(`\\bAS ${stageName}\\s*$`).test(m[0]));
expect(startIdx, `Dockerfile must declare a '${stageName}' stage`).toBeGreaterThanOrEqual(0);
const start = froms[startIdx].index ?? 0;
const end = froms[startIdx + 1]?.index ?? source.length;
return source.slice(start, end);
}
it("keeps per-build runtime metadata out of the weekly CLI-install cache", () => {
const production = stageBody(dockerfile, "production");
const tools = production.search(/^RUN echo "cli-tools-epoch:/m);
const entrypoint = production.search(/^RUN chmod \+x \/usr\/local\/bin\/docker-entrypoint\.sh/m);
const runtime = production.search(/^ENV NODE_ENV=production/m);
const epoch = production.search(/^ARG CLI_TOOLS_CACHE_EPOCH\b/m);
expect(tools).toBeGreaterThanOrEqual(0);
expect(entrypoint).toBeGreaterThan(tools);
expect(epoch).toBeGreaterThanOrEqual(0);
expect(epoch).toBeLessThan(tools);
for (const name of ["PAPERCLIP_BUILD_VERSION", "PAPERCLIP_BUILD_COMMIT"]) {
const declarations = [...production.matchAll(new RegExp(`^ARG ${name}\\b`, "gm"))];
expect(declarations).toHaveLength(1);
expect(declarations[0].index).toBeGreaterThan(entrypoint);
expect(declarations[0].index).toBeLessThan(runtime);
expect(production.slice(runtime)).toContain(`${name}=\${${name}}`);
}
});
describe("docker build-stamp wiring", () => {
it("declares PAPERCLIP_BUILD_COMMIT in the build stage before the server build", () => {
const build = stageBody(dockerfile, "build");
const argIdx = build.search(/^ARG PAPERCLIP_BUILD_COMMIT\b/m);
const serverBuildIdx = build.search(/^RUN pnpm --filter @paperclipai\/server build\b/m);
expect(argIdx, "build stage must declare ARG PAPERCLIP_BUILD_COMMIT").toBeGreaterThanOrEqual(0);
expect(serverBuildIdx, "build stage must run the server build").toBeGreaterThanOrEqual(0);
expect(
argIdx,
"ARG PAPERCLIP_BUILD_COMMIT must precede the server build so the stamp script reads it",
).toBeLessThan(serverBuildIdx);
});
it("passes PAPERCLIP_BUILD_COMMIT as a build-arg for standard and explicit preview builds", () => {
for (const [name, source] of [["standard", workflow], ["preview", previewWorkflow]]) {
expect(source, `${name} must pass the source commit into the image build`)
.toMatch(/^\s*PAPERCLIP_BUILD_COMMIT=\$\{\{ (?:github.sha|inputs.source_ref) \}\}$/m);
}
});
});
describe("Docker Rust dependency cache", () => {
it("caches the locked dependency recipe separately from source and per-build metadata", () => {
const chef = stageBody(dockerfile, "rust-chef");
const planner = stageBody(dockerfile, "runner-plan");
const dependencies = stageBody(dockerfile, "runner-deps");
expect(chef).toContain("FROM rust-toolchain AS rust-chef");
expect(chef).toMatch(/cargo install cargo-chef --version \d+\.\d+\.\d+ --locked/);
expect(planner).toContain("COPY packages/paperclip-runner/runner ./runner");
expect(planner).toContain("cargo chef prepare --recipe-path /tmp/runner-recipe.json");
expect(dependencies).toContain("FROM rust-chef AS runner-deps");
expect(dependencies).toContain("COPY --from=runner-plan /tmp/runner-recipe.json /tmp/runner-recipe.json");
expect(dependencies).toContain("cargo chef cook --release --locked --package paperclip-runner-core --bin paperclip-runnerd");
expect(dependencies).not.toMatch(/COPY .*\.\/runner|COPY .*\.\/protocol|COPY \. \.|PAPERCLIP_BUILD_COMMIT/);
});
it("rebuilds real workspace code and embedded protocol inputs after cooking dependencies", () => {
const native = stageBody(dockerfile, "runner-build");
expect(native).toContain("FROM runner-deps AS runner-build");
for (const source of ["runner", "protocol"]) {
expect(native.indexOf(`COPY packages/paperclip-runner/${source} ./${source}`))
.toBeLessThan(native.indexOf("cargo build --release"));
expect(native).toContain(`COPY packages/paperclip-runner/${source} ./${source}`);
}
expect(native).toContain("cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd");
expect(stageBody(dockerfile, "build")).toContain("FROM runner-build AS build");
});
});
describe("Standard image remote provider pack", () => {
it("ships the build-owned pack in the production base used by downstream images", () => {
const pack = stageBody(dockerfile, "runner-provider-pack");
const production = stageBody(dockerfile, "production");
expect(pack).toContain("FROM build AS runner-provider-pack");
expect(pack).toContain('PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}"');
expect(pack).toContain("build-provider-pack.mjs /provider-pack");
expect(pack).toContain('if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then');
expect(pack).toContain("mkdir -p /provider-pack");
expect(pack).toContain("Skipping remote provider pack");
expect(production).toContain("--from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack");
expect(production).not.toContain("--chown=node:node --from=runner-provider-pack");
expect(production).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack");
expect(production).toContain("gosu 65534:65534 node");
expect(production).toContain("Object.values(manifest.payload.artifacts)");
expect(production).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack");
expect(workflow).toMatch(/^\s*target: production$/m);
expect(dockerfile).not.toContain("provision-grok.mjs");
});
it("lets explicit cloud preview builds inherit the same pack and configuration", () => {
const cloud = stageBody(dockerfile, "cloud");
expect(cloud).toContain("FROM production AS cloud");
expect(cloud).not.toContain("/opt/paperclip-runner/provider-pack");
expect(cloud).not.toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH");
expect(dockerfile.match(/build-provider-pack\.mjs \/provider-pack/g)).toHaveLength(1);
expect(previewWorkflow).toMatch(/^\s*target: cloud$/m);
});
});