From 6395cae0726ad0345d5040bd0969c7a006c262fd Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 1 Oct 2026 11:07:50 -0500 Subject: [PATCH] fix(runner): ship provider pack in the standard Docker image (#14854) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote OpenCode and ACPX runs need a provider pack from the application build. > - Cloud now builds its application image from the standard production image. > - The provider pack was added only to the legacy cloud image target. > - The standard image therefore cannot supply the pack to downstream Cloud images. > - This pull request adds the pack to the production image and lets the cloud target inherit it. > - Remote runs can then use the pack that matches the application source commit. ## Linked Issues or Issue Description Refs #13827. Refs #14024. The standard production image does not include the remote provider pack. Downstream Cloud images inherit that omission. Remote OpenCode and ACPX runs fail with `runner_remote_provider_artifact_incompatible` and ask for `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`. ## What Changed - Build and copy the provider pack into the standard production image. - Set the pack path and check that an unprivileged user can read its artifacts and execute Node. - Let the legacy cloud target inherit the pack from production. - Add regression checks for production packaging and cloud inheritance. - Document stamped image behavior and the default pack path. ## Verification - The 12 focused Docker stamp and provider-pack reuse tests pass on commit `4a11f8d52aead55f85527c8e82c6d7f2644ce0da`. - The new packaging regression failed against the old Dockerfile and passed with the fix. - On the current commit, `pnpm build` and `pnpm -r typecheck` pass. All seven standard-image contract tests also pass. - The current-head CI build, typecheck, test, browser, and native Runner checks passed. The local full suite hit one chat-channel assertion failure; that exact test passed in isolation. The remaining local run was stopped after CI completed to avoid duplicating its full suite. An earlier run on the pre-rebase base had a heartbeat comment batching timeout; the external chat wait integration suite passed all 142 tests in isolation. - [The stamped preview image build passed](https://github.com/paperclipai/paperclip/actions/runs/36885002850/job/110446106393), including the production-stage provider pack build, copy, and unprivileged artifact readability/executable check. Publication, compatibility validation, and deployment of this exact commit to a staging QA instance passed. - Reproduced the exact missing-pack error on an existing staging image with Paperclip Runner, ACPX, and Claude in a remote Daytona computer. The legacy Claude adapter succeeds with the same account and computer. After deploying this commit, the same native task succeeded: it computed `5050` with a real remote shell command, wrote a proof file, read it back in a separate call, uploaded the file as a deliverable, and completed the task. The uploaded file contents and Done state persisted after a page reload. The run trace confirms Paperclip Runner, ACPX, and Claude. The first run took 2m 59s, including approximately 97s of remote artifact preparation. A second native run read the unchanged file from the prior run and completed successfully. Its startup took about 120s; this verifies repeated execution and file persistence, not fast provider-pack reuse. ## Risks - Stamped standard images now include the provider pack and its build cost. A pack build failure now fails the production image build. - Unstamped local builds still skip pack generation. Setting the path alone does not create a pack. - No database, provider authentication, or runner verification rules change. ## Model Used OpenAI Codex, GPT-6. The exact serving model identifier and context window are not exposed in this session. Capabilities used: repository inspection, code editing, shell verification, and browser testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip --- Dockerfile | 52 ++++++++++--------- doc/DOCKER.md | 13 +++++ docs/deploy/environment-variables.md | 2 +- .../src/__tests__/docker-build-stamp.test.ts | 31 +++++++---- 4 files changed, 63 insertions(+), 35 deletions(-) diff --git a/Dockerfile b/Dockerfile index eddc097b0ae..1db10902f67 100644 --- a/Dockerfile +++ b/Dockerfile @@ -144,6 +144,21 @@ RUN pnpm --filter @paperclipai/server build RUN test -f server/dist/index.js || (echo "ERROR: server build output missing" && exit 1) RUN rm -rf packages/paperclip-runner/runner/target +# Remote OpenCode and ACPX runs require a controller-owned provider pack to +# verify the sandbox installation or stage matching assets. Ship it in the +# standard image so downstream Cloud compositions inherit the same artifacts. +# Grok's native executable stays an external sandbox prerequisite; this pack +# contains only its launcher. +FROM build AS runner-provider-pack +# Unstamped local builds remain usable, but cannot qualify a remote pack. +# Never invent a source revision to make an unqualified pack look verified. +RUN mkdir -p /provider-pack \ + && if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then \ + PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}" node packages/paperclip-runner/scripts/build-provider-pack.mjs /provider-pack; \ + else \ + echo "Skipping remote provider pack: supply a full PAPERCLIP_BUILD_COMMIT to enable remote OpenCode/ACPX execution"; \ + fi + FROM base AS production ARG USER_UID=1000 ARG USER_GID=1000 @@ -169,6 +184,17 @@ RUN chmod +x /usr/local/bin/docker-entrypoint.sh COPY --chown=node:node --from=build /app /app +COPY --from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack +# Managed deployments can remap node's UID at startup. This immutable pack +# contains public code and integrity metadata, never credentials; it must remain +# readable afterward. +# Keep it root-owned and verify access as an unrelated unprivileged UID. +RUN chmod -R a+rX /opt/paperclip-runner/provider-pack \ + && if [ -f /opt/paperclip-runner/provider-pack/provider-pack.json ]; then \ + gosu 65534:65534 node -e 'const fs = require("node:fs"); const path = require("node:path"); const root = "/opt/paperclip-runner/provider-pack"; const manifest = JSON.parse(fs.readFileSync(path.join(root, "provider-pack.json"), "utf8")); for (const artifact of Object.values(manifest.payload.artifacts)) fs.readFileSync(path.join(root, artifact.path)); fs.accessSync(path.join(root, manifest.payload.artifacts.nodeCommand.path), fs.constants.X_OK);'; \ + fi +ENV PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack + # Declare per-build metadata after the stable RUN layers. Docker includes # in-scope ARG values in a RUN's environment even when its command does not # mention them; declaring these earlier invalidates the weekly tool cache. @@ -213,8 +239,8 @@ CMD ["node", "--import", "./server/node_modules/tsx/dist/loader.mjs", "server/di # dist/ to exist in the image — the default image ships only their source, # so auto-install logs "bundle not present" and skips. The plugins are # built in this separate target so the default (self-hosted) image stays -# lean; CI pins the default build to `--target production`, which is -# byte-identical to before this stage existed. +# lean; CI pins the default build to `--target production`. Both targets +# inherit the build-owned remote provider pack from the standard image. # # The sandbox providers are intentionally excluded from the pnpm workspace # (see pnpm-workspace.yaml), so each installs standalone exactly as its @@ -293,29 +319,7 @@ RUN set -eu; \ test -n "$specifiers" || { echo "ERROR: CLOUD_BUNDLED_SERVER_DEPS names no package" >&2; exit 1; }; \ pnpm add --ignore-workspace --no-lockfile $specifiers -# ACPX remote runs require a controller-owned provider pack to verify the -# sandbox installation or stage matching assets. Grok's native executable stays -# an external sandbox prerequisite; this pack contains only its launcher. -FROM build AS cloud-provider-pack -# Unstamped local builds remain usable, but cannot qualify a remote pack. -# Never invent a source revision to make an unqualified pack look verified. -RUN mkdir -p /provider-pack \ - && if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then \ - PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}" node packages/paperclip-runner/scripts/build-provider-pack.mjs /provider-pack; \ - else \ - echo "Skipping remote provider pack: supply a full PAPERCLIP_BUILD_COMMIT to enable remote ACPX execution"; \ - fi - FROM production AS cloud -COPY --from=cloud-provider-pack /provider-pack /opt/paperclip-runner/provider-pack -# Cloud remaps node's UID at startup. This immutable pack contains public code -# and integrity metadata, never credentials; it must remain readable afterward. -# Keep it root-owned and verify access as an unrelated unprivileged UID. -RUN chmod -R a+rX /opt/paperclip-runner/provider-pack \ - && if [ -f /opt/paperclip-runner/provider-pack/provider-pack.json ]; then \ - gosu 65534:65534 node -e 'const fs = require("node:fs"); const path = require("node:path"); const root = "/opt/paperclip-runner/provider-pack"; const manifest = JSON.parse(fs.readFileSync(path.join(root, "provider-pack.json"), "utf8")); for (const artifact of Object.values(manifest.payload.artifacts)) fs.readFileSync(path.join(root, artifact.path)); fs.accessSync(path.join(root, manifest.payload.artifacts.nodeCommand.path), fs.constants.X_OK);'; \ - fi -ENV PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack COPY --chown=node:node --from=cloud-plugins /app/packages/plugins/sandbox-providers /app/packages/plugins/sandbox-providers # Land the isolated install inside the server's own `node_modules`, the # directory Node's module resolution walks up to from `/app/server` for diff --git a/doc/DOCKER.md b/doc/DOCKER.md index 897a8f3cd19..7d9bb1ea954 100644 --- a/doc/DOCKER.md +++ b/doc/DOCKER.md @@ -40,6 +40,19 @@ and ARM64. Canonical master pushes also publish for its immutable multi-platform digest. Downstream services can compose their own images from this public base without rebuilding Core. +Stamped standard images include the build-owned remote provider pack at +`/opt/paperclip-runner/provider-pack` and set +`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` to that directory. Downstream +compositions and the explicit `cloud` target inherit both. The pack is required +on the controller for remote OpenCode and ACPX runs, even when the sandbox has +preinstalled providers; setting the environment variable alone cannot repair an +image that omits the pack. Rebuild and redeploy from a standard image containing +the pack if runs fail with `runner_remote_provider_artifact_incompatible`. + +Local Docker builds must supply a full `PAPERCLIP_BUILD_COMMIT` to include a +qualified pack. Unstamped builds remain usable for local adapters and skip pack +generation. The pack stays root-owned and readable by remapped runtime UIDs. + The legacy recurring public `-cloud` publisher is retired. Master pushes, release tags, and manual `Docker` dispatches no longer build that variant. Existing `-cloud` tags and digests remain in the registry for rollback; their diff --git a/docs/deploy/environment-variables.md b/docs/deploy/environment-variables.md index 78ca705b7e6..a663f1f0e80 100644 --- a/docs/deploy/environment-variables.md +++ b/docs/deploy/environment-variables.md @@ -25,7 +25,7 @@ All environment variables that Paperclip uses for server configuration. | `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` | (host build) | Host-local path to a `paperclip-runnerd` artifact built for the remote target OS and architecture. Required when Paperclip and the remote sandbox do not share a compatible platform; build metadata and the required transport mode are verified before launch. | | `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH` | (unset) | Optional host-local path to a Codex executable built for the remote target OS and architecture. For remote Codex-backed runners, Paperclip stages and verifies this executable beside `paperclip-runnerd`. | | `PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC` | (unset) | Optional pinned npm package spec (for example, `@openai/codex@0.156.0`) installed inside each fresh remote lease when its Codex harness is not baked into the sandbox image. Mutually exclusive with `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH`; Paperclip verifies the installed executable before starting `runnerd`. | -| `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` | (unset) | Host-local path to the immutable provider pack built by `pnpm --filter @paperclipai/paperclip-runner build:provider-pack`. The pack includes its target-built Node 24.11 runtime, locked production dependencies, OpenCode proxy/executable, and ACPX sidecar. Remote OpenCode and ACPX fail closed without it. A preinstalled pack is accepted only when its complete digested manifest matches this build-owned pack; otherwise Paperclip stages this pack into the sandbox. | +| `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` | `/opt/paperclip-runner/provider-pack` in Docker; otherwise unset | Host-local path to the immutable provider pack built by `pnpm --filter @paperclipai/paperclip-runner build:provider-pack`. Stamped standard Docker images include the pack; downstream compositions and the `cloud` target inherit it. Unstamped local Docker builds skip pack generation. The pack includes its target-built Node 24.11+ runtime, locked production dependencies, OpenCode proxy/executable, and ACPX sidecar. Remote OpenCode and ACPX fail closed without it. A preinstalled pack is accepted only when its complete digested manifest matches this build-owned pack; otherwise Paperclip stages this pack into the sandbox. | | `PAPERCLIP_HIDDEN_SETTINGS` | (unset) | Comma-separated settings surfaces to hide from the UI and floor at the API, for operators hosting Paperclip for others (managed cloud, internal shared server). See [Hiding settings surfaces](#hiding-settings-surfaces). | | `PAPERCLIP_SETTING_DEFAULTS` | (unset) | JSON object replacing the schema default of selected instance settings, for hosting operators. See [Operator setting defaults](#operator-setting-defaults). | diff --git a/server/src/__tests__/docker-build-stamp.test.ts b/server/src/__tests__/docker-build-stamp.test.ts index 8d2b726bba5..3e52e6e4652 100644 --- a/server/src/__tests__/docker-build-stamp.test.ts +++ b/server/src/__tests__/docker-build-stamp.test.ts @@ -106,21 +106,32 @@ describe("Docker Rust dependency cache", () => { }); -describe("Cloud remote provider pack", () => { - it("ships a build-owned pack without provisioning Grok on the controller", () => { - const pack = stageBody(dockerfile, "cloud-provider-pack"); - const cloud = stageBody(dockerfile, "cloud"); +describe("Standard image remote provider pack", () => { + it("ships the build-owned pack in the production base used by downstream images", () => { + const pack = stageBody(dockerfile, "runner-provider-pack"); + const production = stageBody(dockerfile, "production"); + expect(pack).toContain("FROM build AS runner-provider-pack"); expect(pack).toContain('PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}"'); expect(pack).toContain("build-provider-pack.mjs /provider-pack"); expect(pack).toContain('if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then'); expect(pack).toContain("mkdir -p /provider-pack"); expect(pack).toContain("Skipping remote provider pack"); - expect(cloud).toContain("--from=cloud-provider-pack /provider-pack /opt/paperclip-runner/provider-pack"); - expect(cloud).not.toContain("--chown=node:node --from=cloud-provider-pack"); - expect(cloud).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack"); - expect(cloud).toContain("gosu 65534:65534 node"); - expect(cloud).toContain("Object.values(manifest.payload.artifacts)"); - expect(cloud).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack"); + expect(production).toContain("--from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack"); + expect(production).not.toContain("--chown=node:node --from=runner-provider-pack"); + expect(production).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack"); + expect(production).toContain("gosu 65534:65534 node"); + expect(production).toContain("Object.values(manifest.payload.artifacts)"); + expect(production).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack"); + expect(workflow).toMatch(/^\s*target: production$/m); expect(dockerfile).not.toContain("provision-grok.mjs"); }); + + it("lets explicit cloud preview builds inherit the same pack and configuration", () => { + const cloud = stageBody(dockerfile, "cloud"); + expect(cloud).toContain("FROM production AS cloud"); + expect(cloud).not.toContain("/opt/paperclip-runner/provider-pack"); + expect(cloud).not.toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH"); + expect(dockerfile.match(/build-provider-pack\.mjs \/provider-pack/g)).toHaveLength(1); + expect(previewWorkflow).toMatch(/^\s*target: cloud$/m); + }); });