fix(runner): ship provider pack in the standard Docker image (#14854)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote OpenCode and ACPX runs need a provider pack from the
application build.
> - Cloud now builds its application image from the standard production
image.
> - The provider pack was added only to the legacy cloud image target.
> - The standard image therefore cannot supply the pack to downstream
Cloud images.
> - This pull request adds the pack to the production image and lets the
cloud target inherit it.
> - Remote runs can then use the pack that matches the application
source commit.

## Linked Issues or Issue Description

Refs #13827. Refs #14024.

The standard production image does not include the remote provider pack.
Downstream Cloud images inherit that omission. Remote OpenCode and ACPX
runs fail with `runner_remote_provider_artifact_incompatible` and ask
for `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`.

## What Changed

- Build and copy the provider pack into the standard production image.
- Set the pack path and check that an unprivileged user can read its
artifacts and execute Node.
- Let the legacy cloud target inherit the pack from production.
- Add regression checks for production packaging and cloud inheritance.
- Document stamped image behavior and the default pack path.

## Verification

- The 12 focused Docker stamp and provider-pack reuse tests pass on
commit `4a11f8d52aead55f85527c8e82c6d7f2644ce0da`.
- The new packaging regression failed against the old Dockerfile and
passed with the fix.
- On the current commit, `pnpm build` and `pnpm -r typecheck` pass. All
seven standard-image contract tests also pass.
- The current-head CI build, typecheck, test, browser, and native Runner
checks passed. The local full suite hit one chat-channel assertion
failure; that exact test passed in isolation. The remaining local run
was stopped after CI completed to avoid duplicating its full suite. An
earlier run on the pre-rebase base had a heartbeat comment batching
timeout; the external chat wait integration suite passed all 142 tests
in isolation.
- [The stamped preview image build
passed](https://github.com/paperclipai/paperclip/actions/runs/36885002850/job/110446106393),
including the production-stage provider pack build, copy, and
unprivileged artifact readability/executable check. Publication,
compatibility validation, and deployment of this exact commit to a
staging QA instance passed.
- Reproduced the exact missing-pack error on an existing staging image
with Paperclip Runner, ACPX, and Claude in a remote Daytona computer.
The legacy Claude adapter succeeds with the same account and computer.
After deploying this commit, the same native task succeeded: it computed
`5050` with a real remote shell command, wrote a proof file, read it
back in a separate call, uploaded the file as a deliverable, and
completed the task. The uploaded file contents and Done state persisted
after a page reload. The run trace confirms Paperclip Runner, ACPX, and
Claude. The first run took 2m 59s, including approximately 97s of remote
artifact preparation. A second native run read the unchanged file from
the prior run and completed successfully. Its startup took about 120s;
this verifies repeated execution and file persistence, not fast
provider-pack reuse.

## Risks

- Stamped standard images now include the provider pack and its build
cost. A pack build failure now fails the production image build.
- Unstamped local builds still skip pack generation. Setting the path
alone does not create a pack.
- No database, provider authentication, or runner verification rules
change.

## Model Used

OpenAI Codex, GPT-6. The exact serving model identifier and context
window are not exposed in this session. Capabilities used: repository
inspection, code editing, shell verification, and browser testing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>

Dockerfile

@@ -144,6 +144,21 @@ RUN pnpm --filter @paperclipai/server build
 RUN test -f server/dist/index.js || (echo "ERROR: server build output missing" && exit 1)
 RUN rm -rf packages/paperclip-runner/runner/target
 
+# Remote OpenCode and ACPX runs require a controller-owned provider pack to
+# verify the sandbox installation or stage matching assets. Ship it in the
+# standard image so downstream Cloud compositions inherit the same artifacts.
+# Grok's native executable stays an external sandbox prerequisite; this pack
+# contains only its launcher.
+FROM build AS runner-provider-pack
+# Unstamped local builds remain usable, but cannot qualify a remote pack.
+# Never invent a source revision to make an unqualified pack look verified.
+RUN mkdir -p /provider-pack \
+  && if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then \
+    PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}" node packages/paperclip-runner/scripts/build-provider-pack.mjs /provider-pack; \
+  else \
+    echo "Skipping remote provider pack: supply a full PAPERCLIP_BUILD_COMMIT to enable remote OpenCode/ACPX execution"; \
+  fi
+
 FROM base AS production
 ARG USER_UID=1000
 ARG USER_GID=1000
@@ -169,6 +184,17 @@ RUN chmod +x /usr/local/bin/docker-entrypoint.sh
 
 COPY --chown=node:node --from=build /app /app
 
+COPY --from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack
+# Managed deployments can remap node's UID at startup. This immutable pack
+# contains public code and integrity metadata, never credentials; it must remain
+# readable afterward.
+# Keep it root-owned and verify access as an unrelated unprivileged UID.
+RUN chmod -R a+rX /opt/paperclip-runner/provider-pack \
+  && if [ -f /opt/paperclip-runner/provider-pack/provider-pack.json ]; then \
+    gosu 65534:65534 node -e 'const fs = require("node:fs"); const path = require("node:path"); const root = "/opt/paperclip-runner/provider-pack"; const manifest = JSON.parse(fs.readFileSync(path.join(root, "provider-pack.json"), "utf8")); for (const artifact of Object.values(manifest.payload.artifacts)) fs.readFileSync(path.join(root, artifact.path)); fs.accessSync(path.join(root, manifest.payload.artifacts.nodeCommand.path), fs.constants.X_OK);'; \
+  fi
+ENV PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack
+
 # Declare per-build metadata after the stable RUN layers. Docker includes
 # in-scope ARG values in a RUN's environment even when its command does not
 # mention them; declaring these earlier invalidates the weekly tool cache.
@@ -213,8 +239,8 @@ CMD ["node", "--import", "./server/node_modules/tsx/dist/loader.mjs", "server/di
 # dist/ to exist in the image — the default image ships only their source,
 # so auto-install logs "bundle not present" and skips. The plugins are
 # built in this separate target so the default (self-hosted) image stays
-# lean; CI pins the default build to `--target production`, which is
-# byte-identical to before this stage existed.
+# lean; CI pins the default build to `--target production`. Both targets
+# inherit the build-owned remote provider pack from the standard image.
 #
 # The sandbox providers are intentionally excluded from the pnpm workspace
 # (see pnpm-workspace.yaml), so each installs standalone exactly as its
@@ -293,29 +319,7 @@ RUN set -eu; \
   test -n "$specifiers" || { echo "ERROR: CLOUD_BUNDLED_SERVER_DEPS names no package" >&2; exit 1; }; \
   pnpm add --ignore-workspace --no-lockfile $specifiers
 
-# ACPX remote runs require a controller-owned provider pack to verify the
-# sandbox installation or stage matching assets. Grok's native executable stays
-# an external sandbox prerequisite; this pack contains only its launcher.
-FROM build AS cloud-provider-pack
-# Unstamped local builds remain usable, but cannot qualify a remote pack.
-# Never invent a source revision to make an unqualified pack look verified.
-RUN mkdir -p /provider-pack \
-  && if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then \
-    PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}" node packages/paperclip-runner/scripts/build-provider-pack.mjs /provider-pack; \
-  else \
-    echo "Skipping remote provider pack: supply a full PAPERCLIP_BUILD_COMMIT to enable remote ACPX execution"; \
-  fi
-
 FROM production AS cloud
-COPY --from=cloud-provider-pack /provider-pack /opt/paperclip-runner/provider-pack
-# Cloud remaps node's UID at startup. This immutable pack contains public code
-# and integrity metadata, never credentials; it must remain readable afterward.
-# Keep it root-owned and verify access as an unrelated unprivileged UID.
-RUN chmod -R a+rX /opt/paperclip-runner/provider-pack \
-  && if [ -f /opt/paperclip-runner/provider-pack/provider-pack.json ]; then \
-    gosu 65534:65534 node -e 'const fs = require("node:fs"); const path = require("node:path"); const root = "/opt/paperclip-runner/provider-pack"; const manifest = JSON.parse(fs.readFileSync(path.join(root, "provider-pack.json"), "utf8")); for (const artifact of Object.values(manifest.payload.artifacts)) fs.readFileSync(path.join(root, artifact.path)); fs.accessSync(path.join(root, manifest.payload.artifacts.nodeCommand.path), fs.constants.X_OK);'; \
-  fi
-ENV PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack
 COPY --chown=node:node --from=cloud-plugins /app/packages/plugins/sandbox-providers /app/packages/plugins/sandbox-providers
 # Land the isolated install inside the server's own `node_modules`, the
 # directory Node's module resolution walks up to from `/app/server` for

doc/DOCKER.md

@@ -40,6 +40,19 @@ and ARM64. Canonical master pushes also publish
 for its immutable multi-platform digest. Downstream services can compose their
 own images from this public base without rebuilding Core.
 
+Stamped standard images include the build-owned remote provider pack at
+`/opt/paperclip-runner/provider-pack` and set
+`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` to that directory. Downstream
+compositions and the explicit `cloud` target inherit both. The pack is required
+on the controller for remote OpenCode and ACPX runs, even when the sandbox has
+preinstalled providers; setting the environment variable alone cannot repair an
+image that omits the pack. Rebuild and redeploy from a standard image containing
+the pack if runs fail with `runner_remote_provider_artifact_incompatible`.
+
+Local Docker builds must supply a full `PAPERCLIP_BUILD_COMMIT` to include a
+qualified pack. Unstamped builds remain usable for local adapters and skip pack
+generation. The pack stays root-owned and readable by remapped runtime UIDs.
+
 The legacy recurring public `-cloud` publisher is retired. Master pushes,
 release tags, and manual `Docker` dispatches no longer build that variant.
 Existing `-cloud` tags and digests remain in the registry for rollback; their

docs/deploy/environment-variables.md

@@ -25,7 +25,7 @@ All environment variables that Paperclip uses for server configuration.
 | `PAPERCLIP_RUNNER_REMOTE_BINARY_PATH` | (host build) | Host-local path to a `paperclip-runnerd` artifact built for the remote target OS and architecture. Required when Paperclip and the remote sandbox do not share a compatible platform; build metadata and the required transport mode are verified before launch. |
 | `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH` | (unset) | Optional host-local path to a Codex executable built for the remote target OS and architecture. For remote Codex-backed runners, Paperclip stages and verifies this executable beside `paperclip-runnerd`. |
 | `PAPERCLIP_RUNNER_REMOTE_CODEX_NPM_SPEC` | (unset) | Optional pinned npm package spec (for example, `@openai/codex@0.156.0`) installed inside each fresh remote lease when its Codex harness is not baked into the sandbox image. Mutually exclusive with `PAPERCLIP_RUNNER_REMOTE_CODEX_PATH`; Paperclip verifies the installed executable before starting `runnerd`. |
-| `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` | (unset) | Host-local path to the immutable provider pack built by `pnpm --filter @paperclipai/paperclip-runner build:provider-pack`. The pack includes its target-built Node 24.11 runtime, locked production dependencies, OpenCode proxy/executable, and ACPX sidecar. Remote OpenCode and ACPX fail closed without it. A preinstalled pack is accepted only when its complete digested manifest matches this build-owned pack; otherwise Paperclip stages this pack into the sandbox. |
+| `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` | `/opt/paperclip-runner/provider-pack` in Docker; otherwise unset | Host-local path to the immutable provider pack built by `pnpm --filter @paperclipai/paperclip-runner build:provider-pack`. Stamped standard Docker images include the pack; downstream compositions and the `cloud` target inherit it. Unstamped local Docker builds skip pack generation. The pack includes its target-built Node 24.11+ runtime, locked production dependencies, OpenCode proxy/executable, and ACPX sidecar. Remote OpenCode and ACPX fail closed without it. A preinstalled pack is accepted only when its complete digested manifest matches this build-owned pack; otherwise Paperclip stages this pack into the sandbox. |
 | `PAPERCLIP_HIDDEN_SETTINGS` | (unset) | Comma-separated settings surfaces to hide from the UI and floor at the API, for operators hosting Paperclip for others (managed cloud, internal shared server). See [Hiding settings surfaces](#hiding-settings-surfaces). |
 | `PAPERCLIP_SETTING_DEFAULTS` | (unset) | JSON object replacing the schema default of selected instance settings, for hosting operators. See [Operator setting defaults](#operator-setting-defaults). |
 

server/src/__tests__/docker-build-stamp.test.ts

@@ -106,21 +106,32 @@ describe("Docker Rust dependency cache", () => {
 });
 
 
-describe("Cloud remote provider pack", () => {
-  it("ships a build-owned pack without provisioning Grok on the controller", () => {
-    const pack = stageBody(dockerfile, "cloud-provider-pack");
-    const cloud = stageBody(dockerfile, "cloud");
+describe("Standard image remote provider pack", () => {
+  it("ships the build-owned pack in the production base used by downstream images", () => {
+    const pack = stageBody(dockerfile, "runner-provider-pack");
+    const production = stageBody(dockerfile, "production");
+    expect(pack).toContain("FROM build AS runner-provider-pack");
     expect(pack).toContain('PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}"');
     expect(pack).toContain("build-provider-pack.mjs /provider-pack");
     expect(pack).toContain('if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then');
     expect(pack).toContain("mkdir -p /provider-pack");
     expect(pack).toContain("Skipping remote provider pack");
-    expect(cloud).toContain("--from=cloud-provider-pack /provider-pack /opt/paperclip-runner/provider-pack");
-    expect(cloud).not.toContain("--chown=node:node --from=cloud-provider-pack");
-    expect(cloud).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack");
-    expect(cloud).toContain("gosu 65534:65534 node");
-    expect(cloud).toContain("Object.values(manifest.payload.artifacts)");
-    expect(cloud).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack");
+    expect(production).toContain("--from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack");
+    expect(production).not.toContain("--chown=node:node --from=runner-provider-pack");
+    expect(production).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack");
+    expect(production).toContain("gosu 65534:65534 node");
+    expect(production).toContain("Object.values(manifest.payload.artifacts)");
+    expect(production).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack");
+    expect(workflow).toMatch(/^\s*target: production$/m);
     expect(dockerfile).not.toContain("provision-grok.mjs");
   });
+
+  it("lets explicit cloud preview builds inherit the same pack and configuration", () => {
+    const cloud = stageBody(dockerfile, "cloud");
+    expect(cloud).toContain("FROM production AS cloud");
+    expect(cloud).not.toContain("/opt/paperclip-runner/provider-pack");
+    expect(cloud).not.toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH");
+    expect(dockerfile.match(/build-provider-pack\.mjs \/provider-pack/g)).toHaveLength(1);
+    expect(previewWorkflow).toMatch(/^\s*target: cloud$/m);
+  });
 });