Files
PaperClipAI/doc/DOCKER.md
T
Nicky LeachandClaude Fable 5.2 647641a99b feat(ci): let docker.yml copy the published Runner image, behind a repository variable (#21340)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner is the part that executes agent work. `master` publishes
it alone as an attested image, once for each set of its inputs
> - Since #21332 the application image takes its Runner only from the
`runner-artifacts` stage, and a build can replace that stage with the
published image
> - No workflow does that yet, so every application image still builds
its own Runner, and nothing on an application image says which Runner it
has
> - A build that copies the published Runner must also build the
lockfile that Runner was built from. The workflow refreshes the lockfile
before every build, and the server build then refuses the published
Runner
> - This pull request lets `docker.yml` copy the published Runner image
and label the application image with it, behind a repository variable
that is off, and makes that build use the committed lockfile
> - The benefit is one Runner for all images of the same inputs, named
on the image, with a switch that goes back to today's builds without a
revert

## Linked Issues or Issue Description

Refs #21204, #21229, #21230 (the Runner image), #21306 (the resolver),
#21307 (the report-only job), #21308 (the server reads a pack from its
Runner image), and #21331, #21332, #21333 (the application image takes
its Runner from `runner-artifacts`). No issue exists for this change.

**Problem or motivation**

`docker.yml` builds the application image on `master` and on tags. Each
build makes its own Runner. The report-only job that runs the resolver
on `master` feeds no build. Downstream image composers cannot tell from
an application image which Runner it has.

There is a second problem, which only shows when a build copies the
published Runner. The Runner's build stamp covers the whole
`pnpm-lock.yaml`. The step that vendors the Runner into the server fails
when the stamp does not match the tree. `docker.yml` refreshes
`pnpm-lock.yaml` on the runner before every build. The refresh rewrites
the file whenever the manifest of any package is ahead of it, also a
package the Runner does not read. On the tree of this pull request it
rewrites the file even for a pristine checkout: the registry changed the
deprecation text of one package. The published Runner was built from the
committed file, so the image build would fail.

**Proposed solution**

One job resolves the Runner once for the run. Both architecture builds
need it and add its answer to their build: a build context that replaces
the `runner-artifacts` stage, two labels, and the build argument with
which the build checks the Runner. A build that copies the published
Runner does not refresh the lockfile: it builds the committed tree, the
tree the Runner was published for. The job runs only when the repository
variable `RUNNER_IMAGE_CONSUME_ENABLED` is exactly `true`. This pull
request merges with the variable unset, so no image changes when it
merges.

**Alternatives considered**

- Resolve in each architecture build. The two builds could then
disagree, and one image index would hold two Runners.
- Write the labels in the `Dockerfile`. A `LABEL` cannot be left out for
an inline build.
- For the lockfile, copy the Runner image's `/pnpm-lock.yaml` over
`/app/pnpm-lock.yaml` in the `build` stage. Nothing after that step
reads the file: the two vendor checks are its only readers in `build`,
`production` copies it and takes `node_modules` from `deps`, both
`cloud-*` stages install with `--ignore-workspace --no-lockfile`, and
the server reads only a plugin's own lockfile and the pack's own
lockfile at run time. But the image would then ship a lockfile that its
`node_modules` did not come from, the copy changes that file's layer for
every inline build, and each `Dockerfile` edit makes a new Runner image.
- For the lockfile, bind-mount the Runner image's lockfile over the file
for the two steps that run the vendor check. This also edits the
`Dockerfile`, and it makes the lockfile part of the check always true
for a published Runner.
- For the lockfile, let the resolver refresh and compare. The answer
would depend on the registry at two different moments.
- Skipping the refresh needs no `Dockerfile` edit, keeps the stamp check
as strict as it is, and builds exactly the tree the inputs digest was
computed from.

**Roadmap alignment**

This is build infrastructure. It adds no product feature and does not
overlap an item in `ROADMAP.md`.

This pull request is built on `runner/app-image-consume`. Merge in this
order:

1. #21331, #21332, #21333: the application image takes its Runner from
the `runner-artifacts` stage, checks the inputs digest a build names,
and stops early on a stale lockfile.
2. `runner/app-image-consume`: the wait and the `unsupported` answer of
the resolver.
3. This pull request.

The same change for the preview cloud image build in `release.yml` is
deferred. It has no pull request yet.

## What Changed

- `.github/workflows/docker.yml`: the job `runner_image` replaces
`runner_image_report`.
- It runs only when `vars.RUNNER_IMAGE_CONSUME_ENABLED == 'true'`, in
this repository. A fork never runs it.
- Steps: `inputs` with no credential; `resolve`; then `gh attestation
verify` of the full index against the signer `runner-image.yml` on
`refs/heads/master`.
- For a `master` push, `resolve` runs with `--wait` and the job token.
For a tag or a manual run it does not wait and receives no token.
- Only the verification step writes the job's outputs. An image that
does not verify fails that step with a warning and writes no output. The
job is `continue-on-error`: the job is red, the run is not, and the
builds are inline.
- Permissions: `contents: read`, `actions: read`, `attestations: read`.
No registry login, no write.
- Limits: 5 minutes for `inputs`, 55 for the lookup, 5 for the
verification, 70 for the job. The wait of 40 minutes ends inside them.
- `build-and-push` needs `runner_image` and adds its outputs to the
build: `build-contexts`, one more line of `build-args`
(`PAPERCLIP_RUNNER_EXPECT_INPUTS_DIGEST`), two more lines of `labels`.
With empty outputs these are the values the build had before.
- The step "Refresh lockfile for Docker build context" runs only when
the answer is not `published`.
- The step that looks for a merge-queue image of the commit runs only
when the answer is not `published`. The two steps after it already
depend on its result.
- Labels on an image that copied the published Runner:
`io.paperclip.runner.image=ghcr.io/paperclipai/paperclip-runner@sha256:<full
index digest>` and `io.paperclip.runner.inputs-digest=<digest>`. An
image without them made its own Runner.
- The `Dockerfile` does not change. The Runner image digest does not
move.
- Tests that pin lines this changes follow:
`scripts/merge-group-image.test.mjs` (the condition of the merge-queue
lookup), `scripts/reuse-canary-image.test.mjs` (the `needs` of the build
job), and `docker-build-stamp.test.ts` (no workflow names a build
context; `docker.yml` now names exactly one).
- `server/src/__tests__/docker-runner-workflows.test.ts`: the
report-only tests give way to tests of the new job.
- `server/scripts/prepare-runner-vendor.test.mjs`: one fixture test of
the lockfile case.
- `doc/runner-image-contract.md` and `doc/DOCKER.md`: how an application
image gets its Runner, the labels and what their absence means, the
variable, the wait, the verification, the lockfile rule, the merge-queue
rule and the rollback.

## Verification

### Real builds

`docker buildx build --target build --platform linux/arm64 --output
type=cacheonly`, with a `docker-container` builder made for this test.
The Runner image is the publication form
(`PAPERCLIP_RUNNER_IMAGE_PUBLICATION=1`, `--target runner-artifacts`)
built from a clean export of this branch, exported as an OCI layout:
inputs `c4f9d089…34f0`, image `sha256:bef7e9cb…ffef`. The application
builds receive `--build-context
runner-artifacts=oci-layout://…@sha256:bef7e9cb…` and
`PAPERCLIP_RUNNER_EXPECT_INPUTS_DIGEST=c4f9d089…`. The refresh command
is the one in the workflow, run with pnpm 9.15.4 in the export: `pnpm
install --resolution-only --ignore-scripts --no-frozen-lockfile`.

| Build context | Result |
| --- | --- |
| The committed tree | Passes in 78 s. Both vendor checks: `reusing the
current Runner build (inputs unchanged …; 1333 files and 1860 outputs
checked)` |
| `ui/package.json`: `clsx` `^2.1.1` → `^2.1.0`, then the refresh
command (what the workflow did before this change) | Fails at `RUN node
server/scripts/prepare-runner-vendor.mjs`:
`PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1 forbids the Runner rebuild
this step needs (inputs changed since the recorded build; 1333 files
checked)` |
| The committed tree, then only the refresh command | Fails the same
way. The refresh changed only the `deprecated` text of one package |
| The same `ui/package.json` change with no refresh (what the workflow
does now for a `published` answer) | Passes in 66 s. `deps` resolves the
manifest inside the image; both vendor checks reuse the Runner |
| One line added to `packages/paperclip-runner/src/index.ts` | Fails at
the vendor step, as before: a Runner of other inputs is still refused |

Lockfile behind the Runner's own manifests
(`packages/paperclip-runner/package.json`: `smol-toml` `^1.9.1` →
`^1.9.0`), no build context:

| Step | Result |
| --- | --- |
| `runner-image-resolve.mjs inputs`, then `resolve` | `lock: behind`, no
digest; `inline (lock-behind)` |
| Build without the refresh | Fails in 4 s at the `runner-lockfile`
stage: `ERROR: pnpm-lock.yaml is behind the manifests the Runner is
installed from …` |
| Build after the refresh command (what the workflow does for an
`inline` answer) | Passes in 193 s: the `runner-lockfile` stage passes,
the build makes its own Runner with its pack, and both vendor checks
reuse it |

### Tests

macOS (Node 26.12.0) and Linux (`node:24.22.3-bookworm`, linux/arm64,
with `jq` and `zstd` added), on this branch:

- `pnpm exec vitest run` on `docker-runner-workflows`,
`runner-image-resolve`, `runner-image-workflow`, `docker-build-stamp`,
`container-init-reaping`, `cloud-image-bundled-plugins`,
`docker-image-layer-reuse`, `cloud-image-sentry`,
`runner-lockfile-guards` and `runner-image-identity` in
`server/src/__tests__`, and
`server/scripts/prepare-runner-vendor.test.mjs`: 225 tests in 11 files
pass on Linux; on macOS 224 pass and 1 is skipped (it needs bash 4.4).
- `node --test .github/scripts/tests/*.test.mjs
scripts/merge-group-image.test.mjs scripts/reuse-canary-image.test.mjs
scripts/cloud-source-verification.test.mjs
scripts/standard-image-contract.test.mjs
scripts/preview-artifacts.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs
scripts/__tests__/release-dry-run-notes.test.mjs`: 991 of 991 pass on
both.
- The changed test files typecheck with the options of
`server/src/__tests__/tsconfig.json`.

What the workflow tests hold:

- Variable off: the job's condition is the variable and the repository,
with no alternative. With empty outputs, the build step has exactly the
3 build arguments, 0 build contexts and 1 label entry it had before, and
the refresh step runs.
- Published: both architecture builds are one job that reads only the
outputs of `runner_image`, with no architecture in those lines.
- The verification step runs as real shell against a stub `gh`. When
`gh` succeeds it writes the context, the labels and the build argument.
When `gh` fails the step exits 1, writes no output and prints the
warning. The step has no `continue-on-error`; the job has. The condition
of the build job does not name the resolver job and has no `success()`.
- The lookup step runs as real shell against a stub `node`. It passes
`--wait` only when the event is a `master` push.
- Credentials: only the lookup and the verification steps have a token.
The steps that build have no `env`.
- The refresh step has the condition `needs.runner_image.outputs.mode !=
'published'`, and it is the only `pnpm install` of the build job.
- The fixture test gives the vendor step a Runner that another tree
built for the committed lockfile. The step reuses it. After a change of
one specifier of another package in the lockfile the step fails and
names the lockfile. With the committed lockfile back it reuses the
Runner again.
- The merge-queue workflow and the pull-request check do not name the
resolver, the variable, a build context or a Runner label.

Read-only checks against the real services:

- `gh attestation verify
oci://ghcr.io/paperclipai/paperclip-runner@sha256:0cd8250b…1bf7 --repo
paperclipai/paperclip --signer-workflow
paperclipai/paperclip/.github/workflows/runner-image.yml --source-ref
refs/heads/master`: exit 0 for the full index, and exit 0 for the slim
index `sha256:69ab2001…69b3`.
- The same command with `--signer-workflow …/docker.yml`: exit 1 for
both. With `--source-ref refs/heads/not-master`: exit 1. With `--repo
cli/cli`: exit 1. For a digest that does not exist: exit 1. With no
token: exit 4.
- The parsed YAML of `docker.yml` differs from its base only in the
resolver job, the `needs` of the build job, the conditions of the
refresh step and of the merge-queue lookup, and the three build inputs.

Never run:

- `docker.yml`. Only GitHub can run it, and it has no pull-request
trigger.
- A build with the variable on. The first one is the first run after a
maintainer sets the variable.
- A build that pulls the published Runner from the registry as
`docker-image://`. The real builds used the same image form from a local
OCI layout.
- An `amd64` build.
- The `production` and `cloud` targets. The real builds stopped at
`build`, the last stage that checks the Runner.

## Risks

- With the variable on, an application build on `master` starts after
the resolver job. That is 2 to 3 minutes when the image exists, and up
to 40 minutes from the push when the commit's own Runner publication is
in progress. `docker.yml` holds one concurrency slot for each ref, so a
long wait can make more `master` commits lose their image build to a
newer push.
- A build that copies the published Runner ships the committed
`pnpm-lock.yaml` in `/app`, while `deps` resolves newer manifests of
other packages inside the image. A local `docker build` has always done
this; a CI image did not. Nothing in the image reads that file.
- Any other build that gives the `Dockerfile` a published Runner and a
refreshed lockfile fails at the vendor step. That is the intended answer
to a tree the Runner was not built for, and the message names the
lockfile.
- The pack of an image that copied the published Runner names the commit
that published the Runner (`runnerSourceRevision`), which can be older
than the commit of the image. A consumer that compares the two must
accept that before the variable is set.
- A failed verification makes the resolver job red on a green run. That
is on purpose: it means a published tag whose attestation does not
verify.
- `scripts/merge-group-image.mjs` refuses to promote a merge-queue image
for a push that changes `docker.yml`. The push that lands this change is
such a push. The merge queue is not enabled.
- With the variable off there is no report of the resolver on `master`
any more: the job that replaced the report-only job does not run.
- Rollback: set the variable to another value or delete it. The next run
builds inline, with the refresh. Images that exist do not change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude
Code with tool use (shell, file edits, local Docker builds, read-only
calls to the GitHub API and the container registry).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-13 14:47:20 -08:00

487 lines
23 KiB
Markdown

# Docker Quickstart
Run Paperclip in Docker without installing Node or pnpm locally.
All commands below assume you are in the **project root** (the directory containing `package.json`), not inside `docker/`.
## Building the image
```sh
docker build -t paperclip-local .
```
The Dockerfile installs common agent tools (`git`, `gh`, `curl`, `wget`, `ripgrep`, `python3`) and the Claude, Codex, and OpenCode CLIs.
Build arguments:
| Arg | Default | Purpose |
|-----|---------|---------|
| `USER_UID` | `1000` | UID for the container `node` user (match your host UID to avoid permission issues on bind mounts) |
| `USER_GID` | `1000` | GID for the container `node` group |
| `CLI_TOOLS_CACHE_EPOCH` | empty | Refresh the CLI-install layer; CI supplies the current ISO week |
| `PAPERCLIP_BUILD_VERSION` | empty | Runtime version when Git metadata is unavailable |
| `PAPERCLIP_BUILD_COMMIT` | empty | Source commit written into the server build stamp and runtime environment |
| `PAPERCLIP_RUNNER_EXPECT_INPUTS_DIGEST` | empty | Inputs digest of the published Runner image a build was given as `runner-artifacts`; the build fails on any other Runner. Leave it empty for a build that makes its own Runner |
Changing the build version or commit preserves the CLI-install cache. The
tool layer refreshes when its weekly epoch, base image, installation command,
or earlier build inputs change. Local builds can set a new epoch explicitly
to refresh tools without clearing the entire build cache.
```sh
docker build -t paperclip-local \
--build-arg USER_UID=$(id -u) --build-arg USER_GID=$(id -g) .
```
The image starts the server from a runtime bundle, `server/dist-runtime/index.mjs`,
with plain `node`. The build creates it from the server's `tsc` output with
`pnpm --filter @paperclipai/server build:runtime` (`server/scripts/bundle-runtime.mjs`),
compiling in the workspace packages that otherwise load as TypeScript source.
This avoids transpiling them through the tsx loader on every boot. To check a
build boots, run `scripts/server-boot-smoke.sh node server/dist-runtime/index.mjs`;
PR CI runs the same check after `pnpm build`.
## Standard images and downstream composition
The Docker workflow publishes the standard `production` target for Linux AMD64
and ARM64. Canonical master pushes also publish
`ghcr.io/paperclipai/paperclip:sha-<FULL_SHA>` and a GitHub/Sigstore attestation
for its immutable multi-platform digest. Downstream services can compose their
own images from this public base without rebuilding Core.
Stamped standard images include the build-owned remote provider pack at
`/opt/paperclip-runner/provider-pack` and set
`PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH` to that directory. Downstream
compositions and the explicit `cloud` target inherit both. The pack is required
on the controller for remote OpenCode and ACPX runs, even when the sandbox has
preinstalled providers; setting the environment variable alone cannot repair an
image that omits the pack. Rebuild and redeploy from a standard image containing
the pack if runs fail with `runner_remote_provider_artifact_incompatible`.
Local Docker builds must supply a full `PAPERCLIP_BUILD_COMMIT` to include a
qualified pack. Unstamped builds remain usable for local adapters and skip pack
generation. The pack stays root-owned and readable by remapped runtime UIDs.
The legacy recurring public `-cloud` publisher is retired. Master pushes,
release tags, and manual `Docker` dispatches no longer build that variant.
Existing `-cloud` tags and digests remain in the registry for rollback; their
release-channel aliases no longer advance. This change deletes no images,
cache tags, or migrator artifacts.
The `cloud` Dockerfile target remains available for explicit
[preview builds](preview-release-artifacts.md). Those requests still publish a
full-SHA `-cloud` tag when needed. They do not advance a release channel or
replace downstream private composition.
A published image alone does not prove source tests or migration compatibility.
Downstream deployment tooling must verify [source proof](cloud-build-readiness.md),
the standard image attestation, the exact-source migrator, and its own composed
image before rollout. Resolve immutable digests instead of deploying mutable tags.
## Runner image
The `runner-artifacts` target builds the Runner alone: the daemon, the Runner
package's build output, the remote provider pack and its transport archives.
The `runner-artifacts-slim` target is the same image without the transport
archives. Neither depends on the server or UI build.
`.github/workflows/runner-image.yml` builds and checks
both on pull requests, for Linux AMD64 and ARM64, and publishes them from
`master` as `ghcr.io/paperclipai/paperclip-runner`. See
[`doc/runner-image-contract.md`](runner-image-contract.md) for the layout, the
identity rules and how to consume the image.
The application image takes its Runner from the `runner-artifacts` stage. It
has no Runner build of its own, no Rust toolchain and no C compiler. Every
build of the `production` or `cloud` target builds that stage itself, once,
unless it is given a published Runner image in its place (see "The Runner of
the application image" below):
- The `build` stage copies the Runner's build output and its build stamp into
`packages/paperclip-runner`. It sets `PAPERCLIP_RUNNER_VENDOR_REQUIRE_CURRENT=1`,
so the server build vendors that output and fails when the stamp does not
match the source tree. It never builds the Runner again.
- The `runner-provider-pack` stage copies the pack and its transport archives
out of `runner-artifacts` and moves the pack's large files to a tree of their
own. The image keeps its paths under `/opt/paperclip-runner/` and its layers.
The daemon in the application image is therefore the stripped binary compiled
on Debian bookworm, and the pack's Node is the pinned bookworm build. The
build runs the steps of the Runner's `build` script that write output, not its
drift checks; repository CI runs those. See "The application image" in the
contract for the rules a consumer of the application image can rely on.
The Runner stages install with a frozen lockfile, so `docker build` fails when
`pnpm-lock.yaml` is behind the manifests they read: the root `package.json`,
`packages/paperclip-runner` and `packages/paperclip-eval-kernel`. The
`runner-lockfile` stage makes that check from those manifests and the lockfile
alone, so it runs as soon as the `base` stage is built. The `deps` stage starts
from it, so the application's dependency install cannot start before the check
passes. A stale lockfile fails the build within seconds of the check running,
with a message that names the command below. The Runner's own frozen install
fails on the same lockfile, but it comes after the daemon compile. Stages that
do not depend on the check can already be running when it fails: the Runner's
toolchain download and daemon compile, and the tool layer of the `production`
stage. The failed check stops the build and cancels them. The `deps` stage
still resolves every other manifest change inside the image. Refresh the
lockfile in the checkout, then build again. Do not commit the result (see
"Dependency Lockfile Policy" in `doc/DEVELOPING.md`):
```sh
pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
```
### The Runner of the application image
An application image either makes its own Runner in its build (inline) or
copies the published Runner image of the same inputs. A local build, a fork,
a pull request and the merge-queue build are always inline.
`.github/workflows/docker.yml` in this repository copies the published image
when the repository variable `RUNNER_IMAGE_CONSUME_ENABLED` is exactly `true`
and the image of the commit's inputs is published and its attestation
verifies. On a `master` push it waits for the commit's own run of
`runner-image.yml`, at most 40 minutes from the push. In every other case the
build is inline, with a notice: no state of the Runner image fails an
application image build. A build that copies the published image builds the
committed `pnpm-lock.yaml`: the workflow does not refresh it first, and the
`deps` stage resolves the other packages' manifests inside the image.
An image that copied the published Runner has the labels
`io.paperclip.runner.image` (`ghcr.io/paperclipai/paperclip-runner@sha256:<digest>`)
and `io.paperclip.runner.inputs-digest`. An image without them made its own
Runner and is as complete. To go back to inline builds everywhere, set the
variable to another value or delete it; nothing needs a revert. See "The
Runner of the application image" in
[`doc/runner-image-contract.md`](runner-image-contract.md).
`.github/workflows/docker.yml` builds the application image on `master` and on
tags only. `.github/workflows/docker-app-check.yml` builds the `build` and
`runner-provider-pack` stages on a pull request that changes the `Dockerfile`,
`.dockerignore` or the step that vendors the Runner into the server, so a
change to how the application image gets its Runner fails before it merges.
It refreshes the lockfile before the build, as `docker.yml` does. It holds no
credential: it reads the build cache that `docker.yml` writes on `master`
anonymously, never writes it, and exports and publishes nothing.
## One-liner (build + run)
```sh
docker build -t paperclip-local . && \
docker run --name paperclip \
-p 3100:3100 \
-e HOST=0.0.0.0 \
-e PAPERCLIP_HOME=/paperclip \
-e BETTER_AUTH_SECRET=$(openssl rand -hex 32) \
-e PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=$(openssl rand -hex 32) \
-v "$(pwd)/data/docker-paperclip:/paperclip" \
paperclip-local
```
Open: `http://localhost:3100`
Data persistence:
- Embedded PostgreSQL data
- uploaded assets
- local secrets key
- local agent workspace data
All persisted under your bind mount (`./data/docker-paperclip` in the example above).
## Docker Compose
### Persistent signing secrets
Before the first launch, generate two independent values with `openssl rand -hex 32`
and save them in `docker/.env`. Keep any existing deployment's values unchanged:
```dotenv
BETTER_AUTH_SECRET=<session-signing-secret>
PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=<tool-action-signing-secret>
```
Restrict this file to its owner (`chmod 600 docker/.env`) and keep it out of version
control. Compose reads it for both commands below. Reuse the values on every
restart. Rotate the tool-action secret only after outstanding approvals are
resolved, because pending signatures use the previous value.
### Quickstart (embedded SQLite)
Single container, no external database. Data persists via a bind mount.
```sh
docker compose -f docker/docker-compose.quickstart.yml up --build
```
Defaults:
- host port: `3100`
- persistent data dir: `./data/docker-paperclip`
Optional overrides:
```sh
PAPERCLIP_PORT=3200 PAPERCLIP_DATA_DIR=../data/pc \
docker compose -f docker/docker-compose.quickstart.yml up --build
```
**Note:** `PAPERCLIP_DATA_DIR` is resolved relative to the compose file (`docker/`), so `../data/pc` maps to `data/pc` in the project root.
If you change host port or use a non-local domain, set `PAPERCLIP_PUBLIC_URL` to the external URL you will use in browser/auth flows.
Pass `OPENAI_API_KEY` and/or `ANTHROPIC_API_KEY` to enable local adapter runs.
### Full stack (with PostgreSQL)
Paperclip server + PostgreSQL 17. The database is health-checked before the server starts.
```sh
docker compose -f docker/docker-compose.yml up --build
```
PostgreSQL data persists in a named Docker volume (`pgdata`). Paperclip data persists in `paperclip-data`.
Both Compose files pass `PAPERCLIP_TOOL_ACTION_SIGNING_SECRET` to the server.
When configured, keep it independent from `BETTER_AUTH_SECRET`. When unset,
the server derives a separate signing key from its persisted secrets master key.
Keep that key with database backups, and share the signing key or master key
across processes serving the same instance. Approvals fail closed if valid key
material cannot be loaded; rotating it can invalidate approvals and retry matching.
### Untrusted PR review
Isolated container for reviewing untrusted pull requests with Codex or Claude, without exposing your host machine. See `doc/UNTRUSTED-PR-REVIEW.md` for the full workflow.
```sh
docker compose -f docker/docker-compose.untrusted-review.yml build
docker compose -f docker/docker-compose.untrusted-review.yml run --rm --service-ports review
```
## Authenticated Compose (Single Public URL)
For authenticated deployments, set one canonical public URL and let Paperclip derive auth/callback defaults:
```yaml
services:
paperclip:
environment:
PAPERCLIP_DEPLOYMENT_MODE: authenticated
PAPERCLIP_DEPLOYMENT_EXPOSURE: private
PAPERCLIP_PUBLIC_URL: https://desk.koker.net
```
`PAPERCLIP_PUBLIC_URL` is used as the primary source for:
- auth public base URL
- Better Auth base URL defaults
- bootstrap invite URL defaults
- hostname allowlist defaults (hostname extracted from URL)
For fresh `authenticated/private` Docker or appliance-style installs, the first
admin can now be claimed entirely from the browser after sign-in. Open the
Paperclip URL, sign in or create an account, then choose `Claim this instance`
on the setup screen. This browser claim is disabled for `authenticated/public`;
public deployments should run the high-entropy CLI invite fallback instead:
```sh
pnpm paperclipai auth bootstrap-ceo
```
Granular overrides remain available if needed (`PAPERCLIP_AUTH_PUBLIC_BASE_URL`, `BETTER_AUTH_URL`, `BETTER_AUTH_TRUSTED_ORIGINS`, `PAPERCLIP_ALLOWED_HOSTNAMES`).
Set `PAPERCLIP_ALLOWED_HOSTNAMES` explicitly only when you need additional hostnames beyond the public URL host (for example Tailscale/LAN aliases or multiple private hostnames).
### Optional Vercel Connect credentials
Vercel Connect's backend integration is retained for controlled testing and
existing Vercel-backed connections, but its new-connection UI is currently
withheld from **Apps → Browse**. Setting
`PAPERCLIP_VERCEL_CONNECT_ENABLED=true` does not expose a customer-facing setup
entry. Native provider setup screens remain unchanged. Vercel-hosted deployments use the
workload OIDC token Vercel injects. Other hosted and self-hosted deployments
can provide `PAPERCLIP_VERCEL_CONNECT_ACCESS_TOKEN` as a deployment bootstrap
secret only when that token type is accepted by the live Connect API:
```yaml
services:
paperclip:
environment:
PAPERCLIP_VERCEL_CONNECT_ENABLED: "true"
PAPERCLIP_VERCEL_CONNECT_ACCESS_TOKEN: ${PAPERCLIP_VERCEL_CONNECT_ACCESS_TOKEN}
```
Do not save that access token in a company secret or connection config. It is
instance bootstrap authority for the operator-selected Vercel account. A token's
long expiry and broad Vercel scope do not prove Connect compatibility; validate
it with connector metadata before rollout. Workload OIDC takes precedence when
both authorities are present. Turning the feature flag off hides new
Vercel-backed setup; existing connections keep resolving while workload OIDC or
the bootstrap token remains available. Missing or invalid authority fails
closed. See the [Vercel Connect operator guide](./connections/VERCEL-CONNECT.md).
## Claude + Codex Local Adapters in Docker
The image pre-installs:
- `claude` (Anthropic Claude Code CLI)
- `codex` (OpenAI Codex CLI)
- `python3` (used to run local subscription sign-in in a pseudo-terminal)
Start the container, then open **Apps → Connections** to connect a Claude or
Codex subscription in your browser. You can also enter a provider API key there.
You do not run a shell command or configure a CLI home.
```sh
docker run --name paperclip \
-p 3100:3100 \
-e HOST=0.0.0.0 \
-e PAPERCLIP_HOME=/paperclip \
-v "$(pwd)/data/docker-paperclip:/paperclip" \
paperclip-local
```
Notes:
- The image provides the CLI and Python prerequisites for local subscription sign-in.
- Adapter environment checks in Paperclip surface missing authentication or CLI prerequisites.
## Podman Quadlet (systemd)
The `docker/quadlet/` directory contains unit files to run Paperclip + PostgreSQL as systemd services via Podman Quadlet.
| File | Purpose |
|------|---------|
| `docker/quadlet/paperclip.pod` | Pod definition — groups containers into a shared network namespace |
| `docker/quadlet/paperclip.container` | Paperclip server — joins the pod, connects to Postgres at `127.0.0.1` |
| `docker/quadlet/paperclip-db.container` | PostgreSQL 17 — joins the pod, health-checked |
### Setup
1. Build the image (see above).
2. Copy quadlet files to your systemd directory:
```sh
# Rootless (recommended)
cp docker/quadlet/*.pod docker/quadlet/*.container \
~/.config/containers/systemd/
# Or rootful
sudo cp docker/quadlet/*.pod docker/quadlet/*.container \
/etc/containers/systemd/
```
3. Create a secrets env file (keep out of version control):
```sh
cat > ~/.config/containers/systemd/paperclip.env <<EOL
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=$(openssl rand -hex 32)
POSTGRES_USER=paperclip
POSTGRES_PASSWORD=paperclip
POSTGRES_DB=paperclip
DATABASE_URL=postgres://paperclip:paperclip@127.0.0.1:5432/paperclip
# OPENAI_API_KEY=sk-...
# ANTHROPIC_API_KEY=sk-...
EOL
```
4. Create the data directory and start:
```sh
mkdir -p ~/.local/share/paperclip
systemctl --user daemon-reload
systemctl --user start paperclip-pod
```
### Quadlet management
```sh
journalctl --user -u paperclip -f # App logs
journalctl --user -u paperclip-db -f # DB logs
systemctl --user status paperclip-pod # Pod status
systemctl --user restart paperclip-pod # Restart all
systemctl --user stop paperclip-pod # Stop all
```
### Quadlet notes
- **First boot**: Unlike Docker Compose's `condition: service_healthy`, Quadlet's `After=` only waits for the DB unit to *start*, not for PostgreSQL to be ready. On a cold first boot you may see one or two restart attempts in `journalctl --user -u paperclip` while PostgreSQL initialises — this is expected and resolves automatically via `Restart=on-failure`.
- Containers in a pod share `localhost`, so Paperclip reaches Postgres at `127.0.0.1:5432`.
- PostgreSQL data persists in the `paperclip-pgdata` named volume.
- Paperclip data persists at `~/.local/share/paperclip`.
- For rootful quadlet deployment, remove `%h` prefixes and use absolute paths.
## Onboard Smoke Test (Ubuntu + npm only)
Use this when you want to mimic a fresh machine that only has Ubuntu + npm and verify:
- `npx paperclipai onboard --yes` completes
- the server binds to `0.0.0.0:3100` so host access works
- onboard/run banners and startup logs are visible in your terminal
Build + run:
```sh
./scripts/docker-onboard-smoke.sh
```
Open: `http://localhost:3131` (default smoke host port)
Useful overrides:
```sh
HOST_PORT=3200 PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
PAPERCLIP_DEPLOYMENT_MODE=authenticated PAPERCLIP_DEPLOYMENT_EXPOSURE=private ./scripts/docker-onboard-smoke.sh
SMOKE_DETACH=true SMOKE_METADATA_FILE=/tmp/paperclip-smoke.env PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
```
Notes:
- Persistent data is mounted at `./data/docker-onboard-smoke` by default.
- Container runtime user id defaults to your local `id -u` so the mounted data dir stays writable while avoiding root runtime.
- Smoke script defaults to `authenticated/private` mode so `HOST=0.0.0.0` can be exposed to the host.
- Smoke script defaults host port to `3131` to avoid conflicts with local Paperclip on `3100`.
- Smoke script also defaults `PAPERCLIP_PUBLIC_URL` to `http://localhost:<HOST_PORT>` so bootstrap invite URLs and auth callbacks use the reachable host port instead of the container's internal `3100`.
- In authenticated mode, the smoke script defaults `SMOKE_AUTO_BOOTSTRAP=true` and drives the real bootstrap path automatically: it signs up a real user, runs `paperclipai auth bootstrap-ceo` inside the container to mint a real bootstrap invite, accepts that invite over HTTP, and verifies board session access.
- Run the script in the foreground to watch the onboarding flow; stop with `Ctrl+C` after validation.
- Set `SMOKE_DETACH=true` to leave the container running for automation and optionally write shell-ready metadata to `SMOKE_METADATA_FILE`.
- Set `SMOKE_CONTAINER_NAME` to fix the container's name up front. Automation that has to collect diagnostics when the script *fails* needs a name it already knows, rather than one it can only read back out of a successful run. Defaults to the image name.
- The container's logs are dumped to `SMOKE_LOG_FILE` (default `$TMPDIR/<container name>.log`) before the script tears the container down, so a run that never became ready still leaves its logs behind.
- The image definition is in `docker/Dockerfile.onboard-smoke`.
## General Notes
- The `docker-entrypoint.sh` adjusts the container `node` user UID/GID at startup to match the values passed via `USER_UID`/`USER_GID`, avoiding permission issues on bind-mounted volumes.
- Before it drops privileges, the entrypoint makes sure `PAPERCLIP_HOME` is owned by the `node` user. Each boot checks only the home directory and its first three levels. The first boot (and any boot after a UID/GID change) also checks the full tree, and then writes `PAPERCLIP_HOME/.paperclip-ownership-verified`. While that marker is current, later boots skip the full-tree walk, which can take minutes on homes with many files. A mismatch triggers a recursive `chown`. If files deep in the tree get the wrong owner (for example, after a restore as root), delete the marker and restart to force a full check. If a check fails (for example, with an I/O error), the entrypoint does not write the marker, so the next boot checks again.
- The entrypoint writes timestamped lines when it starts, around the full ownership check or any repair (with the duration), and just before it starts the server, so you can measure the pre-start time from the container log.
- Paperclip data persists via Docker volumes/bind mounts (compose) or at `~/.local/share/paperclip` (quadlet).
## Native Runner build cache
The image compiles the native Runner once, in the `runner-artifacts-runnerd`
stage, with the package-owned Rust compiler, the release profile and locked
Cargo dependencies. That stage reads only the files Cargo reads.
Ordinary server, UI or Runner TypeScript changes can reuse the entire native
build through the existing registry cache (`mode=max`). A change to Rust
source, embedded protocol inputs, a dependency manifest, the Cargo lockfile or
the compiler compiles the daemon and its dependencies again. Each platform
gets its own native build; no cross-architecture binary is reused. No
additional GitHub Actions cache is created.
The `runner-artifacts-build` stage takes the build commit as an argument, so
every stamped build runs it: the frozen install, the Runner's TypeScript
build and the provider pack. Cargo finds the compiled daemon current there.
Cloud builds import one registry cache: the first available full-SHA cache in
the current commit's ten-entry first-parent ancestry, with the legacy cache
as a final fallback. Each build still exports its own SHA cache with
`mode=max`. In fresh-builder checks, importing several historical manifests
missed native layers that a single matching manifest reused. The selector
inspects metadata after Docker login, stops at the first available cache, and
permits a cold build if no cache can be read.
Rust input file times are normalized in both stages so fresh checkouts do not
force Cargo to rebuild unchanged source. The application build runs no Cargo
command. Cache misses only cost compilation time.