mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 20:44:16 +02:00
2cc79923691848fea68c2ed0b611a9ce2c00bfaa
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandPaperclip
|
341cfbcea8 |
fix(docker): remap the node user without walking the app home (#16475)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The container image sets the node user's home to the app home, `/paperclip`, which is the data volume in container deployments > - When `USER_UID`/`USER_GID` differ from the image, the entrypoint runs `usermod -u` and `usermod -g` before the app starts > - `usermod` also re-owns every file under the user's home, so every boot walks the whole data volume first > - On a large volume this delayed each boot by about six minutes, and platforms that stop the old container before starting the new one serve errors for that whole time > - This pull request runs the remap while the home points at a path that does not exist, then restores it > - The benefit is that a UID/GID remap takes milliseconds, and deploys no longer wait for a full volume walk ## Linked Issues or Issue Description Related: #15992 (skips the full-tree ownership probe on routine boots) and #10530 (ownership before the privilege drop). This removes the remaining per-boot full-tree walk. **What happened?** A deployment sets `USER_UID=1001`/`USER_GID=1001` on an image built with the default 1000. Every boot logs `Updating node UID to 1001`, and the next log line (`Updating node GID to 1001`) appeared six minutes later. During that time the app was not running. The volume holds many agent worktrees and dependency trees. No file had the old owner, so the walk changed nothing. **Expected behavior** Remapping the node account should not depend on the size of the data volume. Ownership repair belongs to the existing tiered ownership probe, which records a marker and skips the full walk on routine boots. **Steps to reproduce** 1. Build the image with the default `USER_UID=1000`. 2. Run it as root with `USER_UID=1001`, `USER_GID=1001` and a large app home. 3. Every boot spends the time of a full home walk in `usermod -o -u` before the app starts. Measured in the runtime image (Debian trixie) with 200,000 files under a user's home: `usermod -u` took 1,204 ms and re-owned every file. With the home moved away first, the three `usermod` calls took 124 ms and changed no file. ## What Changed - `scripts/docker-entrypoint.sh`: when a UID or GID remap is needed, read the node user's home with `getent passwd node` and point it at `/nonexistent/paperclip-uid-remap`. Then run the same UID and GID commands as before, and restore the home. Without a passwd entry, the remap runs as before. The ownership probe and repair logic are unchanged. After a real ID change, the home directory has the old owner, so the shallow probe detects the mismatch on the same boot and repairs the tree once. - `server/src/__tests__/docker-entrypoint.test.ts`: stubs `getent` and adds tests for the call order (UID+GID remap, GID-only remap), for repair after the remap, and for the fallback without a passwd entry. ## Verification - `pnpm exec vitest run src/__tests__/docker-entrypoint.test.ts` (in `server/`): 22 passed. - `sh -n scripts/docker-entrypoint.sh`: passed. - Manual check in the runtime image with a throwaway user and 200,000 files (numbers above). The UID changed, file owners were not touched, and the passwd home was restored. ## Risks - `usermod -d` without `-m` only edits the passwd entry. The home is restored right after the ID change. If the entrypoint fails between the two calls, `set -e` stops the container before the privilege drop, and the next boot sets the home again, so a partial state does not persist past a restart. - Ownership after a real remap now depends on the existing probe instead of `usermod`. The probe's tests already cover the remap, fresh-mount, deep-mismatch and marker cases. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs, live measurement in a running container). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
4f399cec9f |
fix(docker): skip the full-tree ownership walk on routine boots (#15992)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Docker image keeps all instance state under `PAPERCLIP_HOME`,
and deployments mount a volume there
> - Before it drops privileges, the entrypoint makes sure that the
`node` user owns the home (#10530)
> - The ownership probe is a first-mismatch `find` over the whole tree.
When ownership is correct, `-quit` never fires, so every boot stats
every file on the volume
> - On homes with many agent worktrees and `node_modules` trees, this
walk takes minutes before Node starts. That is close to common 300s
healthcheck windows
> - This pull request keeps a cheap shallow probe on every boot. It runs
the full-tree probe only once per volume and node UID/GID, and records
the result in a marker file
> - The benefit is that routine boots no longer scale with the file
count, and the original root-owned-volume fix stays in place
## Linked Issues or Issue Description
Refs #10431 (the open follow-up about the remaining cost of the
ownership walk). Related: #10530 (the change that added the full-tree
probe).
**What happened?**
On a hosted multi-instance deployment, we measured the time from
container start to the first app log line. Instances with many files on
the volume (agent worktrees with `node_modules`) took about 190-235s.
Small instances took 5-22s. One 30 GB volume with few files took 24s, so
the cost follows the file count, not the volume size. Almost all of this
time is the `find "$home_dir" \( ! -user node -o ! -group node \) -print
-quit` walk in `scripts/docker-entrypoint.sh`. When ownership is already
correct, `-quit` never fires, so the walk stats every inode on every
boot.
**Expected behavior**
When ownership is already correct, the entrypoint pre-start phase is
short and does not depend on the number of files in `PAPERCLIP_HOME`. A
fresh root-owned volume, a UID/GID remap, and an upgrade from an older
image are still repaired before the privilege drop.
**Steps to reproduce**
1. Run the image with a volume on `/paperclip` that holds a few hundred
thousand files, all owned by `node:node`.
2. Restart the container.
3. Compare the container start time with the first server log line. The
gap grows with the file count, even though no `chown` runs.
**Paperclip version or commit**
`master` at
|
||
|
Devin Foley
|
dd1a7f5290 |
Ensure app-home ownership before the privilege drop, not only on remap (#10530)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Docker image persists all instance state (project checkouts, worktrees, run logs, uploads) under `PAPERCLIP_HOME`, and deployments mount a volume there for durability > - The entrypoint starts as root and drops privileges to the `node` user, but it fixes `PAPERCLIP_HOME` ownership only when it remaps the user's UID/GID > - A freshly mounted volume arrives root-owned and shadows the image's build-time `chown`, so a default-UID boot drops privileges onto an unwritable home and the server crashes on its first `mkdir` > - This pull request makes the entrypoint probe the home's ownership and chown whenever it does not match the runtime user, before the privilege drop > - The benefit is that the image works out of the box on any platform-managed volume, with the common already-correct boot staying chown-free ## Linked Issues or Issue Description No public issue exists — describing the bug inline (per the bug report template). **What happened?** Running the image with a freshly created volume mounted at `/paperclip` (a Docker named volume, a Kubernetes PV, or any platform-managed volume) and the default `USER_UID`/`USER_GID` crashes on boot: `Error: EACCES: permission denied, mkdir '/paperclip/instances/default/logs'`. **Expected behavior** The container boots and initializes its instance tree on the mounted volume, exactly as it does when `/paperclip` is the image's own (build-time chowned) directory. **Steps to reproduce** 1. `docker volume create paperclip-data` 2. `docker run -v paperclip-data:/paperclip ghcr.io/paperclipai/paperclip:<any current tag>` 3. Observe the EACCES crash on the first `mkdir` under `/paperclip`. **Root cause** `scripts/docker-entrypoint.sh` chowns `/paperclip` only inside its UID/GID remap branch (`changed=1`). A fresh volume mount is root-owned and shadows the image's build-time `chown node:node /paperclip`; with the default 1000:1000 no remap happens, so no chown happens, and `gosu node` drops onto an unwritable home. **Paperclip version or commit:** reproduces on `master` and any published image. **Deployment mode:** any; observed on managed-cloud volume mounts and reproducible with plain Docker named volumes. **Installation method:** Docker image (`ghcr.io/paperclipai/paperclip`). **Related PRs (dedup search):** no open or merged PR touches the entrypoint ownership logic; the entrypoint's privilege-handling tests were added previously and this extends them. No duplicate found. ## What Changed - `scripts/docker-entrypoint.sh`: the remap-conditional `chown` is replaced by an ownership probe — after any UID/GID remap, the entrypoint stats `PAPERCLIP_HOME` (default `/paperclip`) and runs `chown -R node:node` only when the owner does not match the runtime user, before `exec gosu node`. Covers fresh root-owned mounts and trees written under a previous UID mapping; the already-correct boot performs no chown. The unprivileged (non-root start) branch is unchanged. - `server/src/__tests__/docker-entrypoint.test.ts`: `stat` stub added to the harness; new cases for the fresh root-owned mount with default UID/GID and for `PAPERCLIP_HOME`-relative probing; the remap case now models the post-remap ownership mismatch. ## Verification - `pnpm vitest run server/src/__tests__/docker-entrypoint.test.ts` — 7 passed (5 existing behaviors unchanged, 2 new). - Live on a managed deployment: a container that crash-looped with the EACCES above boots cleanly once the home is chowned before the drop (the same effect this entrypoint change produces; forced there by a UID remap as an interim workaround). ## Risks - Low. Behavior changes only for boots where `PAPERCLIP_HOME` exists with mismatched ownership — exactly the boots that crash today. `chown -R` on a large previously-mismatched tree adds one-time boot latency; correctly-owned homes skip it entirely. Kubernetes restricted / OpenShift non-root starts keep the existing exec-directly path untouched. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic; Claude Code CLI with extended thinking and tool use; tests executed locally via Vitest). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicates; extends the existing entrypoint privilege tests) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Jannes StubbemannandClaude Opus 4.6
|
c21f70ef1c |
fix: skip gosu when already running as target user (#2908)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The reference container image must be deployable on both Docker Compose (where it starts as root and `gosu`'s a `USER_UID`/`USER_GID` switch) and Kubernetes (where the pod is typically constrained by PodSecurity) > - The Kubernetes operator (paperclipinc/paperclip-operator#45) sets `runAsNonRoot: true`, `runAsUser: 1000`, `allowPrivilegeEscalation: false`, and `drop: ALL` capabilities by default — the unconditional `usermod` + `gosu` flow in the entrypoint requires root + `CAP_SETUID` / `CAP_SETGID`, making the image undeployable on any cluster enforcing baseline or restricted PodSecurity > - Without root, neither the user remap nor `gosu` can ever succeed — so the fix is a runtime branch: non-root starts `exec` the command directly (warning if the runtime UID/GID differs from the requested one), while root starts keep the existing `usermod`+`gosu` flow > - This also covers platforms that assign arbitrary UIDs (OpenShift restricted SCC), which previously crashed with a cryptic `usermod: Permission denied` > - The benefit is one image that works for both deployment shapes with no operator-side workaround — pure superset, no breaking change ## Linked Issues or Issue Description Refs paperclipinc/paperclip-operator#45 (cross-repo) — the operator's default pod security context (`runAsNonRoot: true`, `runAsUser: 1000`, `allowPrivilegeEscalation: false`, `drop: ALL`) is blocked by this entrypoint behavior. The operator shipped a stopgap (paperclipinc/paperclip-operator#46 lets the CRD override the security context); this PR is the image-side fix that makes the secure defaults work out of the box. Supersedes #2904 (v1 of this branch). No in-repo issue covers this directly — problem described in-PR following the bug-report template: **What happened** The entrypoint unconditionally runs `usermod`/`groupmod`/`chown` + `exec gosu node`, which requires root plus `CAP_SETUID` / `CAP_SETGID` — any non-root start crashes (`gosu` cannot drop privileges; a mismatched UID dies earlier at `usermod: Permission denied`), making the reference image undeployable on clusters enforcing baseline or restricted PodSecurity. **Expected behavior** A non-root container `exec`s the command directly (with a clear warning if its UID/GID differs from the requested `USER_UID`/`USER_GID`, since a remap is impossible without root). The existing root + `usermod` + `gosu` flow is preserved for Docker Compose, where the container starts as root and switches to the requested UID/GID. **Deployment mode** Kubernetes with baseline/restricted PodSecurity and OpenShift-style arbitrary-UID platforms (failing cases); Docker Compose root-start (must keep working). ## What Changed - **`scripts/docker-entrypoint.sh`** — branch on the runtime UID: - **Non-root start** → `exec "$@"` directly. If the runtime UID/GID differs from `USER_UID`/`USER_GID`, print a one-line warning to stderr first (the remap is impossible without root; the warning keeps volume-permission mismatches diagnosable instead of failing cryptically inside `usermod`). - **Root start** → unchanged: `usermod`/`groupmod` remap when requested, `chown` of `/paperclip` when a remap happened, then `exec gosu node "$@"`. ## Verification **Automated:** `server/src/__tests__/docker-entrypoint.test.ts` runs the real entrypoint with `id`/`usermod`/`groupmod`/`chown`/`gosu` stubbed via PATH and asserts all five privilege branches (root+defaults, root+remap, non-root match, arbitrary non-root UID, GID mismatch) — runs in the regular server suite, no Docker needed. **Manual (Docker):** ran the entrypoint on `node:lts-trixie-slim` (the image's actual base) across the full matrix, with `gosu` stubbed to a marker: - [x] Root start, defaults → no remap, `gosu node` invoked (Docker Compose flow unchanged) - [x] Root start, `USER_UID=1001`/`USER_GID=1001` → `Updating node UID/GID to 1001` + `gosu node` (remap flow unchanged) - [x] Non-root `--user 1000:1000` (the operator's `runAsUser: 1000` shape) → silent direct exec, command runs as 1000:1000 - [x] Non-root `--user 1234:1234` (arbitrary UID) → warning `running unprivileged as 1234:1234; cannot remap to requested 1000:1000`, then direct exec (previously: crash) - [x] Non-root `--user 1000:1001` (GID mismatch) → warning, then direct exec - [x] Baseline check: master's entrypoint fails for any non-root start (gosu/usermod require root) End-to-end cluster verification under restricted PodSecurity exercises the same branch as the `--user 1000:1000` case above; the operator repo's deploy is the natural place for that smoke test once this ships in an image tag. ## Risks - **Backward-compatible.** Docker Compose / root-entrypoint path is byte-for-byte the same flow — `usermod`/`gosu` runs whenever the container starts as root. - **Behavior change only for previously-broken starts.** Non-root containers used to crash; they now run. The only observable difference for a *working* deployment is none. - **Mismatched non-root UID/GID warns instead of failing.** Deliberate: the remap is impossible without root, and arbitrary-UID platforms (OpenShift) rely on group-writable volumes; a hard fail would keep them broken. The stderr warning preserves diagnosability. - **No new env vars, no API surface.** Pure entrypoint behavior change gated on the runtime UID. - **Restricted PodSecurity ready.** The non-root branch needs no Linux capabilities — works under `drop: ALL`. ## Model Used Claude Opus 4.6; rebase, non-root generalization, and verification matrix by Claude Fable 5 (1M context). ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Checked ROADMAP.md — not in conflict with planned core work (agent-runtime sandbox images use `tini`, no gosu — unaffected) - [x] Tests run locally and pass (new `docker-entrypoint.test.ts` covering all five privilege branches, plus a manual Docker matrix on the real base image; see Verification) - [x] No UI changes - [x] Documented risks above - [x] Will address all Greptile and reviewer comments before merge Unblocks the default (non-overridden) security context of paperclipinc/paperclip-operator#45 / #46. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Cody (Radius Red)andPaperclip
|
420cd4fd8d |
chore(docker): improve base image and organize docker files
- Add wget, ripgrep, python3, and GitHub CLI (gh) to base image - Add OPENCODE_ALLOW_ALL_MODELS=true to production ENV - Move compose files, onboard-smoke Dockerfile to docker/ - Move entrypoint script to scripts/docker-entrypoint.sh - Add Podman Quadlet unit files (pod, app, db containers) - Add docker/README.md with build, compose, and quadlet docs - Add scripts/docker-build-test.sh for local build validation - Update all doc references for new file locations - Keep main Dockerfile at project root (no .dockerignore changes needed) Co-Authored-By: Paperclip <noreply@paperclip.ing> |