mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 20:44:16 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The container image sets the node user's home to the app home, `/paperclip`, which is the data volume in container deployments > - When `USER_UID`/`USER_GID` differ from the image, the entrypoint runs `usermod -u` and `usermod -g` before the app starts > - `usermod` also re-owns every file under the user's home, so every boot walks the whole data volume first > - On a large volume this delayed each boot by about six minutes, and platforms that stop the old container before starting the new one serve errors for that whole time > - This pull request runs the remap while the home points at a path that does not exist, then restores it > - The benefit is that a UID/GID remap takes milliseconds, and deploys no longer wait for a full volume walk ## Linked Issues or Issue Description Related: #15992 (skips the full-tree ownership probe on routine boots) and #10530 (ownership before the privilege drop). This removes the remaining per-boot full-tree walk. **What happened?** A deployment sets `USER_UID=1001`/`USER_GID=1001` on an image built with the default 1000. Every boot logs `Updating node UID to 1001`, and the next log line (`Updating node GID to 1001`) appeared six minutes later. During that time the app was not running. The volume holds many agent worktrees and dependency trees. No file had the old owner, so the walk changed nothing. **Expected behavior** Remapping the node account should not depend on the size of the data volume. Ownership repair belongs to the existing tiered ownership probe, which records a marker and skips the full walk on routine boots. **Steps to reproduce** 1. Build the image with the default `USER_UID=1000`. 2. Run it as root with `USER_UID=1001`, `USER_GID=1001` and a large app home. 3. Every boot spends the time of a full home walk in `usermod -o -u` before the app starts. Measured in the runtime image (Debian trixie) with 200,000 files under a user's home: `usermod -u` took 1,204 ms and re-owned every file. With the home moved away first, the three `usermod` calls took 124 ms and changed no file. ## What Changed - `scripts/docker-entrypoint.sh`: when a UID or GID remap is needed, read the node user's home with `getent passwd node` and point it at `/nonexistent/paperclip-uid-remap`. Then run the same UID and GID commands as before, and restore the home. Without a passwd entry, the remap runs as before. The ownership probe and repair logic are unchanged. After a real ID change, the home directory has the old owner, so the shallow probe detects the mismatch on the same boot and repairs the tree once. - `server/src/__tests__/docker-entrypoint.test.ts`: stubs `getent` and adds tests for the call order (UID+GID remap, GID-only remap), for repair after the remap, and for the fallback without a passwd entry. ## Verification - `pnpm exec vitest run src/__tests__/docker-entrypoint.test.ts` (in `server/`): 22 passed. - `sh -n scripts/docker-entrypoint.sh`: passed. - Manual check in the runtime image with a throwaway user and 200,000 files (numbers above). The UID changed, file owners were not touched, and the passwd home was restored. ## Risks - `usermod -d` without `-m` only edits the passwd entry. The home is restored right after the ID change. If the entrypoint fails between the two calls, `set -e` stops the container before the privilege drop, and the next boot sets the home again, so a partial state does not persist past a restart. - Ownership after a real remap now depends on the existing probe instead of `usermod`. The probe's tests already cover the remap, fresh-mount, deep-mismatch and marker cases. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs, live measurement in a running container). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
153 lines
6.5 KiB
Bash
153 lines
6.5 KiB
Bash
#!/bin/sh
|
|
set -e
|
|
|
|
# Timestamped progress lines make the pre-start phase (container start to
|
|
# the first app log line) measurable from the container log alone.
|
|
log() {
|
|
echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) docker-entrypoint.sh: $*"
|
|
}
|
|
|
|
log "starting"
|
|
|
|
# Capture runtime UID/GID from environment variables, defaulting to 1000
|
|
PUID=${USER_UID:-1000}
|
|
PGID=${USER_GID:-1000}
|
|
|
|
# Without root we can neither remap the node user (usermod/groupmod/chown)
|
|
# nor switch users (gosu needs CAP_SETUID/CAP_SETGID), so exec directly.
|
|
# This covers Kubernetes restricted PodSecurity (runAsNonRoot + runAsUser)
|
|
# as well as platforms that assign arbitrary UIDs (e.g. OpenShift); for the
|
|
# latter a UID/GID mismatch is unfixable here, so warn instead of letting
|
|
# usermod fail cryptically and keep volume-permission issues diagnosable.
|
|
if [ "$(id -u)" -ne 0 ]; then
|
|
if [ "$(id -u)" -ne "$PUID" ] || [ "$(id -g)" -ne "$PGID" ]; then
|
|
echo "docker-entrypoint.sh: running unprivileged as $(id -u):$(id -g); cannot remap to requested ${PUID}:${PGID}" >&2
|
|
fi
|
|
log "starting app as $(id -u):$(id -g)"
|
|
exec "$@"
|
|
fi
|
|
|
|
# Adjust the node user's UID/GID if they differ from the runtime request.
|
|
#
|
|
# `usermod -u` and `usermod -g` also re-own every file under the user's home
|
|
# directory. When that home is the app home (the data volume), the walk can
|
|
# take minutes on a large home, and the app cannot start until it ends. In
|
|
# that case the remap runs while the home points at a placeholder path that
|
|
# does not exist, so usermod changes only the account, and the ownership probe
|
|
# below repairs the app home when the IDs really changed (the home directory
|
|
# itself then has the old owner, which the shallow probe detects on the same
|
|
# boot). A passwd home outside the app home is not covered by that probe, so
|
|
# it keeps the plain usermod behavior, which re-owns it.
|
|
#
|
|
# The placeholder embeds the original home, so a boot that stops between the
|
|
# two usermod -d calls is repaired on the next start of the same container.
|
|
remap_home_prefix="/nonexistent/paperclip-uid-remap"
|
|
node_home="$(getent passwd node 2>/dev/null | cut -d: -f6 || true)"
|
|
case "$node_home" in
|
|
"$remap_home_prefix"/*)
|
|
node_home="${node_home#"$remap_home_prefix"}"
|
|
echo "Restoring node home to $node_home after an interrupted remap"
|
|
usermod -d "$node_home" node
|
|
;;
|
|
esac
|
|
|
|
remap_uid=0
|
|
remap_gid=0
|
|
[ "$(id -u node)" -ne "$PUID" ] && remap_uid=1
|
|
[ "$(id -g node)" -ne "$PGID" ] && remap_gid=1
|
|
app_home="${PAPERCLIP_HOME:-/paperclip}"
|
|
move_home=0
|
|
if { [ "$remap_uid" -eq 1 ] || [ "$remap_gid" -eq 1 ]; } \
|
|
&& [ -n "$node_home" ] && [ "${node_home%/}" = "${app_home%/}" ]; then
|
|
move_home=1
|
|
usermod -d "$remap_home_prefix$node_home" node
|
|
fi
|
|
|
|
if [ "$remap_uid" -eq 1 ]; then
|
|
echo "Updating node UID to $PUID"
|
|
usermod -o -u "$PUID" node
|
|
fi
|
|
|
|
if [ "$remap_gid" -eq 1 ]; then
|
|
echo "Updating node GID to $PGID"
|
|
groupmod -o -g "$PGID" node
|
|
usermod -g "$PGID" node
|
|
fi
|
|
|
|
if [ "$move_home" -eq 1 ]; then
|
|
usermod -d "$node_home" node
|
|
fi
|
|
|
|
# Ensure the app home is owned by the runtime user BEFORE dropping
|
|
# privileges -- not only after a UID/GID remap. A freshly mounted volume
|
|
# (Docker named volume, Railway volume, Kubernetes PV) arrives root-owned
|
|
# and shadows the image's build-time chown, so with the default UID the old
|
|
# remap-only condition dropped privileges onto an unwritable home and the
|
|
# server crashed on its first mkdir.
|
|
#
|
|
# A full-tree probe costs one stat per file. On a home with many agent
|
|
# worktrees and node_modules trees that is minutes of boot time, so the
|
|
# probe is tiered:
|
|
# 1. A shallow probe (the home and its first few levels) runs on every
|
|
# boot. It catches a fresh root-owned mount and a UID/GID remap, because
|
|
# both change the owner of the home directory itself.
|
|
# 2. A full-tree probe runs only when the marker file below is missing or
|
|
# records a different node UID/GID. It catches a mismatch deeper in the
|
|
# tree (init containers, backup restores, files written before a remap,
|
|
# the first boot after an upgrade from an older image).
|
|
# Any mismatch repairs the whole tree. After a clean probe or a repair, the
|
|
# marker records the verified node UID/GID, so later boots skip the full
|
|
# walk. Delete the marker to force a full probe on the next boot.
|
|
home_dir="${PAPERCLIP_HOME:-/paperclip}"
|
|
ownership_marker="$home_dir/.paperclip-ownership-verified"
|
|
ownership_shallow_depth=3
|
|
|
|
# Sets needs_repair=1 when any entry is not node:node, and probe_failed=1
|
|
# when find exits non-zero (for example, an I/O error), so an incomplete
|
|
# walk is never recorded as verified. The arguments bound the walk (empty
|
|
# for the full tree); -quit stops at the first mismatch.
|
|
probe_ownership() {
|
|
if ! first_mismatch="$(find "$home_dir" "$@" \( ! -user node -o ! -group node \) -print -quit 2>/dev/null)"; then
|
|
probe_failed=1
|
|
log "ownership probe of $home_dir failed; it will run again on the next boot"
|
|
fi
|
|
if [ -n "$first_mismatch" ]; then
|
|
needs_repair=1
|
|
fi
|
|
}
|
|
|
|
if [ -d "$home_dir" ]; then
|
|
expected_marker="v1 $(id -u node):$(id -g node)"
|
|
current_marker="$(cat "$ownership_marker" 2>/dev/null || true)"
|
|
needs_repair=0
|
|
probe_failed=0
|
|
probe_ownership -maxdepth "$ownership_shallow_depth"
|
|
if [ "$needs_repair" -eq 0 ] && [ "$current_marker" != "$expected_marker" ]; then
|
|
log "full ownership probe of $home_dir (no current marker)"
|
|
probe_started=$(date +%s)
|
|
probe_ownership
|
|
log "full ownership probe finished in $(( $(date +%s) - probe_started ))s"
|
|
fi
|
|
|
|
if [ "$needs_repair" -eq 1 ]; then
|
|
log "repairing ownership of $home_dir to node:node"
|
|
repair_started=$(date +%s)
|
|
chown -R node:node "$home_dir"
|
|
log "ownership repair finished in $(( $(date +%s) - repair_started ))s"
|
|
fi
|
|
|
|
# Record the marker only when the tree is known good: a successful
|
|
# recursive chown (set -e stops the script if it fails), or probes that
|
|
# all completed without a mismatch.
|
|
if [ "$current_marker" != "$expected_marker" ] && { [ "$needs_repair" -eq 1 ] || [ "$probe_failed" -eq 0 ]; }; then
|
|
# Write as node so the marker itself never trips the shallow probe
|
|
# (root-squashed network mounts included). Best effort: a read-only
|
|
# home only costs a full probe on the next boot.
|
|
# shellcheck disable=SC2016 # $1/$2 expand in the inner shell
|
|
gosu node sh -c 'printf "%s\n" "$1" > "$2"' sh "$expected_marker" "$ownership_marker" 2>/dev/null || true
|
|
fi
|
|
fi
|
|
|
|
log "starting app as node"
|
|
exec gosu node "$@"
|