Files
PaperClipAI/scripts/release.sh
T
fc5a30805e feat(cli): add managed install, update, and service lifecycle (#10045)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Operators need a predictable installation path that survives beyond
an ephemeral `npx` process
> - A durable installation needs an owned per-user payload store, stable
command shim, safe shell integration, and supported service lifecycle
> - Updates must preserve recoverability by backing up data, installing
side-by-side, verifying the new payload, and retaining rollback state
> - Bootstrap scripts and privileged service operations must fail closed
across download, filesystem, ownership, and consent boundaries
> - This pull request integrates managed install, update, rollback,
service, uninstall, doctor, bootstrap-installer, and runtime-serving
support into one workflow
> - The benefit is a recoverable, inspectable, and documented
installation lifecycle with explicit safety boundaries across Linux,
macOS, containers, WSL, npm, npx, and source checkouts

## Linked Issues or Issue Description

### Problem

Paperclip lacks a first-class durable installation and lifecycle
workflow. Operators currently have to assemble npm/npx installation,
PATH setup, background-service management, updates, rollback,
diagnostics, and uninstall behavior themselves. That makes upgrades
harder to recover, creates inconsistent behavior across platforms, and
leaves shell/download/service trust boundaries without one documented
implementation.

### Proposed Solution

Add a managed per-user install store and stable shim, a verified shell
bootstrap installer, service lifecycle commands, install-mode-aware
update/rollback behavior, doctor checks, and documentation. Managed
updates back up the database, install and smoke-test a side-by-side
payload, atomically switch `current`, and retain prior payloads. The
shell installer pins registry/download trust boundaries and requires
explicit consent for non-interactive privileged actions.

### Alternatives Considered

- Keep recommending `npx`: simple for evaluation, but ephemeral and
unsuitable for stable services, atomic updates, or rollback.
- Require global npm installation only: familiar, but cannot provide the
owned side-by-side payload store and retained rollback semantics.
- Split the capability across multiple PRs: rejected because install,
update, service, uninstall, bootstrap, and serving behavior share
contracts and security boundaries that need review together.

### Related Pull Requests

- Supersedes #10042 and #10044 with one integrated final diff.
- Incorporates and replaces the closed preparatory work in #10032 and
#10034.

## What Changed

- Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall,
service lifecycle, onboarding integration, and managed-install doctor
checks.
- Added a private managed payload store, verified manifest/marker
ownership, exclusive mutation locks, atomic manifest/current/shim
writes, retained previous payloads, and provenance validation.
- Added npm and GitHub-ref install sources with exact target resolution,
registry isolation, database backup, side-by-side verification, atomic
activation, service restart coordination, and failure rollback.
- Made managed-update backups report actionable service-start and
`--no-backup` recovery guidance for unreachable databases, while clean
never-onboarded instances skip an empty backup.
- Added systemd user and launchd service definitions, status/health/log
commands, single-instance coordination, stale-port recovery, and
explicit sudo/lingering consent handling.
- Added the `scripts/install.sh` bootstrap path with checked two-stage
downloads, pinned public npm registry usage, platform checks,
dry-run/non-interactive controls, and Docker fixtures.
- Added embedded Postgres/native bootstrap integration,
hot-restart/systemd-notify serving support, passive update notices,
configuration contracts, README/CLI/install documentation, and focused
regression tests.
- Security re-review should explicitly re-verify: (1)
`addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or
non-regular rc files, assert current-user ownership, preserve
restrictive modes, and replace atomically; (2) managed shim replacement
rejects unsafe parents, foreign-owned or multiply linked files, and uses
checked atomic replacement; (3) the shell installer and sudo path
preserve explicit consent and checked downloads; and (4) installed
service/runtime serving remains bound to the validated managed shim and
instance configuration.

## Verification

- `bash -n scripts/install.sh scripts/clean-install-git.sh
scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh`
- `pnpm exec vitest run cli/src/__tests__/install-store.test.ts
cli/src/__tests__/install-command.test.ts
cli/src/__tests__/managed-install-check.test.ts
cli/src/__tests__/onboard-service.test.ts
cli/src/__tests__/service-health-check.test.ts
cli/src/__tests__/service-manager.test.ts
cli/src/__tests__/update-command.test.ts
cli/src/__tests__/update-notice.test.ts
packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests
passed
- `pnpm --dir cli typecheck`
- `pnpm --dir cli build`
- Follow-up verification: `pnpm exec vitest run
cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli
typecheck`, `pnpm --dir cli build`, and `pnpm --filter
@paperclipai/server typecheck`.
- `pnpm -r typecheck`
- `pnpm build`
- Full `pnpm test:run` exercised all suites; an injected static AWS
credential changed one unrelated doctor expectation, which passed when
those credentials were removed. A second run cleared that case and
exposed stale pre-existing adapter-utils `dist` output; rebuilding
`@paperclipai/adapter-utils` made the isolated test pass. The updated PR
CI is the authoritative clean-workspace full-suite run.

## Risks

- Installer/update code writes executable shims, symlinks, shell rc
blocks, service definitions, and managed payloads; ownership,
regular-file, symlink, hard-link, marker, and path-containment checks
fail closed before destructive changes.
- The bootstrap installer executes downloaded tooling; downloads are
staged and checked before execution, npm traffic is pinned to the public
registry, and non-interactive privileged behavior requires explicit
consent.
- Linux lingering may invoke `sudo`; the command is surfaced and
confirmed before execution, and unsupported service managers fall back
to foreground-run guidance.
- Database migrations remain forward-only; payload rollback does not
reverse migrations, so managed updates create a backup before activation
unless explicitly disabled.
- Service restart and runtime serving touch process/port ownership;
lifecycle locks, health/version checks, and stable-shim service
definitions reduce split-brain and stale-process risk.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with
reasoning, repository/API access, shell execution, and test tooling. The
runtime did not expose a reliable context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:52:23 -07:00

372 lines
13 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
# shellcheck source=./release-lib.sh
. "$REPO_ROOT/scripts/release-lib.sh"
CLI_DIR="$REPO_ROOT/cli"
channel=""
release_date=""
dry_run=false
skip_verify=false
print_version_only=false
tag_name=""
cleanup_on_exit=false
usage() {
cat <<'EOF'
Usage:
./scripts/release.sh <canary|stable> [--date YYYY-MM-DD] [--dry-run] [--skip-verify] [--print-version]
Examples:
./scripts/release.sh canary
./scripts/release.sh canary --date 2026-03-17 --dry-run
./scripts/release.sh stable
./scripts/release.sh stable --date 2026-03-17 --dry-run
./scripts/release.sh stable --date 2026-03-18 --print-version
Notes:
- Stable versions use YYYY.MDD.P, where M is the UTC month, DD is the
zero-padded UTC day, and P is the same-day stable patch slot.
- Canary releases publish YYYY.MDD.P-canary.N under the npm dist-tag
"canary" and create the git tag canary/vYYYY.MDD.P-canary.N.
- Stable releases publish YYYY.MDD.P under the npm dist-tag "latest" and
create the git tag vYYYY.MDD.P.
- Non-dry-run stable release notes must already exist at releases/vYYYY.MDD.P.md.
- The script rewrites versions temporarily and restores the working tree on
exit. Tags always point at the original source commit, not a generated
release commit.
EOF
}
restore_publish_artifacts() {
if [ -f "$CLI_DIR/package.dev.json" ]; then
mv "$CLI_DIR/package.dev.json" "$CLI_DIR/package.json"
fi
rm -f "$CLI_DIR/README.md"
rm -rf "$REPO_ROOT/server/ui-dist"
for pkg_dir in server packages/adapters/claude-local packages/adapters/codex-local; do
rm -rf "$REPO_ROOT/$pkg_dir/skills"
done
}
cleanup_release_state() {
restore_publish_artifacts
tracked_changes="$(git -C "$REPO_ROOT" diff --name-only; git -C "$REPO_ROOT" diff --cached --name-only)"
if [ -n "$tracked_changes" ]; then
printf '%s\n' "$tracked_changes" | sort -u | while IFS= read -r path; do
[ -z "$path" ] && continue
git -C "$REPO_ROOT" checkout -q HEAD -- "$path" || true
done
fi
untracked_changes="$(git -C "$REPO_ROOT" ls-files --others --exclude-standard)"
if [ -n "$untracked_changes" ]; then
printf '%s\n' "$untracked_changes" | while IFS= read -r path; do
[ -z "$path" ] && continue
if [ -d "$REPO_ROOT/$path" ]; then
rm -rf "$REPO_ROOT/$path"
else
rm -f "$REPO_ROOT/$path"
fi
done
fi
}
set_cleanup_trap() {
cleanup_on_exit=true
trap cleanup_release_state EXIT
}
while [ $# -gt 0 ]; do
case "$1" in
canary|stable)
if [ -n "$channel" ]; then
release_fail "only one release channel may be provided."
fi
channel="$1"
;;
--date)
shift
[ $# -gt 0 ] || release_fail "--date requires YYYY-MM-DD."
release_date="$1"
;;
--dry-run) dry_run=true ;;
--skip-verify) skip_verify=true ;;
--print-version) print_version_only=true ;;
-h|--help)
usage
exit 0
;;
*)
release_fail "unexpected argument: $1"
;;
esac
shift
done
[ -n "$channel" ] || {
usage
exit 1
}
PUBLISH_REMOTE="$(resolve_release_remote)"
fetch_release_remote "$PUBLISH_REMOTE"
CURRENT_BRANCH="$(git_current_branch)"
CURRENT_SHA="$(git -C "$REPO_ROOT" rev-parse HEAD)"
LAST_STABLE_TAG="$(get_last_stable_tag)"
CURRENT_STABLE_VERSION="$(get_current_stable_version)"
RELEASE_DATE="${release_date:-$(utc_date_iso)}"
PUBLIC_PACKAGE_INFO="$(list_public_package_info)"
PUBLIC_PACKAGE_NAMES=()
while IFS= read -r package_name; do
[ -n "$package_name" ] || continue
PUBLIC_PACKAGE_NAMES+=("$package_name")
done < <(printf '%s\n' "$PUBLIC_PACKAGE_INFO" | cut -f2)
[ -n "$PUBLIC_PACKAGE_INFO" ] || release_fail "no public packages were found in the workspace."
# Pre-fetch published versions for every public package in parallel so the
# version helpers below do not each issue one serial `npm view` call per
# package (see scripts/release-registry-versions.mjs).
RELEASE_PACKAGE_VERSIONS_FILE="$(mktemp)"
export RELEASE_PACKAGE_VERSIONS_FILE
node "$REPO_ROOT/scripts/release-registry-versions.mjs" fetch "${PUBLIC_PACKAGE_NAMES[@]}" > "$RELEASE_PACKAGE_VERSIONS_FILE"
TARGET_STABLE_VERSION="$(next_stable_version "$RELEASE_DATE" "${PUBLIC_PACKAGE_NAMES[@]}")"
TARGET_PUBLISH_VERSION="$TARGET_STABLE_VERSION"
DIST_TAG="latest"
if [ "$channel" = "canary" ]; then
require_on_master_branch
TARGET_PUBLISH_VERSION="$(next_canary_version "$TARGET_STABLE_VERSION" "${PUBLIC_PACKAGE_NAMES[@]}")"
DIST_TAG="canary"
tag_name="$(canary_tag_name "$TARGET_PUBLISH_VERSION")"
else
tag_name="$(stable_tag_name "$TARGET_STABLE_VERSION")"
fi
rm -f "$RELEASE_PACKAGE_VERSIONS_FILE"
unset RELEASE_PACKAGE_VERSIONS_FILE
if [ "$print_version_only" = true ]; then
printf '%s\n' "$TARGET_PUBLISH_VERSION"
exit 0
fi
NOTES_FILE="$(release_notes_file "$TARGET_STABLE_VERSION")"
require_clean_worktree
require_npm_publish_auth "$dry_run"
if [ "$channel" = "stable" ] && [ "$dry_run" = false ] && [ ! -f "$NOTES_FILE" ]; then
release_fail "stable release notes file is required at $NOTES_FILE before publishing stable."
fi
if [ "$channel" = "canary" ] && [ -f "$NOTES_FILE" ]; then
release_info " ✓ Stable release notes already exist at $NOTES_FILE"
fi
if git_local_tag_exists "$tag_name" || git_remote_tag_exists "$tag_name" "$PUBLISH_REMOTE"; then
release_fail "git tag $tag_name already exists locally or on $PUBLISH_REMOTE."
fi
# Fresh (non-cached) existence check, batched in parallel. Prints the
# offending package@version pairs itself before failing.
node "$REPO_ROOT/scripts/release-registry-versions.mjs" assert-absent "$TARGET_PUBLISH_VERSION" "${PUBLIC_PACKAGE_NAMES[@]}" \
|| release_fail "npm version ${TARGET_PUBLISH_VERSION} already exists for one or more packages."
release_info ""
release_info "==> Release plan"
release_info " Remote: $PUBLISH_REMOTE"
release_info " Channel: $channel"
release_info " Current branch: ${CURRENT_BRANCH:-<detached>}"
release_info " Source commit: $CURRENT_SHA"
release_info " Last stable tag: ${LAST_STABLE_TAG:-<none>}"
release_info " Current stable version: $CURRENT_STABLE_VERSION"
release_info " Release date (UTC): $RELEASE_DATE"
release_info " Target stable version: $TARGET_STABLE_VERSION"
if [ "$channel" = "canary" ]; then
release_info " Canary version: $TARGET_PUBLISH_VERSION"
else
release_info " Stable version: $TARGET_PUBLISH_VERSION"
fi
release_info " Dist-tag: $DIST_TAG"
release_info " Git tag: $tag_name"
if [ "$channel" = "stable" ]; then
release_info " Release notes: $NOTES_FILE"
fi
set_cleanup_trap
# The release flow already prepares ui/dist before packaging. Reuse that output
# so server prepack does not rebuild the UI a second time during preview/publish.
export PAPERCLIP_RELEASE_REUSE_UI_DIST=1
if [ "$skip_verify" = false ]; then
release_info ""
release_info "==> Step 1/7: Verification gate..."
cd "$REPO_ROOT"
pnpm -r typecheck
pnpm test:run
pnpm build
else
release_info ""
release_info "==> Step 1/7: Verification gate skipped (--skip-verify)"
fi
release_info ""
release_info "==> Step 2/7: Building workspace artifacts..."
cd "$REPO_ROOT"
pnpm build
node "$REPO_ROOT/scripts/build-standalone-public-packages.mjs"
bash "$REPO_ROOT/scripts/prepare-server-ui-dist.sh"
for pkg_dir in server packages/adapters/claude-local packages/adapters/codex-local; do
rm -rf "$REPO_ROOT/$pkg_dir/skills"
cp -r "$REPO_ROOT/skills" "$REPO_ROOT/$pkg_dir/skills"
done
release_info " ✓ Workspace build complete"
release_info ""
release_info "==> Step 3/7: Rewriting workspace versions..."
set_public_package_version "$TARGET_PUBLISH_VERSION"
release_info " ✓ Versioned workspace to $TARGET_PUBLISH_VERSION"
release_info ""
release_info "==> Step 4/7: Building publishable CLI bundle..."
"$REPO_ROOT/scripts/build-npm.sh" --skip-checks --skip-typecheck
release_info " ✓ CLI bundle ready"
VERSIONED_PACKAGE_INFO="$(list_public_package_info)"
VERSION_IN_CLI_PACKAGE="$(node -e "console.log(require('$CLI_DIR/package.json').version)")"
if [ "$VERSION_IN_CLI_PACKAGE" != "$TARGET_PUBLISH_VERSION" ]; then
release_fail "versioning drift detected. Expected $TARGET_PUBLISH_VERSION but found $VERSION_IN_CLI_PACKAGE."
fi
VERIFY_ATTEMPTS="${NPM_PUBLISH_VERIFY_ATTEMPTS:-12}"
VERIFY_DELAY_SECONDS="${NPM_PUBLISH_VERIFY_DELAY_SECONDS:-5}"
release_info ""
if [ "$dry_run" = true ]; then
release_info "==> Step 5/7: Previewing publish payloads (--dry-run)..."
while IFS=$'\t' read -r pkg_dir _pkg_name _pkg_version; do
[ -z "$pkg_dir" ] && continue
release_info " --- $pkg_dir ---"
cd "$REPO_ROOT/$pkg_dir"
publish_tool="$(package_publish_tool)"
if [ "$publish_tool" = "npm" ]; then
publish_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-release-package.XXXXXX")"
node "$REPO_ROOT/scripts/prepare-bundled-package.mjs" "$REPO_ROOT/$pkg_dir" "$publish_dir"
cd "$publish_dir"
run_bundled_npm_pack pack --pack-destination "$publish_dir" 2>&1 | tail -3
rm -rf "$publish_dir"
else
pnpm publish --dry-run --no-git-checks --tag "$DIST_TAG" 2>&1 | tail -3
fi
done <<< "$VERSIONED_PACKAGE_INFO"
release_info " [dry-run] Would create git tag $tag_name on $CURRENT_SHA"
else
release_info "==> Step 5/7: Publishing packages to npm..."
while IFS=$'\t' read -r pkg_dir pkg_name pkg_version; do
[ -z "$pkg_dir" ] && continue
release_info " Publishing $pkg_name@$pkg_version"
cd "$REPO_ROOT/$pkg_dir"
publish_tool="$(package_publish_tool)"
if [ "$publish_tool" = "npm" ]; then
publish_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-release-package.XXXXXX")"
node "$REPO_ROOT/scripts/prepare-bundled-package.mjs" "$REPO_ROOT/$pkg_dir" "$publish_dir"
cd "$publish_dir"
fi
if ! publish_package_to_npm_and_wait \
"$DIST_TAG" \
"$pkg_name" \
"$pkg_version" \
"$publish_tool" \
"$VERIFY_ATTEMPTS" \
"$VERIFY_DELAY_SECONDS"; then
if [ "$publish_tool" = "npm" ]; then
rm -rf "$publish_dir"
fi
release_fail "stopping release: npm did not publish and expose ${pkg_name}@${pkg_version}"
fi
if [ "$publish_tool" = "npm" ]; then
rm -rf "$publish_dir"
fi
release_info " ✓ Published version is registry-visible"
done <<< "$VERSIONED_PACKAGE_INFO"
release_info " ✓ Published the full package set under dist-tag $DIST_TAG"
fi
release_info ""
if [ "$dry_run" = true ]; then
release_info "==> Step 6/7: Skipping npm verification in dry-run mode..."
else
release_info "==> Step 6/7: Confirming npm package availability and dist-tag integrity..."
REGISTRY_STATE_VERIFY_ATTEMPTS="${NPM_REGISTRY_STATE_VERIFY_ATTEMPTS:-12}"
REGISTRY_STATE_VERIFY_DELAY_SECONDS="${NPM_REGISTRY_STATE_VERIFY_DELAY_SECONDS:-5}"
release_info " ✓ Every version was registry-visible before the next package publish"
verify_args=(
--channel "$channel"
--dist-tag "$DIST_TAG"
--target-version "$TARGET_PUBLISH_VERSION"
)
while IFS=$'\t' read -r _pkg_dir pkg_name _pkg_version; do
[ -z "$pkg_name" ] && continue
verify_args+=(--package "$pkg_name")
done <<< "$VERSIONED_PACKAGE_INFO"
release_info " Waiting for npm dist-tags and package metadata to converge..."
if wait_for_release_registry_state \
"$REGISTRY_STATE_VERIFY_ATTEMPTS" \
"$REGISTRY_STATE_VERIFY_DELAY_SECONDS" \
"${verify_args[@]}"; then
:
else
verify_status=$?
if [ "$verify_status" -eq 2 ]; then
release_fail "publish completed, but registry verification failed immediately for ${TARGET_PUBLISH_VERSION}; dist-tag state is wrong or requires operator intervention"
fi
release_fail "publish completed, but npm dist-tags or registry metadata never converged for ${TARGET_PUBLISH_VERSION}"
fi
release_info " Installing paperclipai@$DIST_TAG into a clean prefix..."
if ! verify_npm_installable "paperclipai@$DIST_TAG" "$TARGET_PUBLISH_VERSION"; then
release_fail "paperclipai@$DIST_TAG did not install cleanly at expected version ${TARGET_PUBLISH_VERSION}"
fi
release_info " ✓ Clean-prefix install resolved ${TARGET_PUBLISH_VERSION}"
fi
release_info ""
if [ "$dry_run" = true ]; then
release_info "==> Step 7/7: Dry run complete..."
else
release_info "==> Step 7/7: Creating git tag..."
git -C "$REPO_ROOT" tag "$tag_name" "$CURRENT_SHA"
release_info " ✓ Created tag $tag_name on $CURRENT_SHA"
fi
release_info ""
if [ "$dry_run" = true ]; then
release_info "Dry run complete for $channel ${TARGET_PUBLISH_VERSION}."
else
if [ "$channel" = "canary" ]; then
release_info "Published canary ${TARGET_PUBLISH_VERSION}."
release_info "Install with: npx paperclipai@canary onboard"
release_info "Next step: git push ${PUBLISH_REMOTE} refs/tags/${tag_name}"
else
release_info "Published stable ${TARGET_PUBLISH_VERSION}."
release_info "Next steps:"
release_info " git push ${PUBLISH_REMOTE} refs/tags/${tag_name}"
release_info " ./scripts/create-github-release.sh $TARGET_STABLE_VERSION"
fi
fi