Files
PaperClipAI/scripts/acpx-patch-packaging.test.mjs
T
fc5a30805e feat(cli): add managed install, update, and service lifecycle (#10045)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Operators need a predictable installation path that survives beyond
an ephemeral `npx` process
> - A durable installation needs an owned per-user payload store, stable
command shim, safe shell integration, and supported service lifecycle
> - Updates must preserve recoverability by backing up data, installing
side-by-side, verifying the new payload, and retaining rollback state
> - Bootstrap scripts and privileged service operations must fail closed
across download, filesystem, ownership, and consent boundaries
> - This pull request integrates managed install, update, rollback,
service, uninstall, doctor, bootstrap-installer, and runtime-serving
support into one workflow
> - The benefit is a recoverable, inspectable, and documented
installation lifecycle with explicit safety boundaries across Linux,
macOS, containers, WSL, npm, npx, and source checkouts

## Linked Issues or Issue Description

### Problem

Paperclip lacks a first-class durable installation and lifecycle
workflow. Operators currently have to assemble npm/npx installation,
PATH setup, background-service management, updates, rollback,
diagnostics, and uninstall behavior themselves. That makes upgrades
harder to recover, creates inconsistent behavior across platforms, and
leaves shell/download/service trust boundaries without one documented
implementation.

### Proposed Solution

Add a managed per-user install store and stable shim, a verified shell
bootstrap installer, service lifecycle commands, install-mode-aware
update/rollback behavior, doctor checks, and documentation. Managed
updates back up the database, install and smoke-test a side-by-side
payload, atomically switch `current`, and retain prior payloads. The
shell installer pins registry/download trust boundaries and requires
explicit consent for non-interactive privileged actions.

### Alternatives Considered

- Keep recommending `npx`: simple for evaluation, but ephemeral and
unsuitable for stable services, atomic updates, or rollback.
- Require global npm installation only: familiar, but cannot provide the
owned side-by-side payload store and retained rollback semantics.
- Split the capability across multiple PRs: rejected because install,
update, service, uninstall, bootstrap, and serving behavior share
contracts and security boundaries that need review together.

### Related Pull Requests

- Supersedes #10042 and #10044 with one integrated final diff.
- Incorporates and replaces the closed preparatory work in #10032 and
#10034.

## What Changed

- Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall,
service lifecycle, onboarding integration, and managed-install doctor
checks.
- Added a private managed payload store, verified manifest/marker
ownership, exclusive mutation locks, atomic manifest/current/shim
writes, retained previous payloads, and provenance validation.
- Added npm and GitHub-ref install sources with exact target resolution,
registry isolation, database backup, side-by-side verification, atomic
activation, service restart coordination, and failure rollback.
- Made managed-update backups report actionable service-start and
`--no-backup` recovery guidance for unreachable databases, while clean
never-onboarded instances skip an empty backup.
- Added systemd user and launchd service definitions, status/health/log
commands, single-instance coordination, stale-port recovery, and
explicit sudo/lingering consent handling.
- Added the `scripts/install.sh` bootstrap path with checked two-stage
downloads, pinned public npm registry usage, platform checks,
dry-run/non-interactive controls, and Docker fixtures.
- Added embedded Postgres/native bootstrap integration,
hot-restart/systemd-notify serving support, passive update notices,
configuration contracts, README/CLI/install documentation, and focused
regression tests.
- Security re-review should explicitly re-verify: (1)
`addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or
non-regular rc files, assert current-user ownership, preserve
restrictive modes, and replace atomically; (2) managed shim replacement
rejects unsafe parents, foreign-owned or multiply linked files, and uses
checked atomic replacement; (3) the shell installer and sudo path
preserve explicit consent and checked downloads; and (4) installed
service/runtime serving remains bound to the validated managed shim and
instance configuration.

## Verification

- `bash -n scripts/install.sh scripts/clean-install-git.sh
scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh`
- `pnpm exec vitest run cli/src/__tests__/install-store.test.ts
cli/src/__tests__/install-command.test.ts
cli/src/__tests__/managed-install-check.test.ts
cli/src/__tests__/onboard-service.test.ts
cli/src/__tests__/service-health-check.test.ts
cli/src/__tests__/service-manager.test.ts
cli/src/__tests__/update-command.test.ts
cli/src/__tests__/update-notice.test.ts
packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests
passed
- `pnpm --dir cli typecheck`
- `pnpm --dir cli build`
- Follow-up verification: `pnpm exec vitest run
cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli
typecheck`, `pnpm --dir cli build`, and `pnpm --filter
@paperclipai/server typecheck`.
- `pnpm -r typecheck`
- `pnpm build`
- Full `pnpm test:run` exercised all suites; an injected static AWS
credential changed one unrelated doctor expectation, which passed when
those credentials were removed. A second run cleared that case and
exposed stale pre-existing adapter-utils `dist` output; rebuilding
`@paperclipai/adapter-utils` made the isolated test pass. The updated PR
CI is the authoritative clean-workspace full-suite run.

## Risks

- Installer/update code writes executable shims, symlinks, shell rc
blocks, service definitions, and managed payloads; ownership,
regular-file, symlink, hard-link, marker, and path-containment checks
fail closed before destructive changes.
- The bootstrap installer executes downloaded tooling; downloads are
staged and checked before execution, npm traffic is pinned to the public
registry, and non-interactive privileged behavior requires explicit
consent.
- Linux lingering may invoke `sudo`; the command is surfaced and
confirmed before execution, and unsupported service managers fall back
to foreground-run guidance.
- Database migrations remain forward-only; payload rollback does not
reverse migrations, so managed updates create a backup before activation
unless explicitly disabled.
- Service restart and runtime serving touch process/port ownership;
lifecycle locks, health/version checks, and stable-shim service
definitions reduce split-brain and stale-process risk.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with
reasoning, repository/API access, shell execution, and test tooling. The
runtime did not expose a reliable context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:52:23 -07:00

198 lines
7.0 KiB
JavaScript

import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import {
existsSync,
lstatSync,
mkdirSync,
mkdtempSync,
readFileSync,
rmSync,
writeFileSync,
} from "node:fs";
import { readFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import cliEsbuildConfig from "../cli/esbuild.config.mjs";
import { bundledCliNpmDependencies } from "./cli-bundled-npm-dependencies.mjs";
import {
createBundledInstallManifest,
materializePublishManifest,
} from "./prepare-bundled-package.mjs";
const rootPackage = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8"));
const adapterUtilsPackage = JSON.parse(
await readFile(new URL("../packages/adapter-utils/package.json", import.meta.url), "utf8"),
);
const dbPackage = JSON.parse(
await readFile(new URL("../packages/db/package.json", import.meta.url), "utf8"),
);
const releaseScript = await readFile(new URL("./release.sh", import.meta.url), "utf8");
const releaseLib = await readFile(new URL("./release-lib.sh", import.meta.url), "utf8");
const buildNpmScript = await readFile(new URL("./build-npm.sh", import.meta.url), "utf8");
test("published packages preserve the patched ACPX runtime", () => {
assert.equal(
rootPackage.pnpm.patchedDependencies["acpx@0.12.0"],
"patches/acpx@0.12.0.patch",
);
assert.equal(adapterUtilsPackage.dependencies.acpx, "0.12.0");
assert.deepEqual(adapterUtilsPackage.bundleDependencies, ["acpx"]);
assert.equal(bundledCliNpmDependencies.has("acpx"), true);
assert.equal(cliEsbuildConfig.external.includes("acpx"), false);
});
test("published packages preserve the patched embedded-postgres runtime", () => {
assert.equal(
rootPackage.pnpm.patchedDependencies["embedded-postgres@18.1.0-beta.16"],
"patches/embedded-postgres@18.1.0-beta.16.patch",
);
assert.deepEqual(dbPackage.bundleDependencies, ["embedded-postgres"]);
assert.equal(bundledCliNpmDependencies.has("embedded-postgres"), true);
assert.equal(cliEsbuildConfig.external.includes("embedded-postgres"), false);
});
test("bundled package staging materializes publishConfig entrypoints", () => {
const staged = materializePublishManifest(adapterUtilsPackage);
assert.equal(staged.publishConfig, undefined);
assert.equal(staged.main, "./dist/index.js");
assert.equal(staged.types, "./dist/index.d.ts");
assert.deepEqual(staged.exports, adapterUtilsPackage.publishConfig.exports);
});
test("bundled package staging materializes workspace dependency versions", () => {
const staged = materializePublishManifest({
name: "@paperclipai/example",
version: "2026.723.0",
dependencies: { exact: "workspace:*", caret: "workspace:^", tilde: "workspace:~" },
});
assert.deepEqual(staged.dependencies, {
exact: "2026.723.0",
caret: "^2026.723.0",
tilde: "~2026.723.0",
});
});
test("bundled package staging installs only dependencies included in the tarball", () => {
const installManifest = createBundledInstallManifest(
{
name: "@paperclipai/db",
version: "2026.723.0-canary.8",
dependencies: {
"@paperclipai/shared": "2026.723.0-canary.8",
"drizzle-orm": "^0.45.2",
"embedded-postgres": "^18.1.0-beta.16",
},
bundleDependencies: ["embedded-postgres"],
},
["embedded-postgres"],
);
assert.deepEqual(installManifest.dependencies, {
"embedded-postgres": "^18.1.0-beta.16",
});
assert.deepEqual(installManifest.bundleDependencies, ["embedded-postgres"]);
});
test("bundled package staging rebuilds npm dependencies and applies the acpx patch", (t) => {
const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-bundled-stage-"));
const sourceDir = join(fixtureDir, "source");
const destinationDir = join(fixtureDir, "destination");
const binDir = join(fixtureDir, "bin");
const callLog = join(fixtureDir, "calls.log");
mkdirSync(sourceDir);
mkdirSync(join(sourceDir, "dist"));
writeFileSync(join(sourceDir, "dist", "index.js"), "export {};\n");
mkdirSync(destinationDir);
mkdirSync(binDir);
writeFileSync(join(sourceDir, "package.json"), JSON.stringify(adapterUtilsPackage));
writeFileSync(callLog, "");
t.after(() => rmSync(fixtureDir, { recursive: true, force: true }));
const writeExecutable = (name, body) => {
writeFileSync(join(binDir, name), body, { mode: 0o755 });
};
writeExecutable(
"pnpm",
`#!/usr/bin/env bash
set -euo pipefail
printf 'pnpm %s\\n' "$*" >> "$FAKE_CALL_LOG"
destination="\${!#}"
cp "$FAKE_SOURCE_PACKAGE" "$destination/package.json"
mkdir -p "$destination/node_modules/.pnpm"
`,
);
writeExecutable(
"npm",
`#!/usr/bin/env bash
set -euo pipefail
printf 'npm %s\\n' "$*" >> "$FAKE_CALL_LOG"
[ "$*" = "install --omit=dev --ignore-scripts --no-audit --no-fund" ]
mkdir -p node_modules/acpx/dist
printf 'unpatched runtime\\n' > node_modules/acpx/dist/runtime.js
`,
);
writeExecutable(
"patch",
`#!/usr/bin/env bash
set -euo pipefail
printf 'patch %s\\n' "$*" >> "$FAKE_CALL_LOG"
target=""
while [ "$#" -gt 0 ]; do
if [ "$1" = "-d" ]; then
target="$2"
shift 2
else
shift
fi
done
patch_input="$(cat)"
grep -q onAgentStderr <<< "$patch_input"
printf 'patched onAgentStderr runtime\\n' > "$target/dist/runtime.js"
`,
);
execFileSync(
process.execPath,
[new URL("./prepare-bundled-package.mjs", import.meta.url).pathname, sourceDir, destinationDir],
{
env: {
...process.env,
PATH: `${binDir}:${process.env.PATH}`,
FAKE_CALL_LOG: callLog,
FAKE_SOURCE_PACKAGE: join(sourceDir, "package.json"),
},
stdio: "pipe",
},
);
const stagedAcpxDir = join(destinationDir, "node_modules/acpx");
assert.equal(lstatSync(stagedAcpxDir).isDirectory(), true);
assert.equal(lstatSync(stagedAcpxDir).isSymbolicLink(), false);
assert.equal(existsSync(join(destinationDir, "node_modules/.pnpm")), false);
assert.match(readFileSync(join(stagedAcpxDir, "dist/runtime.js"), "utf8"), /onAgentStderr/);
assert.match(
readFileSync(callLog, "utf8"),
/patch -p1 --forward -d .*node_modules\/acpx/,
);
});
test("bundled package dry runs preview without querying published versions", () => {
assert.match(releaseScript, /run_bundled_npm_pack pack --pack-destination "\$publish_dir"/);
assert.match(releaseLib, /BUNDLED_NPM_PACK_VERSION="10\.9\.7"/);
assert.match(releaseLib, /BUNDLED_NPM_PUBLISH_VERSION="11\.18\.0"/);
assert.match(releaseLib, /npx --yes "npm@\$BUNDLED_NPM_PACK_VERSION"/);
assert.match(releaseLib, /npx --yes "npm@\$BUNDLED_NPM_PUBLISH_VERSION"/);
assert.match(releaseLib, /"\$@" --loglevel verbose/);
assert.match(releaseLib, /run_bundled_npm_publish publish --tag "\$dist_tag"/);
assert.doesNotMatch(releaseLib, /run_bundled_npm_publish publish "\.\/\$tarball"/);
});
test("npm builds use corepack instead of requiring a global pnpm", () => {
assert.match(buildNpmScript, /corepack pnpm -r typecheck/);
assert.doesNotMatch(buildNpmScript, /^\s*pnpm -r typecheck/m);
});