mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 00:53:38 +02:00
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Operators need a predictable installation path that survives beyond an ephemeral `npx` process > - A durable installation needs an owned per-user payload store, stable command shim, safe shell integration, and supported service lifecycle > - Updates must preserve recoverability by backing up data, installing side-by-side, verifying the new payload, and retaining rollback state > - Bootstrap scripts and privileged service operations must fail closed across download, filesystem, ownership, and consent boundaries > - This pull request integrates managed install, update, rollback, service, uninstall, doctor, bootstrap-installer, and runtime-serving support into one workflow > - The benefit is a recoverable, inspectable, and documented installation lifecycle with explicit safety boundaries across Linux, macOS, containers, WSL, npm, npx, and source checkouts ## Linked Issues or Issue Description ### Problem Paperclip lacks a first-class durable installation and lifecycle workflow. Operators currently have to assemble npm/npx installation, PATH setup, background-service management, updates, rollback, diagnostics, and uninstall behavior themselves. That makes upgrades harder to recover, creates inconsistent behavior across platforms, and leaves shell/download/service trust boundaries without one documented implementation. ### Proposed Solution Add a managed per-user install store and stable shim, a verified shell bootstrap installer, service lifecycle commands, install-mode-aware update/rollback behavior, doctor checks, and documentation. Managed updates back up the database, install and smoke-test a side-by-side payload, atomically switch `current`, and retain prior payloads. The shell installer pins registry/download trust boundaries and requires explicit consent for non-interactive privileged actions. ### Alternatives Considered - Keep recommending `npx`: simple for evaluation, but ephemeral and unsuitable for stable services, atomic updates, or rollback. - Require global npm installation only: familiar, but cannot provide the owned side-by-side payload store and retained rollback semantics. - Split the capability across multiple PRs: rejected because install, update, service, uninstall, bootstrap, and serving behavior share contracts and security boundaries that need review together. ### Related Pull Requests - Supersedes #10042 and #10044 with one integrated final diff. - Incorporates and replaces the closed preparatory work in #10032 and #10034. ## What Changed - Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall, service lifecycle, onboarding integration, and managed-install doctor checks. - Added a private managed payload store, verified manifest/marker ownership, exclusive mutation locks, atomic manifest/current/shim writes, retained previous payloads, and provenance validation. - Added npm and GitHub-ref install sources with exact target resolution, registry isolation, database backup, side-by-side verification, atomic activation, service restart coordination, and failure rollback. - Made managed-update backups report actionable service-start and `--no-backup` recovery guidance for unreachable databases, while clean never-onboarded instances skip an empty backup. - Added systemd user and launchd service definitions, status/health/log commands, single-instance coordination, stale-port recovery, and explicit sudo/lingering consent handling. - Added the `scripts/install.sh` bootstrap path with checked two-stage downloads, pinned public npm registry usage, platform checks, dry-run/non-interactive controls, and Docker fixtures. - Added embedded Postgres/native bootstrap integration, hot-restart/systemd-notify serving support, passive update notices, configuration contracts, README/CLI/install documentation, and focused regression tests. - Security re-review should explicitly re-verify: (1) `addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or non-regular rc files, assert current-user ownership, preserve restrictive modes, and replace atomically; (2) managed shim replacement rejects unsafe parents, foreign-owned or multiply linked files, and uses checked atomic replacement; (3) the shell installer and sudo path preserve explicit consent and checked downloads; and (4) installed service/runtime serving remains bound to the validated managed shim and instance configuration. ## Verification - `bash -n scripts/install.sh scripts/clean-install-git.sh scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh` - `pnpm exec vitest run cli/src/__tests__/install-store.test.ts cli/src/__tests__/install-command.test.ts cli/src/__tests__/managed-install-check.test.ts cli/src/__tests__/onboard-service.test.ts cli/src/__tests__/service-health-check.test.ts cli/src/__tests__/service-manager.test.ts cli/src/__tests__/update-command.test.ts cli/src/__tests__/update-notice.test.ts packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests passed - `pnpm --dir cli typecheck` - `pnpm --dir cli build` - Follow-up verification: `pnpm exec vitest run cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli typecheck`, `pnpm --dir cli build`, and `pnpm --filter @paperclipai/server typecheck`. - `pnpm -r typecheck` - `pnpm build` - Full `pnpm test:run` exercised all suites; an injected static AWS credential changed one unrelated doctor expectation, which passed when those credentials were removed. A second run cleared that case and exposed stale pre-existing adapter-utils `dist` output; rebuilding `@paperclipai/adapter-utils` made the isolated test pass. The updated PR CI is the authoritative clean-workspace full-suite run. ## Risks - Installer/update code writes executable shims, symlinks, shell rc blocks, service definitions, and managed payloads; ownership, regular-file, symlink, hard-link, marker, and path-containment checks fail closed before destructive changes. - The bootstrap installer executes downloaded tooling; downloads are staged and checked before execution, npm traffic is pinned to the public registry, and non-interactive privileged behavior requires explicit consent. - Linux lingering may invoke `sudo`; the command is surfaced and confirmed before execution, and unsupported service managers fall back to foreground-run guidance. - Database migrations remain forward-only; payload rollback does not reverse migrations, so managed updates create a backup before activation unless explicitly disabled. - Service restart and runtime serving touch process/port ownership; lifecycle locks, health/version checks, and stable-shim service definitions reduce split-brain and stale-process risk. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with reasoning, repository/API access, shell execution, and test tooling. The runtime did not expose a reliable context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
224 lines
5.8 KiB
TypeScript
224 lines
5.8 KiB
TypeScript
import * as p from "@clack/prompts";
|
|
import pc from "picocolors";
|
|
import type { PaperclipConfig } from "../config/schema.js";
|
|
import { readConfig, resolveConfigPath } from "../config/store.js";
|
|
import {
|
|
agentJwtSecretCheck,
|
|
configCheck,
|
|
databaseCheck,
|
|
deploymentAuthCheck,
|
|
llmCheck,
|
|
logCheck,
|
|
managedInstallChecks,
|
|
nodeRuntimeCheck,
|
|
portCheck,
|
|
secretsCheck,
|
|
serviceHealthChecks,
|
|
storageCheck,
|
|
type CheckResult,
|
|
} from "../checks/index.js";
|
|
import { loadPaperclipEnvFile } from "../config/env.js";
|
|
import { printPaperclipCliBanner } from "../utils/banner.js";
|
|
import { printUpdateNotice } from "../update-notice.js";
|
|
|
|
const STATUS_ICON = {
|
|
pass: pc.green("✓"),
|
|
warn: pc.yellow("!"),
|
|
fail: pc.red("✗"),
|
|
} as const;
|
|
|
|
export async function doctor(opts: {
|
|
config?: string;
|
|
repair?: boolean;
|
|
yes?: boolean;
|
|
}): Promise<{ passed: number; warned: number; failed: number }> {
|
|
await printUpdateNotice(opts.config);
|
|
printPaperclipCliBanner();
|
|
p.intro(pc.bgCyan(pc.black(" paperclip doctor ")));
|
|
|
|
const configPath = resolveConfigPath(opts.config);
|
|
loadPaperclipEnvFile(configPath);
|
|
const results: CheckResult[] = [];
|
|
|
|
// 1. Config check (must pass before others)
|
|
const cfgResult = configCheck(opts.config);
|
|
results.push(cfgResult);
|
|
printResult(cfgResult);
|
|
|
|
if (cfgResult.status === "fail") {
|
|
return printSummary(results);
|
|
}
|
|
|
|
let config: PaperclipConfig;
|
|
try {
|
|
config = readConfig(opts.config)!;
|
|
} catch (err) {
|
|
const readResult: CheckResult = {
|
|
name: "Config file",
|
|
status: "fail",
|
|
message: `Could not read config: ${err instanceof Error ? err.message : String(err)}`,
|
|
canRepair: false,
|
|
repairHint: "Run `paperclipai configure --section database` or `paperclipai onboard`",
|
|
};
|
|
results.push(readResult);
|
|
printResult(readResult);
|
|
return printSummary(results);
|
|
}
|
|
|
|
// 2. Deployment/auth mode check
|
|
const deploymentAuthResult = deploymentAuthCheck(config);
|
|
results.push(deploymentAuthResult);
|
|
printResult(deploymentAuthResult);
|
|
|
|
// 3. Agent JWT check
|
|
results.push(
|
|
await runRepairableCheck({
|
|
run: () => agentJwtSecretCheck(opts.config),
|
|
configPath,
|
|
opts,
|
|
}),
|
|
);
|
|
|
|
// 4. Secrets adapter check
|
|
results.push(
|
|
await runRepairableCheck({
|
|
run: () => secretsCheck(config, configPath),
|
|
configPath,
|
|
opts,
|
|
}),
|
|
);
|
|
|
|
// 5. Storage check
|
|
results.push(
|
|
await runRepairableCheck({
|
|
run: () => storageCheck(config, configPath),
|
|
configPath,
|
|
opts,
|
|
}),
|
|
);
|
|
|
|
// 6. Database check
|
|
results.push(
|
|
await runRepairableCheck({
|
|
run: () => databaseCheck(config, configPath),
|
|
configPath,
|
|
opts,
|
|
}),
|
|
);
|
|
|
|
// 7. LLM check
|
|
const llmResult = await llmCheck(config);
|
|
results.push(llmResult);
|
|
printResult(llmResult);
|
|
|
|
// 8. Log directory check
|
|
results.push(
|
|
await runRepairableCheck({
|
|
run: () => logCheck(config, configPath),
|
|
configPath,
|
|
opts,
|
|
}),
|
|
);
|
|
|
|
// 9. Port check
|
|
const portResult = await portCheck(config);
|
|
results.push(portResult);
|
|
printResult(portResult);
|
|
|
|
// 10. Runtime and managed install checks
|
|
const nodeResult = nodeRuntimeCheck();
|
|
results.push(nodeResult);
|
|
printResult(nodeResult);
|
|
for (const result of managedInstallChecks()) {
|
|
results.push(result);
|
|
printResult(result);
|
|
}
|
|
|
|
// 11. Background service checks
|
|
for (const result of await serviceHealthChecks(config)) {
|
|
results.push(result);
|
|
printResult(result);
|
|
}
|
|
|
|
// Summary
|
|
return printSummary(results);
|
|
}
|
|
|
|
function printResult(result: CheckResult): void {
|
|
const icon = STATUS_ICON[result.status];
|
|
p.log.message(`${icon} ${pc.bold(result.name)}: ${result.message}`);
|
|
if (result.status !== "pass" && result.repairHint) {
|
|
p.log.message(` ${pc.dim(result.repairHint)}`);
|
|
}
|
|
}
|
|
|
|
async function maybeRepair(
|
|
result: CheckResult,
|
|
opts: { repair?: boolean; yes?: boolean },
|
|
): Promise<boolean> {
|
|
if (result.status === "pass" || !result.canRepair || !result.repair) return false;
|
|
if (!opts.repair) return false;
|
|
|
|
let shouldRepair = opts.yes;
|
|
if (!shouldRepair) {
|
|
const answer = await p.confirm({
|
|
message: `Repair "${result.name}"?`,
|
|
initialValue: true,
|
|
});
|
|
if (p.isCancel(answer)) return false;
|
|
shouldRepair = answer;
|
|
}
|
|
|
|
if (shouldRepair) {
|
|
try {
|
|
await result.repair();
|
|
p.log.success(`Repaired: ${result.name}`);
|
|
return true;
|
|
} catch (err) {
|
|
p.log.error(`Repair failed: ${err instanceof Error ? err.message : String(err)}`);
|
|
}
|
|
}
|
|
return false;
|
|
}
|
|
|
|
async function runRepairableCheck(input: {
|
|
run: () => CheckResult | Promise<CheckResult>;
|
|
configPath: string;
|
|
opts: { repair?: boolean; yes?: boolean };
|
|
}): Promise<CheckResult> {
|
|
let result = await input.run();
|
|
printResult(result);
|
|
|
|
const repaired = await maybeRepair(result, input.opts);
|
|
if (!repaired) return result;
|
|
|
|
// Repairs may create/update the adjacent .env file or other local resources.
|
|
loadPaperclipEnvFile(input.configPath);
|
|
result = await input.run();
|
|
printResult(result);
|
|
return result;
|
|
}
|
|
|
|
function printSummary(results: CheckResult[]): { passed: number; warned: number; failed: number } {
|
|
const passed = results.filter((r) => r.status === "pass").length;
|
|
const warned = results.filter((r) => r.status === "warn").length;
|
|
const failed = results.filter((r) => r.status === "fail").length;
|
|
|
|
const parts: string[] = [];
|
|
parts.push(pc.green(`${passed} passed`));
|
|
if (warned) parts.push(pc.yellow(`${warned} warnings`));
|
|
if (failed) parts.push(pc.red(`${failed} failed`));
|
|
|
|
p.note(parts.join(", "), "Summary");
|
|
|
|
if (failed > 0) {
|
|
p.outro(pc.red("Some checks failed. Fix the issues above and re-run doctor."));
|
|
} else if (warned > 0) {
|
|
p.outro(pc.yellow("All critical checks passed with some warnings."));
|
|
} else {
|
|
p.outro(pc.green("All checks passed!"));
|
|
}
|
|
|
|
return { passed, warned, failed };
|
|
}
|