mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A company accumulates real state — issues, labels, blockers, documents, work products, monitors, attachments, agents, routines — and people need to move that state between instances: self-hosted to cloud, cloud back to self-hosted, or plain backups > - The experimental, flag-gated Cloud Sync transport (#6548) tried to solve this host-to-host: the source pushed into a receiver over HTTPS with a cross-instance consent/token handshake, which required the destination to be publicly reachable and broke for common self-hosted topologies (plain-HTTP LAN/VPN origins); the receiver half never landed upstream at all > - Meanwhile the portability bundle and the existing export/import pages already move companies offline with none of those networking constraints — but silently dropped labels, blockers, issue documents, work products, monitors, and every attachment > - This pull request removes the host-to-host transport and makes Import/Export the single data-movement path: the pages become first-class company-settings destinations, exports declare exactly what they do not carry, and bundle schemaVersion 6 now carries all of the above, with attachments as content-addressed sha256 blobs verified before a single row is written > - The benefit is a migration and backup flow that works between any two instances with no reachability requirements, no cross-instance auth, and no silent data loss ## Linked Issues or Issue Description - Refs #6548 — the original Cloud Sync sender this PR supersedes and removes. - Related, not duplicates: #1697 (goals in the portability manifest — orthogonal field addition), #954 (an earlier import/export + skill-visibility proposal predating the current portability bundle). - No open issue describes this directly, so in brief (feature-request shape): **Problem** — moving a company between instances silently lost labels (imports with label references actually hard-failed), blocker relations, issue documents, work products, monitor state, and all attachments, and the alternative Cloud Sync transport required the destination to be publicly reachable over HTTPS plus a consent handshake, which failed for typical self-hosted setups. **Desired behavior** — one Import/Export flow in company settings that produces a portable bundle carrying all of that data, tells the operator up front what it cannot carry, imports with automations paused, and offers real one-click activation afterwards. ## What Changed - New export fidelity report (`GET /api/companies/:companyId/export/fidelity`) + an "Export fidelity" panel on the Export page listing anything a bundle will not include (now only: approvals, cost history, activity history) - Imports accept `pauseAutomations`; imported agents and routines land paused, the import result reports created routines, and the Import page ends in an activation panel that actually resumes selected agents/activates routines - Export and Import pages promoted into the company-settings nav; the Cloud Upstream wizard, ux-lab page, and API client removed; the old settings route redirects to Export - Host-to-host transport removed: upstream-sync/receiver-client routes and services, CLI `cloud connect`/`cloud push` + keypair store, the shared upstream transfer contract, and the `enableCloudSync` flag; migration `0196` drops the two experimental `cloud_upstream_*` sender tables - Bundle schemaVersion 6: labels (definitions + per-task names, remapped by name on import), blocker relations (`blockedBy` slugs, cycle-tolerant), issue documents (`tasks/<slug>/documents/<key>.md`), work products (system refs nulled), monitors (notes/scheduledBy restored, imported un-armed) - Attachments travel as content-addressed `blobs/<sha256>` entries (deduped; comment-scoped attachments re-link via comment index); every blob is hash-verified **before any write**, so a corrupted bundle cannot leave a partially imported company; both zip codecs now round-trip extensionless/binary entries byte-exactly; the Import page preflights the inline body limit and offers continue-without-attachments - v5 (and older) bundles still import, with an informational warning; bundles newer than v6 are rejected cleanly - Docs: board-operator import/export guide, CLI README, README/ROADMAP updated ## Verification - `pnpm -r` typechecks (shared, db incl. migration numbering/safety checks, server, ui, cli) and `pnpm check:token-gates` — clean - Vitest: full server + shared sweep 4,888 passed / 1 skipped, with the only 3 failures being pre-existing on `master` (2× heartbeat-workspace-branch-containment, 1× workspace-runtime auto-port; reproduced identically with this change stashed); ui + cli suites green; the embedded-Postgres export-fidelity suite applies the full migration chain including the new `0196` against a fresh database - Live end-to-end on a scratch instance: seeded a company with labels, a blocker pair, an issue document, a work product, a monitor, an agent, a routine, and two binary attachments (one comment-scoped) → export → import into a fresh company → labels remapped to new ids, blocker edge and document restored, monitor un-armed with notes intact, attachments byte-identical (sha256-compared through the API), agents/routines paused → activation panel resumed them; a v5-shaped bundle imported with only the info warning; flipping one byte in a blob made the import 422 with **zero** rows created - Reviewer repro: create a company with a labeled issue + attachment → Settings → Export → download → Settings → Import on another company/instance → watch the preview, apply with "start paused", then activate ## Risks - Migration `0196` drops `cloud_upstream_connections`/`cloud_upstream_runs` — experimental tables behind a default-off flag; their connection/run history is intentionally discarded - Breaking removals are all of experimental, flag-gated surface: `/api/upstream-sync/*` + `/api/cloud-upstreams/*` routes, `paperclipai cloud connect|push`, and the `enableCloudSync` flag (stale keys in stored instance settings parse harmlessly) - Import remains non-atomic on mid-apply errors generally (pre-existing behavior); the new blob verification specifically moved ahead of all writes so tampered bundles cannot create partial state - GitHub-sourced imports do not fetch `blobs/*` and skip attachments with a warning ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), via Claude Code CLI with extended thinking, tool use, and subagent orchestration; implementation and review split across Fable 5 subagents, with live end-to-end verification against a running instance ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
220 lines
9.3 KiB
TypeScript
220 lines
9.3 KiB
TypeScript
import { Command } from "commander";
|
|
import { onboard } from "./commands/onboard.js";
|
|
import { doctor } from "./commands/doctor.js";
|
|
import { envCommand } from "./commands/env.js";
|
|
import { configure } from "./commands/configure.js";
|
|
import { addAllowedHostname } from "./commands/allowed-hostname.js";
|
|
import { heartbeatRun } from "./commands/heartbeat-run.js";
|
|
import { runCommand } from "./commands/run.js";
|
|
import { bootstrapCeoInvite } from "./commands/auth-bootstrap-ceo.js";
|
|
import { dbBackupCommand } from "./commands/db-backup.js";
|
|
import { registerEnvLabCommands } from "./commands/env-lab.js";
|
|
import { registerContextCommands } from "./commands/client/context.js";
|
|
import { registerCompanyCommands } from "./commands/client/company.js";
|
|
import { registerIssueCommands } from "./commands/client/issue.js";
|
|
import { registerAgentCommands } from "./commands/client/agent.js";
|
|
import { registerProjectCommands } from "./commands/client/project.js";
|
|
import { registerGoalCommands } from "./commands/client/goal.js";
|
|
import { registerApprovalCommands } from "./commands/client/approval.js";
|
|
import { registerActivityCommands } from "./commands/client/activity.js";
|
|
import { registerDashboardCommands } from "./commands/client/dashboard.js";
|
|
import { registerRoutineCommands } from "./commands/routines.js";
|
|
import { registerPipelineCommands } from "./commands/pipelines.js";
|
|
import { registerFeedbackCommands } from "./commands/client/feedback.js";
|
|
import { registerSecretCommands } from "./commands/client/secrets.js";
|
|
import { registerSkillsCommands } from "./commands/client/skills.js";
|
|
import { registerTeamCommands } from "./commands/client/teams.js";
|
|
import { applyDataDirOverride, type DataDirOptionLike } from "./config/data-dir.js";
|
|
import { loadPaperclipEnvFile } from "./config/env.js";
|
|
import { initTelemetryFromConfigFile, flushTelemetry } from "./telemetry.js";
|
|
import { registerWorktreeCommands } from "./commands/worktree.js";
|
|
import { registerPluginCommands } from "./commands/client/plugin.js";
|
|
import { registerClientAuthCommands } from "./commands/client/auth.js";
|
|
import { registerConnectCommand } from "./commands/client/connect.js";
|
|
import { registerTokenCommands } from "./commands/client/token.js";
|
|
import { registerPromptCommands } from "./commands/client/prompt.js";
|
|
import { registerRunCommands } from "./commands/client/run.js";
|
|
import { registerCostCommands } from "./commands/client/cost.js";
|
|
import { registerWorkspaceCommands } from "./commands/client/workspace.js";
|
|
import { registerAccessCommands } from "./commands/client/access.js";
|
|
import { registerRoutineApiCommands } from "./commands/client/routine-api.js";
|
|
import { registerAdapterCommands } from "./commands/client/adapter.js";
|
|
import { registerAssetCommands } from "./commands/client/asset.js";
|
|
import { registerSkillCommands } from "./commands/client/skill.js";
|
|
import { cliVersion } from "./version.js";
|
|
|
|
const program = new Command();
|
|
const DATA_DIR_OPTION_HELP =
|
|
"Paperclip data directory root (isolates state from ~/.paperclip)";
|
|
|
|
program
|
|
.name("paperclipai")
|
|
.description("Paperclip CLI — setup, diagnose, and configure your instance")
|
|
.version(cliVersion);
|
|
|
|
program.hook("preAction", (_thisCommand, actionCommand) => {
|
|
const options = actionCommand.optsWithGlobals() as DataDirOptionLike;
|
|
const optionNames = new Set(actionCommand.options.map((option) => option.attributeName()));
|
|
applyDataDirOverride(options, {
|
|
hasConfigOption: optionNames.has("config"),
|
|
hasContextOption: optionNames.has("context"),
|
|
});
|
|
loadPaperclipEnvFile(options.config);
|
|
initTelemetryFromConfigFile(options.config);
|
|
});
|
|
|
|
program
|
|
.command("onboard")
|
|
.description("Interactive first-run setup wizard")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--bind <mode>", "Quickstart reachability preset (loopback, lan, tailnet)")
|
|
.option("-y, --yes", "Accept quickstart defaults (trusted local loopback unless --bind is set) and start immediately", false)
|
|
.option("--run", "Start Paperclip immediately after saving config", false)
|
|
.action(onboard);
|
|
|
|
program
|
|
.command("doctor")
|
|
.description("Run diagnostic checks on your Paperclip setup")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--repair", "Attempt to repair issues automatically")
|
|
.alias("--fix")
|
|
.option("-y, --yes", "Skip repair confirmation prompts")
|
|
.action(async (opts) => {
|
|
await doctor(opts);
|
|
});
|
|
|
|
program
|
|
.command("env")
|
|
.description("Print environment variables for deployment")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.action(envCommand);
|
|
|
|
program
|
|
.command("configure")
|
|
.description("Update configuration sections")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("-s, --section <section>", "Section to configure (llm, database, logging, server, storage, secrets)")
|
|
.action(configure);
|
|
|
|
program
|
|
.command("db:backup")
|
|
.description("Create a one-off database backup using current config")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--dir <path>", "Backup output directory (overrides config)")
|
|
.option("--retention-days <days>", "Retention window used for pruning", (value) => Number(value))
|
|
.option("--filename-prefix <prefix>", "Backup filename prefix", "paperclip")
|
|
.option("--json", "Print backup metadata as JSON")
|
|
.action(async (opts) => {
|
|
await dbBackupCommand(opts);
|
|
});
|
|
|
|
program
|
|
.command("allowed-hostname")
|
|
.description("Allow a hostname for authenticated/private mode access")
|
|
.argument("<host>", "Hostname to allow (for example dotta-macbook-pro)")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.action(addAllowedHostname);
|
|
|
|
const run = program
|
|
.command("run")
|
|
.description("Bootstrap local setup (onboard + doctor) and run Paperclip")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("-i, --instance <id>", "Local instance id (default: default)")
|
|
.option("--bind <mode>", "On first run, use onboarding reachability preset (loopback, lan, tailnet)")
|
|
.option("--repair", "Attempt automatic repairs during doctor", true)
|
|
.option("--no-repair", "Disable automatic repairs during doctor")
|
|
.action(runCommand);
|
|
|
|
registerRunCommands(run);
|
|
|
|
const heartbeat = program.command("heartbeat").description("Heartbeat utilities");
|
|
|
|
heartbeat
|
|
.command("run")
|
|
.description("Run one agent heartbeat and stream live logs")
|
|
.requiredOption("-a, --agent-id <agentId>", "Agent ID to invoke")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--context <path>", "Path to CLI context file")
|
|
.option("--profile <name>", "CLI context profile name")
|
|
.option("--api-base <url>", "Base URL for the Paperclip server API")
|
|
.option("--api-key <token>", "Bearer token for agent-authenticated calls")
|
|
.option(
|
|
"--source <source>",
|
|
"Invocation source (timer | assignment | on_demand | automation)",
|
|
"on_demand",
|
|
)
|
|
.option("--trigger <trigger>", "Trigger detail (manual | ping | callback | system)", "manual")
|
|
.option("--timeout-ms <ms>", "Max time to wait before giving up", "0")
|
|
.option("--json", "Output raw JSON where applicable")
|
|
.option("--debug", "Show raw adapter stdout/stderr JSON chunks")
|
|
.action(heartbeatRun);
|
|
|
|
registerContextCommands(program);
|
|
registerConnectCommand(program);
|
|
registerCompanyCommands(program);
|
|
registerIssueCommands(program);
|
|
registerAgentCommands(program);
|
|
registerProjectCommands(program);
|
|
registerGoalCommands(program);
|
|
registerTokenCommands(program);
|
|
registerPromptCommands(program);
|
|
registerApprovalCommands(program);
|
|
registerActivityCommands(program);
|
|
registerDashboardCommands(program);
|
|
registerCostCommands(program);
|
|
registerWorkspaceCommands(program);
|
|
registerAccessCommands(program);
|
|
registerRoutineApiCommands(program);
|
|
registerAdapterCommands(program);
|
|
registerAssetCommands(program);
|
|
registerSkillCommands(program);
|
|
registerRoutineCommands(program);
|
|
registerPipelineCommands(program);
|
|
registerFeedbackCommands(program);
|
|
registerSecretCommands(program);
|
|
registerSkillsCommands(program);
|
|
registerTeamCommands(program);
|
|
registerWorktreeCommands(program);
|
|
registerEnvLabCommands(program);
|
|
registerPluginCommands(program);
|
|
|
|
const auth = program.command("auth").description("Authentication and bootstrap utilities");
|
|
|
|
auth
|
|
.command("bootstrap-ceo")
|
|
.description("Create a one-time bootstrap invite URL for first instance admin")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--force", "Create new invite even if admin already exists", false)
|
|
.option("--expires-hours <hours>", "Invite expiration window in hours", (value) => Number(value))
|
|
.option("--base-url <url>", "Public base URL used to print invite link")
|
|
.action(bootstrapCeoInvite);
|
|
|
|
registerClientAuthCommands(auth);
|
|
|
|
async function main(): Promise<void> {
|
|
let failed = false;
|
|
try {
|
|
await program.parseAsync();
|
|
} catch (err) {
|
|
failed = true;
|
|
console.error(err instanceof Error ? err.message : String(err));
|
|
} finally {
|
|
await flushTelemetry();
|
|
}
|
|
|
|
if (failed) {
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
void main();
|