Files
PaperClipAI/cli/src/index.ts
T
Devin Foley 916c13501f Replace host-to-host Cloud Sync with full-fidelity company Import/Export (#10507)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A company accumulates real state — issues, labels, blockers,
documents, work products, monitors, attachments, agents, routines — and
people need to move that state between instances: self-hosted to cloud,
cloud back to self-hosted, or plain backups
> - The experimental, flag-gated Cloud Sync transport (#6548) tried to
solve this host-to-host: the source pushed into a receiver over HTTPS
with a cross-instance consent/token handshake, which required the
destination to be publicly reachable and broke for common self-hosted
topologies (plain-HTTP LAN/VPN origins); the receiver half never landed
upstream at all
> - Meanwhile the portability bundle and the existing export/import
pages already move companies offline with none of those networking
constraints — but silently dropped labels, blockers, issue documents,
work products, monitors, and every attachment
> - This pull request removes the host-to-host transport and makes
Import/Export the single data-movement path: the pages become
first-class company-settings destinations, exports declare exactly what
they do not carry, and bundle schemaVersion 6 now carries all of the
above, with attachments as content-addressed sha256 blobs verified
before a single row is written
> - The benefit is a migration and backup flow that works between any
two instances with no reachability requirements, no cross-instance auth,
and no silent data loss

## Linked Issues or Issue Description

- Refs #6548 — the original Cloud Sync sender this PR supersedes and
removes.
- Related, not duplicates: #1697 (goals in the portability manifest —
orthogonal field addition), #954 (an earlier import/export +
skill-visibility proposal predating the current portability bundle).
- No open issue describes this directly, so in brief (feature-request
shape): **Problem** — moving a company between instances silently lost
labels (imports with label references actually hard-failed), blocker
relations, issue documents, work products, monitor state, and all
attachments, and the alternative Cloud Sync transport required the
destination to be publicly reachable over HTTPS plus a consent
handshake, which failed for typical self-hosted setups. **Desired
behavior** — one Import/Export flow in company settings that produces a
portable bundle carrying all of that data, tells the operator up front
what it cannot carry, imports with automations paused, and offers real
one-click activation afterwards.

## What Changed

- New export fidelity report (`GET
/api/companies/:companyId/export/fidelity`) + an "Export fidelity" panel
on the Export page listing anything a bundle will not include (now only:
approvals, cost history, activity history)
- Imports accept `pauseAutomations`; imported agents and routines land
paused, the import result reports created routines, and the Import page
ends in an activation panel that actually resumes selected
agents/activates routines
- Export and Import pages promoted into the company-settings nav; the
Cloud Upstream wizard, ux-lab page, and API client removed; the old
settings route redirects to Export
- Host-to-host transport removed: upstream-sync/receiver-client routes
and services, CLI `cloud connect`/`cloud push` + keypair store, the
shared upstream transfer contract, and the `enableCloudSync` flag;
migration `0196` drops the two experimental `cloud_upstream_*` sender
tables
- Bundle schemaVersion 6: labels (definitions + per-task names, remapped
by name on import), blocker relations (`blockedBy` slugs,
cycle-tolerant), issue documents (`tasks/<slug>/documents/<key>.md`),
work products (system refs nulled), monitors (notes/scheduledBy
restored, imported un-armed)
- Attachments travel as content-addressed `blobs/<sha256>` entries
(deduped; comment-scoped attachments re-link via comment index); every
blob is hash-verified **before any write**, so a corrupted bundle cannot
leave a partially imported company; both zip codecs now round-trip
extensionless/binary entries byte-exactly; the Import page preflights
the inline body limit and offers continue-without-attachments
- v5 (and older) bundles still import, with an informational warning;
bundles newer than v6 are rejected cleanly
- Docs: board-operator import/export guide, CLI README, README/ROADMAP
updated

## Verification

- `pnpm -r` typechecks (shared, db incl. migration numbering/safety
checks, server, ui, cli) and `pnpm check:token-gates` — clean
- Vitest: full server + shared sweep 4,888 passed / 1 skipped, with the
only 3 failures being pre-existing on `master` (2×
heartbeat-workspace-branch-containment, 1× workspace-runtime auto-port;
reproduced identically with this change stashed); ui + cli suites green;
the embedded-Postgres export-fidelity suite applies the full migration
chain including the new `0196` against a fresh database
- Live end-to-end on a scratch instance: seeded a company with labels, a
blocker pair, an issue document, a work product, a monitor, an agent, a
routine, and two binary attachments (one comment-scoped) → export →
import into a fresh company → labels remapped to new ids, blocker edge
and document restored, monitor un-armed with notes intact, attachments
byte-identical (sha256-compared through the API), agents/routines paused
→ activation panel resumed them; a v5-shaped bundle imported with only
the info warning; flipping one byte in a blob made the import 422 with
**zero** rows created
- Reviewer repro: create a company with a labeled issue + attachment →
Settings → Export → download → Settings → Import on another
company/instance → watch the preview, apply with "start paused", then
activate

## Risks

- Migration `0196` drops
`cloud_upstream_connections`/`cloud_upstream_runs` — experimental tables
behind a default-off flag; their connection/run history is intentionally
discarded
- Breaking removals are all of experimental, flag-gated surface:
`/api/upstream-sync/*` + `/api/cloud-upstreams/*` routes, `paperclipai
cloud connect|push`, and the `enableCloudSync` flag (stale keys in
stored instance settings parse harmlessly)
- Import remains non-atomic on mid-apply errors generally (pre-existing
behavior); the new blob verification specifically moved ahead of all
writes so tampered bundles cannot create partial state
- GitHub-sourced imports do not fetch `blobs/*` and skip attachments
with a warning

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic), via Claude Code CLI with
extended thinking, tool use, and subagent orchestration; implementation
and review split across Fable 5 subagents, with live end-to-end
verification against a running instance

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-30 11:37:00 -07:00

220 lines
9.3 KiB
TypeScript

import { Command } from "commander";
import { onboard } from "./commands/onboard.js";
import { doctor } from "./commands/doctor.js";
import { envCommand } from "./commands/env.js";
import { configure } from "./commands/configure.js";
import { addAllowedHostname } from "./commands/allowed-hostname.js";
import { heartbeatRun } from "./commands/heartbeat-run.js";
import { runCommand } from "./commands/run.js";
import { bootstrapCeoInvite } from "./commands/auth-bootstrap-ceo.js";
import { dbBackupCommand } from "./commands/db-backup.js";
import { registerEnvLabCommands } from "./commands/env-lab.js";
import { registerContextCommands } from "./commands/client/context.js";
import { registerCompanyCommands } from "./commands/client/company.js";
import { registerIssueCommands } from "./commands/client/issue.js";
import { registerAgentCommands } from "./commands/client/agent.js";
import { registerProjectCommands } from "./commands/client/project.js";
import { registerGoalCommands } from "./commands/client/goal.js";
import { registerApprovalCommands } from "./commands/client/approval.js";
import { registerActivityCommands } from "./commands/client/activity.js";
import { registerDashboardCommands } from "./commands/client/dashboard.js";
import { registerRoutineCommands } from "./commands/routines.js";
import { registerPipelineCommands } from "./commands/pipelines.js";
import { registerFeedbackCommands } from "./commands/client/feedback.js";
import { registerSecretCommands } from "./commands/client/secrets.js";
import { registerSkillsCommands } from "./commands/client/skills.js";
import { registerTeamCommands } from "./commands/client/teams.js";
import { applyDataDirOverride, type DataDirOptionLike } from "./config/data-dir.js";
import { loadPaperclipEnvFile } from "./config/env.js";
import { initTelemetryFromConfigFile, flushTelemetry } from "./telemetry.js";
import { registerWorktreeCommands } from "./commands/worktree.js";
import { registerPluginCommands } from "./commands/client/plugin.js";
import { registerClientAuthCommands } from "./commands/client/auth.js";
import { registerConnectCommand } from "./commands/client/connect.js";
import { registerTokenCommands } from "./commands/client/token.js";
import { registerPromptCommands } from "./commands/client/prompt.js";
import { registerRunCommands } from "./commands/client/run.js";
import { registerCostCommands } from "./commands/client/cost.js";
import { registerWorkspaceCommands } from "./commands/client/workspace.js";
import { registerAccessCommands } from "./commands/client/access.js";
import { registerRoutineApiCommands } from "./commands/client/routine-api.js";
import { registerAdapterCommands } from "./commands/client/adapter.js";
import { registerAssetCommands } from "./commands/client/asset.js";
import { registerSkillCommands } from "./commands/client/skill.js";
import { cliVersion } from "./version.js";
const program = new Command();
const DATA_DIR_OPTION_HELP =
"Paperclip data directory root (isolates state from ~/.paperclip)";
program
.name("paperclipai")
.description("Paperclip CLI — setup, diagnose, and configure your instance")
.version(cliVersion);
program.hook("preAction", (_thisCommand, actionCommand) => {
const options = actionCommand.optsWithGlobals() as DataDirOptionLike;
const optionNames = new Set(actionCommand.options.map((option) => option.attributeName()));
applyDataDirOverride(options, {
hasConfigOption: optionNames.has("config"),
hasContextOption: optionNames.has("context"),
});
loadPaperclipEnvFile(options.config);
initTelemetryFromConfigFile(options.config);
});
program
.command("onboard")
.description("Interactive first-run setup wizard")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("--bind <mode>", "Quickstart reachability preset (loopback, lan, tailnet)")
.option("-y, --yes", "Accept quickstart defaults (trusted local loopback unless --bind is set) and start immediately", false)
.option("--run", "Start Paperclip immediately after saving config", false)
.action(onboard);
program
.command("doctor")
.description("Run diagnostic checks on your Paperclip setup")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("--repair", "Attempt to repair issues automatically")
.alias("--fix")
.option("-y, --yes", "Skip repair confirmation prompts")
.action(async (opts) => {
await doctor(opts);
});
program
.command("env")
.description("Print environment variables for deployment")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.action(envCommand);
program
.command("configure")
.description("Update configuration sections")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("-s, --section <section>", "Section to configure (llm, database, logging, server, storage, secrets)")
.action(configure);
program
.command("db:backup")
.description("Create a one-off database backup using current config")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("--dir <path>", "Backup output directory (overrides config)")
.option("--retention-days <days>", "Retention window used for pruning", (value) => Number(value))
.option("--filename-prefix <prefix>", "Backup filename prefix", "paperclip")
.option("--json", "Print backup metadata as JSON")
.action(async (opts) => {
await dbBackupCommand(opts);
});
program
.command("allowed-hostname")
.description("Allow a hostname for authenticated/private mode access")
.argument("<host>", "Hostname to allow (for example dotta-macbook-pro)")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.action(addAllowedHostname);
const run = program
.command("run")
.description("Bootstrap local setup (onboard + doctor) and run Paperclip")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("-i, --instance <id>", "Local instance id (default: default)")
.option("--bind <mode>", "On first run, use onboarding reachability preset (loopback, lan, tailnet)")
.option("--repair", "Attempt automatic repairs during doctor", true)
.option("--no-repair", "Disable automatic repairs during doctor")
.action(runCommand);
registerRunCommands(run);
const heartbeat = program.command("heartbeat").description("Heartbeat utilities");
heartbeat
.command("run")
.description("Run one agent heartbeat and stream live logs")
.requiredOption("-a, --agent-id <agentId>", "Agent ID to invoke")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("--context <path>", "Path to CLI context file")
.option("--profile <name>", "CLI context profile name")
.option("--api-base <url>", "Base URL for the Paperclip server API")
.option("--api-key <token>", "Bearer token for agent-authenticated calls")
.option(
"--source <source>",
"Invocation source (timer | assignment | on_demand | automation)",
"on_demand",
)
.option("--trigger <trigger>", "Trigger detail (manual | ping | callback | system)", "manual")
.option("--timeout-ms <ms>", "Max time to wait before giving up", "0")
.option("--json", "Output raw JSON where applicable")
.option("--debug", "Show raw adapter stdout/stderr JSON chunks")
.action(heartbeatRun);
registerContextCommands(program);
registerConnectCommand(program);
registerCompanyCommands(program);
registerIssueCommands(program);
registerAgentCommands(program);
registerProjectCommands(program);
registerGoalCommands(program);
registerTokenCommands(program);
registerPromptCommands(program);
registerApprovalCommands(program);
registerActivityCommands(program);
registerDashboardCommands(program);
registerCostCommands(program);
registerWorkspaceCommands(program);
registerAccessCommands(program);
registerRoutineApiCommands(program);
registerAdapterCommands(program);
registerAssetCommands(program);
registerSkillCommands(program);
registerRoutineCommands(program);
registerPipelineCommands(program);
registerFeedbackCommands(program);
registerSecretCommands(program);
registerSkillsCommands(program);
registerTeamCommands(program);
registerWorktreeCommands(program);
registerEnvLabCommands(program);
registerPluginCommands(program);
const auth = program.command("auth").description("Authentication and bootstrap utilities");
auth
.command("bootstrap-ceo")
.description("Create a one-time bootstrap invite URL for first instance admin")
.option("-c, --config <path>", "Path to config file")
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
.option("--force", "Create new invite even if admin already exists", false)
.option("--expires-hours <hours>", "Invite expiration window in hours", (value) => Number(value))
.option("--base-url <url>", "Public base URL used to print invite link")
.action(bootstrapCeoInvite);
registerClientAuthCommands(auth);
async function main(): Promise<void> {
let failed = false;
try {
await program.parseAsync();
} catch (err) {
failed = true;
console.error(err instanceof Error ? err.message : String(err));
} finally {
await flushTelemetry();
}
if (failed) {
process.exit(1);
}
}
void main();