mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 02:01:23 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The pipeline subsystem models repeatable work as items moving through stages, with agent automation, review gates, blockers, drift notices, and linked work. > - Operators need this to be usable as one coherent workflow surface, not just as backend primitives or disconnected route experiments. > - The branch now carries the pipeline data model, service/routes, CLI/tutorial path, aggregation feeds, operator UI, stage automation controls, liveness/retry handling, and follow-up polish that make the primitive reviewable end to end. > - This pull request is the single review target for that pipeline workflow primitive stack. > - The benefit is that reviewers can evaluate the full operator experience and server contract together against `master`. ## Linked Issues or Issue Description No public GitHub issue exists for this work. The underlying feature request is described inline. ### Problem or motivation Paperclip needs a first-class way to model multi-stage agent/company workflows where upstream items can spawn downstream work, request review, carry fields across pipelines, surface drift, retry automation, and show operators where work is blocked or active. Without a unified pipeline primitive, these workflows spread across ad hoc issues, routines, and comments, making the state hard to inspect or operate. ### Proposed solution Add the pipeline workflow primitive stack: database schema and migrations, shared validators/types, server services and REST routes, aggregation and liveness helpers, CLI/tutorial smoke support, and the React operator UI for pipeline lists, boards, item detail, review/learnings views, settings, stage automation, secrets, carry-over fields, and retry/recovery flows. ### Alternatives considered - Keep workflows as loosely linked issues and routines: rejected because operators need a single board/detail/settings surface for repeated workflow patterns. - Ship backend primitives first and defer UI: rejected for this branch because the operator experience is the main way to validate the primitive. - Add a narrower one-off content workflow: rejected because the same primitives are useful across future company processes. ## What Changed - Added and evolved pipeline schema, migrations, shared contracts, server services, REST routes, route tests, and CLI/tutorial smoke support. - Added pipeline aggregation, health/liveness, drift acknowledgment, blocker/carry-over, automation retry, stage automation environment, and permission recovery behavior. - Added the operator UI for pipeline index/board/item detail/settings/review/learnings flows, including stage secrets, automation controls, markdown/item descriptions, linked issue assets, liveness banners, and source automation metadata. - Refactored issue document frame rendering through the shared `DocumentFrameHeader` component to keep document controls consistent with the pipeline document surfaces. - Kept this PR as the single base-branch review target for the current pipeline branch. ## Verification Current branch refresh: - `pnpm vitest run server/src/__tests__/pipelines-service.test.ts` — 31 passed - `pnpm vitest run server/src/__tests__/pipelines-routes.test.ts` — 19 passed - `pnpm --filter ./server typecheck` — passed - `pnpm --filter ./ui typecheck` — passed - Verified Pipelines remains gated by `enablePipelines === true`: sidebar item is hidden unless the flag is enabled, direct pipeline routes redirect to `/dashboard` when disabled, and the Experimental settings UI still has no Pipelines toggle. - GitHub status checks on `df071c710646de625131064c3fb6588b5e97964a` — all complete with no failing conclusions, including Actions, Socket, Superagent/Security, and Greptile Review - Greptile summary on `df071c710646de625131064c3fb6588b5e97964a` — Confidence Score 5/5 - GitHub review-thread sweep — 0 unresolved Greptile threads Previously recorded during branch development: - Server pipeline service/route and aggregation tests - Shared validator tests - UI pipeline page/settings/item-detail/learnings/liveness tests - Pipeline tutorial smoke path ## Risks - High review surface: this is a large feature branch spanning database, shared contracts, server behavior, CLI/docs, and UI. - Migration ordering and schema compatibility need reviewer attention because this branch has been kept current across multiple `master` syncs. - GitHub still reports merge state `BLOCKED` because the PR is awaiting normal human review/branch-protection completion; all current status checks are green. - Branch-name checklist exception: this PR uses the pre-existing requested branch name, which predates the current public-branch naming rule. The PR title/body avoid internal issue references. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent based on GPT-5, with repository tool use, shell execution, git/GitHub CLI operations, and local verification commands. Earlier commits in this branch were assisted by Paperclip agents and other AI coding agents as recorded in commit authorship. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
222 lines
9.4 KiB
TypeScript
222 lines
9.4 KiB
TypeScript
import { Command } from "commander";
|
|
import { onboard } from "./commands/onboard.js";
|
|
import { doctor } from "./commands/doctor.js";
|
|
import { envCommand } from "./commands/env.js";
|
|
import { configure } from "./commands/configure.js";
|
|
import { addAllowedHostname } from "./commands/allowed-hostname.js";
|
|
import { heartbeatRun } from "./commands/heartbeat-run.js";
|
|
import { runCommand } from "./commands/run.js";
|
|
import { bootstrapCeoInvite } from "./commands/auth-bootstrap-ceo.js";
|
|
import { dbBackupCommand } from "./commands/db-backup.js";
|
|
import { registerEnvLabCommands } from "./commands/env-lab.js";
|
|
import { registerContextCommands } from "./commands/client/context.js";
|
|
import { registerCompanyCommands } from "./commands/client/company.js";
|
|
import { registerIssueCommands } from "./commands/client/issue.js";
|
|
import { registerAgentCommands } from "./commands/client/agent.js";
|
|
import { registerProjectCommands } from "./commands/client/project.js";
|
|
import { registerGoalCommands } from "./commands/client/goal.js";
|
|
import { registerApprovalCommands } from "./commands/client/approval.js";
|
|
import { registerActivityCommands } from "./commands/client/activity.js";
|
|
import { registerDashboardCommands } from "./commands/client/dashboard.js";
|
|
import { registerRoutineCommands } from "./commands/routines.js";
|
|
import { registerPipelineCommands } from "./commands/pipelines.js";
|
|
import { registerFeedbackCommands } from "./commands/client/feedback.js";
|
|
import { registerSecretCommands } from "./commands/client/secrets.js";
|
|
import { registerCloudCommands } from "./commands/client/cloud.js";
|
|
import { registerSkillsCommands } from "./commands/client/skills.js";
|
|
import { registerTeamCommands } from "./commands/client/teams.js";
|
|
import { applyDataDirOverride, type DataDirOptionLike } from "./config/data-dir.js";
|
|
import { loadPaperclipEnvFile } from "./config/env.js";
|
|
import { initTelemetryFromConfigFile, flushTelemetry } from "./telemetry.js";
|
|
import { registerWorktreeCommands } from "./commands/worktree.js";
|
|
import { registerPluginCommands } from "./commands/client/plugin.js";
|
|
import { registerClientAuthCommands } from "./commands/client/auth.js";
|
|
import { registerConnectCommand } from "./commands/client/connect.js";
|
|
import { registerTokenCommands } from "./commands/client/token.js";
|
|
import { registerPromptCommands } from "./commands/client/prompt.js";
|
|
import { registerRunCommands } from "./commands/client/run.js";
|
|
import { registerCostCommands } from "./commands/client/cost.js";
|
|
import { registerWorkspaceCommands } from "./commands/client/workspace.js";
|
|
import { registerAccessCommands } from "./commands/client/access.js";
|
|
import { registerRoutineApiCommands } from "./commands/client/routine-api.js";
|
|
import { registerAdapterCommands } from "./commands/client/adapter.js";
|
|
import { registerAssetCommands } from "./commands/client/asset.js";
|
|
import { registerSkillCommands } from "./commands/client/skill.js";
|
|
import { cliVersion } from "./version.js";
|
|
|
|
const program = new Command();
|
|
const DATA_DIR_OPTION_HELP =
|
|
"Paperclip data directory root (isolates state from ~/.paperclip)";
|
|
|
|
program
|
|
.name("paperclipai")
|
|
.description("Paperclip CLI — setup, diagnose, and configure your instance")
|
|
.version(cliVersion);
|
|
|
|
program.hook("preAction", (_thisCommand, actionCommand) => {
|
|
const options = actionCommand.optsWithGlobals() as DataDirOptionLike;
|
|
const optionNames = new Set(actionCommand.options.map((option) => option.attributeName()));
|
|
applyDataDirOverride(options, {
|
|
hasConfigOption: optionNames.has("config"),
|
|
hasContextOption: optionNames.has("context"),
|
|
});
|
|
loadPaperclipEnvFile(options.config);
|
|
initTelemetryFromConfigFile(options.config);
|
|
});
|
|
|
|
program
|
|
.command("onboard")
|
|
.description("Interactive first-run setup wizard")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--bind <mode>", "Quickstart reachability preset (loopback, lan, tailnet)")
|
|
.option("-y, --yes", "Accept quickstart defaults (trusted local loopback unless --bind is set) and start immediately", false)
|
|
.option("--run", "Start Paperclip immediately after saving config", false)
|
|
.action(onboard);
|
|
|
|
program
|
|
.command("doctor")
|
|
.description("Run diagnostic checks on your Paperclip setup")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--repair", "Attempt to repair issues automatically")
|
|
.alias("--fix")
|
|
.option("-y, --yes", "Skip repair confirmation prompts")
|
|
.action(async (opts) => {
|
|
await doctor(opts);
|
|
});
|
|
|
|
program
|
|
.command("env")
|
|
.description("Print environment variables for deployment")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.action(envCommand);
|
|
|
|
program
|
|
.command("configure")
|
|
.description("Update configuration sections")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("-s, --section <section>", "Section to configure (llm, database, logging, server, storage, secrets)")
|
|
.action(configure);
|
|
|
|
program
|
|
.command("db:backup")
|
|
.description("Create a one-off database backup using current config")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--dir <path>", "Backup output directory (overrides config)")
|
|
.option("--retention-days <days>", "Retention window used for pruning", (value) => Number(value))
|
|
.option("--filename-prefix <prefix>", "Backup filename prefix", "paperclip")
|
|
.option("--json", "Print backup metadata as JSON")
|
|
.action(async (opts) => {
|
|
await dbBackupCommand(opts);
|
|
});
|
|
|
|
program
|
|
.command("allowed-hostname")
|
|
.description("Allow a hostname for authenticated/private mode access")
|
|
.argument("<host>", "Hostname to allow (for example dotta-macbook-pro)")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.action(addAllowedHostname);
|
|
|
|
const run = program
|
|
.command("run")
|
|
.description("Bootstrap local setup (onboard + doctor) and run Paperclip")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("-i, --instance <id>", "Local instance id (default: default)")
|
|
.option("--bind <mode>", "On first run, use onboarding reachability preset (loopback, lan, tailnet)")
|
|
.option("--repair", "Attempt automatic repairs during doctor", true)
|
|
.option("--no-repair", "Disable automatic repairs during doctor")
|
|
.action(runCommand);
|
|
|
|
registerRunCommands(run);
|
|
|
|
const heartbeat = program.command("heartbeat").description("Heartbeat utilities");
|
|
|
|
heartbeat
|
|
.command("run")
|
|
.description("Run one agent heartbeat and stream live logs")
|
|
.requiredOption("-a, --agent-id <agentId>", "Agent ID to invoke")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--context <path>", "Path to CLI context file")
|
|
.option("--profile <name>", "CLI context profile name")
|
|
.option("--api-base <url>", "Base URL for the Paperclip server API")
|
|
.option("--api-key <token>", "Bearer token for agent-authenticated calls")
|
|
.option(
|
|
"--source <source>",
|
|
"Invocation source (timer | assignment | on_demand | automation)",
|
|
"on_demand",
|
|
)
|
|
.option("--trigger <trigger>", "Trigger detail (manual | ping | callback | system)", "manual")
|
|
.option("--timeout-ms <ms>", "Max time to wait before giving up", "0")
|
|
.option("--json", "Output raw JSON where applicable")
|
|
.option("--debug", "Show raw adapter stdout/stderr JSON chunks")
|
|
.action(heartbeatRun);
|
|
|
|
registerContextCommands(program);
|
|
registerConnectCommand(program);
|
|
registerCompanyCommands(program);
|
|
registerIssueCommands(program);
|
|
registerAgentCommands(program);
|
|
registerProjectCommands(program);
|
|
registerGoalCommands(program);
|
|
registerTokenCommands(program);
|
|
registerPromptCommands(program);
|
|
registerApprovalCommands(program);
|
|
registerActivityCommands(program);
|
|
registerDashboardCommands(program);
|
|
registerCostCommands(program);
|
|
registerWorkspaceCommands(program);
|
|
registerAccessCommands(program);
|
|
registerRoutineApiCommands(program);
|
|
registerAdapterCommands(program);
|
|
registerAssetCommands(program);
|
|
registerSkillCommands(program);
|
|
registerRoutineCommands(program);
|
|
registerPipelineCommands(program);
|
|
registerFeedbackCommands(program);
|
|
registerSecretCommands(program);
|
|
registerCloudCommands(program);
|
|
registerSkillsCommands(program);
|
|
registerTeamCommands(program);
|
|
registerWorktreeCommands(program);
|
|
registerEnvLabCommands(program);
|
|
registerPluginCommands(program);
|
|
|
|
const auth = program.command("auth").description("Authentication and bootstrap utilities");
|
|
|
|
auth
|
|
.command("bootstrap-ceo")
|
|
.description("Create a one-time bootstrap invite URL for first instance admin")
|
|
.option("-c, --config <path>", "Path to config file")
|
|
.option("-d, --data-dir <path>", DATA_DIR_OPTION_HELP)
|
|
.option("--force", "Create new invite even if admin already exists", false)
|
|
.option("--expires-hours <hours>", "Invite expiration window in hours", (value) => Number(value))
|
|
.option("--base-url <url>", "Public base URL used to print invite link")
|
|
.action(bootstrapCeoInvite);
|
|
|
|
registerClientAuthCommands(auth);
|
|
|
|
async function main(): Promise<void> {
|
|
let failed = false;
|
|
try {
|
|
await program.parseAsync();
|
|
} catch (err) {
|
|
failed = true;
|
|
console.error(err instanceof Error ? err.message : String(err));
|
|
} finally {
|
|
await flushTelemetry();
|
|
}
|
|
|
|
if (failed) {
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
void main();
|