From 19c786a20b09af8de55985c77bfff5aa52629820 Mon Sep 17 00:00:00 2001 From: Dotta Date: Tue, 28 Apr 2026 12:49:00 -0500 Subject: [PATCH] docs: refresh v1 implementation docs Co-Authored-By: Paperclip --- doc/DATABASE.md | 32 ++++++++--------- doc/DEVELOPING.md | 6 ++++ doc/SPEC-implementation.md | 72 ++++++++++++++++++++++++++++++++------ 3 files changed, 82 insertions(+), 28 deletions(-) diff --git a/doc/DATABASE.md b/doc/DATABASE.md index 6e0cbed6d08..15fa4a0afa0 100644 --- a/doc/DATABASE.md +++ b/doc/DATABASE.md @@ -59,11 +59,11 @@ cp .env.example .env # DATABASE_URL=postgres://paperclip:paperclip@localhost:5432/paperclip ``` -Run migrations (once the migration generation issue is fixed) or use `drizzle-kit push`: +Run migrations: ```sh DATABASE_URL=postgres://paperclip:paperclip@localhost:5432/paperclip \ - npx drizzle-kit push + pnpm db:migrate ``` Start the server: @@ -100,37 +100,27 @@ postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com: ### Configure -Set `DATABASE_URL` in your `.env`: +For the application runtime, use a direct PostgreSQL connection unless the database client has explicit prepared-statement configuration for your pooling mode: ```sh -DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:6543/postgres +DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:5432/postgres ``` -For hosted deployments that use a pooled runtime URL, set -`DATABASE_MIGRATION_URL` to the direct connection URL. Paperclip uses it for -startup schema checks/migrations and plugin namespace migrations, while the app -continues to use `DATABASE_URL` for runtime queries: +If you later run the app with a pooled runtime URL, set `DATABASE_MIGRATION_URL` to the direct connection URL. Paperclip uses it for startup schema checks/migrations and plugin namespace migrations, while the app continues to use `DATABASE_URL` for runtime queries: ```sh DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:6543/postgres DATABASE_MIGRATION_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:5432/postgres ``` -If using connection pooling (port 6543), the `postgres` client must disable prepared statements. Update `packages/db/src/client.ts`: - -```ts -export function createDb(url: string) { - const sql = postgres(url, { prepare: false }); - return drizzlePg(sql, { schema }); -} -``` +Do not patch `packages/db/src/client.ts` as a setup step. Add a configuration/env switch for postgres-js prepared statements before documenting pooled Supavisor as the supported runtime path. ### Push the schema ```sh # Use the direct connection (port 5432) for schema changes DATABASE_URL=postgres://postgres.[PROJECT-REF]:[PASSWORD]@...5432/postgres \ - npx drizzle-kit push + pnpm db:migrate ``` ### Free tier limits @@ -153,6 +143,14 @@ The database mode is controlled by `DATABASE_URL`: Your Drizzle schema (`packages/db/src/schema/`) stays the same regardless of mode. +## Plugin database namespaces + +The plugin runtime tracks plugin-owned database namespaces and migrations in `plugin_database_namespaces` and `plugin_migrations`. Hosted deployments that separate runtime and migration connections should set `DATABASE_MIGRATION_URL`; plugin namespace migration work uses the migration connection when present. + +## Backups + +Paperclip supports automatic and manual database backups. See `doc/DEVELOPING.md` for the current `paperclipai db:backup` / `pnpm db:backup` commands and backup retention configuration. + ## Secret storage Paperclip stores secret metadata and versions in: diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md index 2611a0eb397..f221da3948e 100644 --- a/doc/DEVELOPING.md +++ b/doc/DEVELOPING.md @@ -43,6 +43,8 @@ This starts: `pnpm dev` and `pnpm dev:once` are now idempotent for the current repo and instance: if the matching Paperclip dev runner is already alive, Paperclip reports the existing process instead of starting a duplicate. +Issue execution may also use project execution workspace policies and workspace runtime services for per-project worktrees, preview servers, and managed dev commands. Configure those through the project workspace/runtime surfaces rather than starting long-running unmanaged processes when a task needs a reusable service. + ## Storybook The board UI Storybook keeps stories and Storybook config under `ui/storybook/` so component review files stay out of the app source routes. @@ -113,6 +115,8 @@ pnpm test:release-smoke These browser suites are intended for targeted local verification and CI, not the default agent/human test command. +For normal issue work, start with the smallest targeted check that proves the change. Reserve repo-wide typecheck/build/test runs for PR-ready handoff or changes broad enough that narrow checks do not cover the risk. + ## One-Command Local Run For a first-time local install, you can bootstrap and run in one command: @@ -194,6 +198,8 @@ For `codex_local`, Paperclip also manages a per-company Codex home under the ins If the `codex` CLI is not installed or not on `PATH`, `codex_local` agent runs fail at execution time with a clear adapter error. Quota polling uses a short-lived `codex app-server` subprocess: when `codex` cannot be spawned, that provider reports `ok: false` in aggregated quota results and the API server keeps running (it must not exit on a missing binary). +Local adapters require their corresponding CLI/session setup on the machine running Paperclip. External adapters are installed through the adapter/plugin flow and should not require hardcoded imports in `server/` or `ui/`. + ## Worktree-local Instances When developing from multiple git worktrees, do not point two Paperclip servers at the same embedded PostgreSQL data directory. diff --git a/doc/SPEC-implementation.md b/doc/SPEC-implementation.md index b8cbaa8b325..e6ceb980792 100644 --- a/doc/SPEC-implementation.md +++ b/doc/SPEC-implementation.md @@ -1,7 +1,7 @@ # Paperclip V1 Implementation Spec Status: Implementation contract for first release (V1) -Date: 2026-02-17 +Date: 2026-04-28 Audience: Product, engineering, and agent-integration authors Source inputs: `GOAL.md`, `PRODUCT.md`, `SPEC.md`, `DATABASE.md`, current monorepo code @@ -37,8 +37,9 @@ These decisions close open questions from `SPEC.md` for V1. | Visibility | Full visibility to board and all agents in same company | | Communication | Tasks + comments only (no separate chat system) | | Task ownership | Single assignee; atomic checkout required for `in_progress` transition | -| Recovery | No automatic reassignment; control-plane recovery may retry lost execution continuity once, then uses explicit recovery issues or human escalation | -| Agent adapters | Built-in `process` and `http` adapters | +| Recovery | Liveness/watchdog recovery preserves explicit ownership: retry lost execution continuity where safe, otherwise create visible recovery issues or require human escalation (see `doc/execution-semantics.md`) | +| Agent adapters | Built-in `process`, `http`, local CLI/session adapters, and OpenClaw gateway support; external adapters can also be loaded through the adapter plugin flow | +| Plugin framework | Local/self-hosted early plugin runtime is in scope; cloud marketplace and packaged public distribution remain out of scope | | Auth | Mode-dependent human auth (`local_trusted` implicit board in current code; authenticated mode uses sessions), API keys for agents | | Budget period | Monthly UTC calendar window | | Budget enforcement | Soft alerts + hard limit auto-pause | @@ -73,7 +74,7 @@ V1 implementation extends this baseline into a company-centric, governance-aware ## 5.2 Out of Scope (V1) -- Plugin framework and third-party extension SDK +- Cloud-grade plugin marketplace/distribution beyond the local/self-hosted plugin runtime - Revenue/expense accounting beyond model/token costs - Knowledge base subsystem - Public marketplace (ClipHub) @@ -123,6 +124,16 @@ Human auth tables (`users`, `sessions`, and provider-specific auth artifacts) ar - `name` text not null - `description` text null - `status` enum: `active | paused | archived` +- `pause_reason` text null +- `paused_at` timestamptz null +- `issue_prefix` text not null +- `issue_counter` int not null +- `budget_monthly_cents` int not null default 0 +- `spent_monthly_cents` int not null default 0 +- `attachment_max_bytes` int not null +- `require_board_approval_for_new_agents` boolean not null default false +- feedback sharing consent fields +- branding fields such as `brand_color` Invariant: every business record belongs to exactly one company. @@ -133,15 +144,21 @@ Invariant: every business record belongs to exactly one company. - `name` text not null - `role` text not null - `title` text null -- `status` enum: `active | paused | idle | running | error | terminated` +- `icon` text null +- `status` enum: `active | paused | idle | running | error | pending_approval | terminated` - `reports_to` uuid fk `agents.id` null - `capabilities` text null -- `adapter_type` enum: `process | http` +- `adapter_type` text; built-ins include `process`, `http`, `claude_local`, `codex_local`, `gemini_local`, `opencode_local`, `pi_local`, `cursor`, and `openclaw_gateway` - `adapter_config` jsonb not null +- `runtime_config` jsonb not null default `{}` +- `default_environment_id` uuid fk `environments.id` null - `context_mode` enum: `thin | fat` default `thin` - `budget_monthly_cents` int not null default 0 - `spent_monthly_cents` int not null default 0 +- pause fields: `pause_reason`, `paused_at` +- `permissions` jsonb not null default `{}` - `last_heartbeat_at` timestamptz null +- `metadata` jsonb null Invariants: @@ -195,6 +212,7 @@ Invariant: - `id` uuid pk - `company_id` uuid fk not null - `project_id` uuid fk `projects.id` null +- `project_workspace_id` uuid fk `project_workspaces.id` null - `goal_id` uuid fk `goals.id` null - `parent_id` uuid fk `issues.id` null - `title` text not null @@ -202,13 +220,22 @@ Invariant: - `status` enum: `backlog | todo | in_progress | in_review | done | blocked | cancelled` - `priority` enum: `critical | high | medium | low` - `assignee_agent_id` uuid fk `agents.id` null +- `assignee_user_id` text null +- checkout/execution locks: `checkout_run_id`, `execution_run_id`, `execution_agent_name_key`, `execution_locked_at` - `created_by_agent_id` uuid fk `agents.id` null - `created_by_user_id` uuid fk `users.id` null +- identifier fields: `issue_number`, `identifier` +- origin fields: `origin_kind`, `origin_id`, `origin_run_id`, `origin_fingerprint` - `request_depth` int not null default 0 - `billing_code` text null +- `assignee_adapter_overrides` jsonb null +- `execution_policy` jsonb null +- `execution_state` jsonb null +- execution workspace fields: `execution_workspace_id`, `execution_workspace_preference`, `execution_workspace_settings` - `started_at` timestamptz null - `completed_at` timestamptz null - `cancelled_at` timestamptz null +- `hidden_at` timestamptz null Invariants: @@ -261,10 +288,10 @@ Invariant: each event must attach to agent and company; rollups are aggregation, - `id` uuid pk - `company_id` uuid fk not null -- `type` enum: `hire_agent | approve_ceo_strategy` +- `type` enum: `hire_agent | approve_ceo_strategy | budget_override_required | request_board_approval` - `requested_by_agent_id` uuid fk `agents.id` null - `requested_by_user_id` uuid fk `users.id` null -- `status` enum: `pending | approved | rejected | cancelled` +- `status` enum: `pending | revision_requested | approved | rejected | cancelled` - `payload` jsonb not null - `decision_note` text null - `decided_by_user_id` uuid fk `users.id` null @@ -363,6 +390,15 @@ Operational policy: - `document_id` uuid fk not null - `key` text not null (`plan`, `design`, `notes`, etc.) +## 7.16 Current Implementation Addenda + +The current implementation includes additional V1-control-plane tables beyond the original February snapshot: + +- Issue structure and review: `issue_relations` for blockers, `labels`/`issue_labels`, `issue_thread_interactions`, `issue_approvals`, `issue_execution_decisions`, `issue_work_products`, `issue_inbox_archives`, `issue_read_states`, and issue reference mention indexes. +- Execution and workspace control: `execution_workspaces`, `project_workspaces`, `workspace_runtime_services`, `workspace_operations`, `environments`, `environment_leases`, `agent_task_sessions`, `agent_runtime_state`, `agent_wakeup_requests`, heartbeat events, and watchdog decision tables. +- Plugins and routines: `plugins`, plugin config/state/entities/jobs/logs/webhooks, plugin database namespaces/migrations, plugin company settings, and `routines`. +- Access and operations: company memberships, instance roles, principal permission grants, invites, join requests, board API keys, CLI auth challenges, budget policies/incidents, feedback exports/votes, company skills, sidebar preferences, and company logos. + ## 8. State Machines ## 8.1 Agent Status @@ -563,6 +599,17 @@ Dashboard payload must include: - `422` semantic rule violation - `500` server error +## 10.10 Current Implementation API Addenda + +The current app also exposes V1-supporting surfaces for: + +- issue thread interactions (`suggest_tasks`, `ask_user_questions`, `request_confirmation`) +- issue approvals, issue references/search, labels, read state, inbox/archive state, and work products +- execution workspaces, project workspaces, workspace runtime services, and workspace operations +- routines and scheduled/API/webhook triggers +- plugin installation, configuration, state, jobs, logs, webhooks, and plugin database namespace migration +- company import/export preview/apply, feedback export/vote routes, instance backup/config routes, invites, join requests, memberships, and permission grants + ## 11. Heartbeat and Adapter Contract ## 11.1 Adapter Interface @@ -738,13 +785,14 @@ Required UX behaviors: - Node 20+ - `DATABASE_URL` optional -- if unset, auto-use PGlite and push schema +- if unset, auto-use embedded PostgreSQL under `~/.paperclip/instances/default/db` ## 15.2 Migrations - Drizzle migrations are source of truth +- local/dev startup applies pending migrations automatically where supported +- `pnpm db:migrate` applies pending migrations manually - no destructive migration in-place for V1 upgrade path -- provide migration script from existing minimal tables to company-scoped schema ## 15.3 Logging and Audit @@ -799,6 +847,8 @@ A release candidate is blocked unless these pass: ## 18. Delivery Plan +Current implementation note: the milestones below describe the original V1 sequencing. Several systems originally framed as future work have since shipped or advanced materially, including issue documents/interactions, blockers, routines, execution workspaces, import/export portability, authenticated deployment modes, multi-user basics, and the local/self-hosted plugin runtime. + ## Milestone 1: Company Core and Auth - add `companies` and company scoping to existing entities @@ -851,7 +901,7 @@ V1 is complete only when all criteria are true: ## 20. Post-V1 Backlog (Explicitly Deferred) -- plugin architecture +- cloud-grade plugin marketplace/distribution - richer workflow-state customization per team - milestones/labels/dependency graph depth beyond V1 minimum - realtime transport optimization (SSE/WebSockets)